System with optimized multi-channel lighting control

Abstract

A system is provided that includes a lens, a camera, an illumination configuration, one or more processors, and a memory. The lens is configured to input image light from a workpiece and transmit the image light along an imaging light path. The camera is configured to receive the image light and provide an image of a workpiece. The illumination configuration includes an illumination channel configured to illuminate the workpiece to produce the image light. In various embodiments, options are provided for selecting a lighting optimization mode, the lighting optimization mode being at least one of an edge detection lighting optimization mode, a defect detection lighting optimization mode, or a point-from-focus lighting optimization mode. An illumination optimization process may be performed based on the selected illumination optimization mode, and an illumination for illuminating the workpiece is determined, and the determined illumination for the workpiece includes a setting of the illumination channel of the illumination configuration.

Description

Technical Field

[0001] The present disclosure relates to inspection systems, and more particularly to precision systems for inspecting workpieces (e.g., for metrology, defect detection, etc.). Background Art

[0002] An inspection system such as a machine vision inspection system (or simply "vision system") can be utilized to obtain an image of a workpiece for inspection. Such systems can be used for various types of applications (e.g., metrology applications for determining precise dimensional measurements of workpieces, defect detection applications, general workpiece inspection applications, etc.). Such systems typically include a computer, a camera, and an optical system. In certain configurations, a moving mechanism (e.g., a precision workpiece stage, a conveyor, etc.) can be included, which moves to allow the workpiece to be traversed and inspected. An exemplary prior art machine vision inspection system is the QUICK series of PC-based vision systems and software provided by Mitutoyo America Corporation (MAC) of Aurora, Illinois. The features and operation of the QUICK series of vision systems and software are generally described in, for example, the QVPAK 3D CNC Vision Measuring Machine User's Guide published in January 2003, which is hereby incorporated by reference in its entirety. This type of system uses a microscope-type optical system and moves the workpiece stage to provide an inspection image of the workpiece.

[0003] Such systems typically face various types of challenges in inspecting workpieces (e.g., due to variations in the type or surface of the workpiece being inspected, different types of inspection operations to be performed, changes in inspection conditions, etc.). There is a desire for a system that can provide improvements for these problems for various types of inspection operations. Summary of the Invention

[0004] This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

[0005] According to one aspect, a system is provided that includes a lens, a camera, an illumination configuration, one or more processors, and a memory. The lens (e.g., an objective lens) is configured to input image light from a workpiece, wherein the lens is configured to transmit the image light along an imaging optical path and has an optical axis. The camera is configured to receive the image light transmitted along the imaging optical path and provide an image of the workpiece. The illumination configuration includes an illumination channel configured to illuminate the workpiece to generate the image light.

[0006] The memory is coupled to the one or more processors and stores program instructions that, when executed by the one or more processors, cause the one or more processors to at least: provide a representation of multiple sets of illumination channels in a display area; determine that a set of illumination channels has been selected, wherein the selected set of illumination channels includes a plurality of illumination channels; and display a current illumination setting for the selected set of illumination channels, wherein an adjustment to the illumination setting for a set of illumination channels is applied to all of the illumination channels in the set.

[0007] According to another aspect, a method of operating the system is provided. The method includes: providing a representation of multiple sets of illumination channels in a display area; determining that a set of illumination channels has been selected, wherein the selected set of illumination channels includes a plurality of illumination channels; and displaying a current illumination setting for the selected set of illumination channels, wherein an adjustment to the illumination setting for a set of illumination channels is applied to all of the illumination channels in the set.

[0008] According to another aspect, the system is provided as including the lens, the camera, the illumination configuration, the one or more processors, and the memory. The memory is coupled to the one or more processors and stores program instructions that, when executed by the one or more processors, cause the one or more processors to at least: provide an option for selecting an illumination optimization mode, the illumination optimization mode being at least one of an edge detection illumination optimization mode, a defect detection illumination optimization mode, or a points from focus illumination optimization mode; receive a selection of the illumination optimization mode; and perform an illumination optimization process based on the selected illumination optimization mode, wherein the illumination optimization process determines the illumination for illuminating the workpiece, and the determined illumination includes settings of the illumination channels of the illumination configuration. In various embodiments, the option may be used to select the illumination optimization mode from a set of illumination optimization modes.

[0009] According to another aspect, a method for operating the system to perform an illumination optimization process is provided. The method includes: providing an option for selecting an illumination optimization mode, the illumination optimization mode being at least one of an edge detection illumination optimization mode, a defect detection illumination optimization mode, or a points from focus illumination optimization mode; receiving a selection of the illumination optimization mode; and performing an illumination optimization process based on the selected illumination optimization mode, wherein the illumination optimization process determines the illumination for illuminating the workpiece, and the determined illumination includes settings of the illumination channels of the illumination configuration. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1is a diagram showing various typical components of a general precision machine vision inspection system;

[0011] Figure 2 is related to Figure 1 a block diagram of the control system part and the vision component part of a machine vision inspection system similar to the machine vision inspection system;

[0012] Figures 3A to 3C is a diagram of a graphical user interface for lighting control;

[0013] Figure 4 shows an example of different groups of lighting channels;

[0014] Figure 5 is a diagram showing an example of the intersection part between two groups of lighting channels;

[0015] Figure 6 shows an implementation of a technique for representing negative color weights in an image of a workpiece;

[0016] Figure 7 is a flowchart showing a routine for operating a system with multiple groups of lighting channels;

[0017] Figure 8 is a diagram of an exemplary point tool laid on the first edge of a part of a workpiece in an image;

[0018] Figure 9 is a diagram of an exemplary frame tool laid on the first edge of a part of a workpiece in an image;

[0019] Figure 10 is a diagram showing an exemplary method for determining the first edge position based on a set of pixel values;

[0020] Figure 11 is a diagram showing performing metrology operations on workpiece features in an image of a workpiece using one or more video tools;

[0021] Figure 12 is a diagram showing performing metrology operations on workpiece features in an image of a workpiece using one or more video tools;

[0022] Figures 13A to 13C shows images obtained using different settings of lighting channels, which may affect the visibility of different workpiece features (such as defects);

[0023] Figure 14A and Figure 14B is a diagram showing a contrast focus curve, which may be generated by lighting characteristics in a corresponding region of interest and can be used for the focus picking process;

[0024] Figures 15A to 15IA diagram showing the determination of illumination optimization for different illumination optimization modes;

[0025] Figure 16 A flowchart of a routine for determining illumination optimization for a workpiece (e.g., a new workpiece);

[0026] Figure 17 A flowchart of a routine for providing illumination optimization for a workpiece (e.g., an existing workpiece);

[0027] Figure 18 A diagram representing options available in a system related to illumination optimization; and

[0028] Figure 19 A flowchart showing a routine for performing an illumination optimization process. Detailed Description

[0029] Figure 1 A diagram showing various typical components of a general precision machine vision inspection system 10. The machine vision inspection system 10 includes a vision measuring machine 12, which is operably connected to exchange data and control signals with a control computer system 14. The control computer system 14 is also operably connected to exchange data and control signals with a monitor or display 16, a printer 18, a joystick 22, a keyboard 24, and a mouse 26. The monitor or display 16 can display a user interface suitable for controlling and / or programming the operation of the machine vision inspection system 10. It should be understood that in various embodiments, a touchscreen tablet computer, etc., can replace and / or redundantly provide the functions of any one or all of the elements 14, 16, 22, 24, and 26.

[0030] Those skilled in the art should understand that the control computer system 14 can generally be implemented using any suitable computing system or device, including a distributed or networked computing environment, etc. Such a computing system or device can include one or more general-purpose or special-purpose processors (e.g., non-custom or custom devices) that execute software to perform the functions described herein. The software can be stored in a memory, such as random access memory (RAM), read-only memory (ROM), flash memory, etc., or a combination of such components. The software can also be stored in one or more storage devices, such as optical discs, flash memory devices, or any other type of non-volatile storage medium for storing data. The software can include one or more program modules, and the one or more program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. In a distributed computing environment, the functions of the program modules can be combined or distributed across multiple computing systems or devices and accessed via service calls configured in a wired or wireless manner.

[0031] The vision measuring machine 12 includes a movable workpiece table 32 and an optical imaging system 34 that may include a zoom lens or interchangeable objective lenses. The zoom lens or interchangeable objective lenses typically provide various magnifications for the images provided by the oil optical imaging system 34. Various embodiments of the machine vision inspection system 10 are also described in commonly assigned U.S. Patent Nos. 7,454,053, 7,324,682, 8,111,905, and 8,111,938, each of which is hereby incorporated by reference in its entirety.

[0032] As will be described in more detail below, for various applications (e.g., including the use of variable focal length (VFL) lenses such as tunable acoustic gradient (TAG) lenses), it may be desirable to provide illumination from multiple directions, such as to better achieve imaging of non-planar workpieces (e.g., workpieces including at least some surfaces that are not orthogonal to the system optical axis (such as surfaces that may be angled / tilted and / or parallel, etc. relative to the optical axis)). As will be described in more detail below, the illumination configuration (e.g., including illumination configuration 230, etc.) may include illumination channels that provide such illumination from multiple directions.

[0033] Figure 2 is a block diagram of a control system portion 120 and a vision component portion 200 of a machine vision inspection system 100 that is similar to the Figure 1 machine vision inspection system described herein and includes certain features disclosed herein. As will be described in more detail below, the control system portion 120 is used to control the vision component portion 200. The vision component portion 200 includes an optical component portion 205, light sources 220, 230, 240, and a workpiece table 210 having a central transparent portion 212. The workpiece table 210 is controllably movable along the X-axis and Y-axis, which are located in a plane generally parallel to the surface of the workpiece table on which the workpiece 20 may be placed.

[0034] The optical component section 205 includes a camera system 260 (i.e., including a camera), an interchangeable objective lens 250, and a variable focal length (VFL) lens 270. In various embodiments, the VFL lens 270 can be an adjustable acoustic gradient (“TAG” or “TAGLENS”) lens that utilizes sound waves in a fluid medium to create a lens effect. Sound waves can be generated by applying an electric field at a resonant frequency to a piezoelectric tube surrounding the fluid medium to create a time-varying density and refractive index distribution in the fluid of the lens, which modulates the optical power of the lens and thus modulates the focal length (or effective focal position) of its optical system. The TAG lens can be used to scan a range of focal lengths (i.e., periodically modulate its optical power) at high speed and periodically at a resonant frequency greater than 30 kHz, or greater than 70 kHz, or greater than 100 kHz, or greater than 400 kHz, up to, for example, 1.0 MHz. Such a lens can be understood in more detail through the teachings of the article “High speed varifocal imaging with a tunable acoustic gradient index of refraction lens” (Optics Letters, Vol. 33, No. 18, September 15, 2008), which is hereby incorporated herein by reference in its entirety. The TAG (aka TAGLENS) lens and associated controllable signal generator are available from, for example, Mitutoyo Corporation of Kanagawa, Japan. As a specific example, certain TAG lenses are capable of periodic modulation with a modulation frequency up to 1.0 MHz. The operating principles and various aspects of the application of the TAG lens are described in more detail in U.S. Patent Nos. 10,178,321, 10,101,572, 9,930,243, 9,736,355, 9,726,876, 9,143,674, 8,194,307, and 7,627,162, which are hereby incorporated herein by reference in their entirety.

[0035] In various embodiments, the optical component section 205 can also include a turret lens assembly 223 having lenses 226 and 228. As an alternative to the turret lens assembly, in various embodiments, a fixed or manually interchangeable magnification-changing lens, or a zoom lens configuration, etc., can be included. In various embodiments, the interchangeable objective lens 250 can be selected from a set of fixed magnification objective lenses included as part of a variable magnification lens section (e.g., a set of objective lenses corresponding to magnifications such as 0.5x, 1x, 2x or 2.5x, 5x, 7.5x, 10x, 20x or 25x, 50x, 100x, etc.).

[0036] By using a controllable motor 294, the optical component section 205 can be controllably moved along a Z-axis that is substantially orthogonal to the X-axis and the Y-axis. The controllable motor drives an actuator to move the optical component section 205 along the Z-axis to change the focus of the image of the workpiece 20. The controllable motor 294 is connected to the input / output interface 130 via a signal line 296. As will be described in more detail below, in order to change the focus of the image within a small range, or as an alternative to moving the optical component section 205, the VFL (TAG) lens 270 can be controlled via a signal line 234' by a lens control interface 134 to periodically modulate the optical power of the VFL lens 270, thereby modulating the effective focus position of the optical component section 205. The lens control interface 134 can include a VFL lens controller 180, as will be described in more detail below. The workpiece 20 can be placed on the workpiece stage 210. The workpiece stage 210 can be controlled to move relative to the optical component section 205 such that the field of view of the interchangeable objective lens 250 moves between positions on the workpiece 20 and / or between multiple workpieces 20, and so on.

[0037] One or more of the workpiece stage light source 220, the illumination configuration 230, and the coaxial light source 240 can respectively emit source light 222, 232, and / or 242 to illuminate one workpiece 20 or multiple workpieces 20. In various exemplary embodiments, pulsed (e.g., stroboscopic) illumination can be used. For example, during image exposure, the illumination configuration 230 can emit stroboscopic source light 232 toward a central volume CV where at least a portion of the workpiece 20 is located. In another example, during image exposure, the coaxial light source 240 can emit stroboscopic source light 242 along a path that includes a beam splitter 290 (e.g., a partial mirror / reflection surface). The source light 232, 242 is reflected as image light 255, and the image light for imaging passes through the interchangeable objective lens 250, the turret lens assembly 223, and the VFL (TAG) lens 270 and is captured by the camera system 260. The workpiece image exposure including the image of the workpiece 20 is captured by the camera system 260 and output to the control system section 120 via a signal line 262.

[0038] In various embodiments, the illumination configuration 230 includes a plurality of illumination channels configured to illuminate the workpiece 20 to generate image light 255, where each illumination channel is configured to direct light toward the central volume CV (e.g., where at least a portion of the workpiece 20 can be located). As described above, the objective lens 250 is configured to input the image light 255 from the workpiece 20, where the objective lens 250 is configured to transmit the image light along the imaging optical path and has an optical axis OA. In Figure 2In the example of, the objective lens 250 transmits the image light along the imaging optical path passing through the VFL lens 270 to the camera 260. The camera 260 is configured to receive the image light transmitted along the imaging optical path and provide an image of the workpiece 20. As will be described in more detail below, the focal position corresponding to the focus of the image is configured to be variable within the focal range along the optical axis. In various embodiments, the illumination configuration 230 is controlled through the illumination control interface 133 (e.g., including a light controller portion for controlling the illumination configuration 230, such as the light controller portion 133n).

[0039] Various light sources (e.g., light sources 220, 230, 240) can be characterized as including illumination channels and can be connected to the illumination control interface 133 of the control system portion 120 through associated signal lines (e.g., bus 221, 231, 241 respectively). The control system portion 120 can control the turret lens assembly 223 to rotate along the axis 224 to select a turret lens through the signal line or bus 223' to change the image magnification.

[0040] As Figure 2 shown, in various exemplary embodiments, the control system portion 120 includes a controller 125 (e.g., including one or more processors or operating as part thereof), an input / output interface 130, a memory 140, a workpiece program generator and actuator 170, and a power supply portion 190. Each of these components and additional components described below can be interconnected through one or more data / control buses and / or application programming interfaces, or through direct connections between the respective elements. The input / output interface 130 includes an imaging control interface 131, a motion control interface 132, an illumination control interface 133, and a lens control interface 134. The lens control interface 134 can include or be connected to a VFL lens controller 180, which includes circuits and / or routines for controlling various image exposures synchronized with the periodic focal position modulation provided by the VFL (TAG) lens 270. In some embodiments, the lens control interface 134 and the VFL lens controller 180 can be combined and / or indistinguishable.

[0041] The illumination control interface 133 may include illumination control elements 133a - 133n that control, for example, the selection, power, on / off switching, and pulse / strobe timing (if applicable) of respective corresponding light sources / illumination channels of the machine vision inspection system 100. In various embodiments, examples of strobe illumination may be considered a type of pulsed illumination as the term is used herein. In some embodiments, the illumination control interface 133 may provide a pulse / strobe timing signal to one or more of the illumination control elements 133a - 133n such that they provide image exposure pulse / strobe timing (e.g., which may be synchronized with a desired phase time modulated by the VFL lens focus position according to certain stored calibration data).

[0042] The memory 140 may include an image file memory portion 141, an edge detection memory portion 140ed, a workpiece program memory portion 142 that may include one or more part programs, etc., and a video tool portion 143. The video tool portion 143 includes a video tool portion 143a and other video tool portions (e.g., 143n) that determine the GUI, image processing operations, etc. of each corresponding video tool, and a region of interest (ROI) generator 143roi that supports automatic, semi - automatic, and / or manual operations for defining various ROIs that can be operated on in the various video tools included in the video tool portion 143. Examples of the operation of such video tools for locating edge features and performing other workpiece feature inspection operations are described in more detail in certain previously incorporated references and in U.S. Patent No. 7,627,162, which is hereby incorporated herein by reference in its entirety.

[0043] The video tool section 143 includes an autofocus video tool 143af that determines a GUI for focus height (i.e., effective focus position (Z coordinate / Z height)) measurement operations, image processing operations, and the like. In various embodiments, the autofocus video tool 143af may further include a high-speed focus height tool that can be used to measure the focus height at high speed using the hardware shown in FIG. 3, as described in more detail in U.S. Patent No. 9,143,674 incorporated hereinabove. In various embodiments, the high-speed focus height tool may be a special mode of the autofocus video tool 143af, otherwise it may operate according to the conventional methods of the autofocus video tool, or the operation of the autofocus video tool 143af may include only the operation of the high-speed focus height tool. According to known methods, high-speed autofocus and / or focus position of an image region or region of interest can be determined based on the analysis of an image to determine the corresponding focus characteristic values (e.g., quantitative contrast metric values and / or quantitative focus metric values) of each region. For example, such methods are disclosed in U.S. Patent Nos. 8,111,905, 7,570,795, and 7,030,351, which are hereby incorporated herein by reference in their entirety.

[0044] In the context of the present disclosure, and as is known to those of ordinary skill in the art, the term "video tool" generally refers to a relatively complex set of automated or programmed operations that a machine vision user can implement through a relatively simple user interface. For example, a video tool may include a complex set of pre-programmed image processing operations and calculations that can be applied and customized in a particular instance by adjusting several variables or parameters that control the operations and calculations. In addition to the underlying operations and calculations, a video tool also includes a user interface that allows the user to adjust these parameters for a particular instance of the video tool. It is noted that the visible user interface functionality is sometimes referred to as the video tool, with the underlying operations being implied.

[0045] One or more display devices 136 (e.g., Figure 1 the display 16) and one or more input devices 138 (e.g., Figure 1 the joystick 22, keyboard 24, and mouse 26) may be connected to the input / output interface 130. The display device 136 and the input device 138 can be used to display a user interface that may include various graphical user interface (GUI) features that can be used to perform inspection / measurement operations, and / or create and / or modify part programs, view images captured by the camera system 260, and / or directly control the vision component section 200.

[0046] In various exemplary embodiments, when a user creates a part program for a workpiece 20 using the machine vision inspection system 100, the user generates part program instructions by operating the machine vision inspection system 100 in a learning mode to provide a desired image acquisition training sequence. For example, the training sequence can include locating specific workpiece features of a representative workpiece in the field of view (FOV), setting the lighting level, focusing or autofocusing, acquiring an image, and providing an inspection training sequence applied to the image (e.g., an instance of using one of the vision tools on the workpiece feature). The learning mode operation causes the sequence to be captured or recorded and converted into corresponding part program instructions. When the part program is executed, these instructions will cause the machine vision inspection system to reproduce the trained image acquisition and cause the inspection operation to automatically inspect specific workpiece features (i.e., corresponding features at corresponding locations) on one or more in-run workpieces, where the specific workpiece features match the representative workpiece used when creating the part program.

[0047] In various embodiments, the imaging optical path OPATH (also referred to herein as the workpiece imaging optical path) includes various optical components arranged along a path that conveys image light 255 from the workpiece 20 to the camera 260. The image light generally propagates in the direction of its optical axis OA. In Figure 2 the illustrated embodiment, all optical axes OA are aligned. However, it should be understood that this embodiment is only intended to be exemplary and not restrictive. More generally, the imaging optical path OPATH can include mirrors and / or other optical elements and can take any form operable to image the workpiece 20 using a camera (e.g., camera 260) according to known principles. In the illustrated embodiment, the imaging optical path OPATH includes a VFL lens 270 (which can be included in a 4f imaging configuration) and is at least partially used to image the workpiece 20 during workpiece image exposure. In various embodiments, the Z height (e.g., the height of a surface point on the workpiece) can correspond to and / or alternatively be referred to as the Z coordinate and / or the focus position, and these terms can be used interchangeably in certain instances herein.

[0048] In various embodiments, the VFL lens controller 180 can control the drive signal of the VFL lens 270 to periodically modulate the optical power of the VFL lens 270 within an optical power range that occurs at corresponding phase timings within a periodic modulation. Alternatively, the controllable motor 294 can be utilized to move the optical component portion 205 along the Z-axis to change the effective focal position of the optical component portion 205. In either case, the camera 260 (e.g., including an image sensor) receives the light transmitted along the imaging optical path OPATH during an image exposure and provides a corresponding camera image. The objective lens 250 inputs the image light from the workpiece 20 during the image exposure and transmits the image light along the imaging optical path OPATH passing through the VFL lens 270 to the camera 260 during the image exposure to provide an image of the workpiece in the corresponding camera image. In embodiments utilizing the VFL lens 270, the effective focal position in front of the objective lens 250 during the image exposure corresponds to the optical power of the VFL lens 270 during that image exposure. The illumination control interface 133 can be configured to control the image exposure timing for the camera image.

[0049] In various embodiments, the illumination configuration LC (e.g., including the illumination configuration 230, the light source / illumination channel 240, etc.) is configured to emit source light 232 / 242 (e.g., pulsed / stroboscopic illumination in some cases) towards a central volume CV that includes at least a portion of the workpiece 20. The objective lens 250 receives the image light 255 (e.g., workpiece light) focused at an effective focal position near the workpiece 20 and outputs the image light 255 to propagate along the optical path OPATH.

[0050] In various embodiments, Figure 2 the illumination configuration 230 includes a plurality of illumination channels configured to illuminate the workpiece 20 to generate the image light 255, wherein each illumination channel is configured to direct the light 232 towards the central volume CV (e.g., where at least a portion of the workpiece 220 can be located). The objective lens 250 is configured to input the image light 255 from the workpiece 20 and is configured to transmit the image light 255 along the imaging optical path in the OPATH, wherein the objective lens 250 has an optical axis OA. The camera 260 is configured to receive the image light 255 transmitted along the imaging optical path OPATH and provide an image of the workpiece 20, wherein the focal position corresponding to the focus of the image is configured to be variable within a focal range along the optical axis OA.

[0051] In various embodiments, the VFL lens controller 180 is configured to control the VFL lens to periodically modulate the optical power of the VFL lens 270 within a certain optical power range at an operating frequency so as to change the focal position of the system at multiple positions within the focal range. Alternatively, the controllable motor 294 can be utilized to move the optical component portion 205 along the Z-axis to change the effective focal position of the optical component portion 205. In either case, in various embodiments, a camera 260 can be utilized to acquire an image stack that includes multiple images of the workpiece 20 (e.g., as described in more detail below with respect to Figure 14A and Figure 14B ), where each image of the image stack corresponds to a different focal position along the optical axis OA (e.g., because each image corresponds to a different Z-height in the example of Figure 2 ). The focus curve data can be determined based at least in part on the analysis of the images of the image stack, where the focus curve data indicates the three-dimensional positions of multiple surface points on the workpiece 20 (e.g., as part of a focus picking process performed by the focus picking section 146, etc.).

[0052] In some embodiments, data or signals from the camera 260 can be used to determine when an imaging surface region (e.g., including one or more surface points of the workpiece 20) is at an effective focal position. For example, known "maximum contrast" or "best focus image" analysis can be used to analyze a set of images (such as a portion of the image stack) acquired by the camera 260 at different effective focal positions (Z-heights) to determine when the imaging surface region of the workpiece 20 is located at the corresponding effective focal position (Z-height). However, more generally, any other suitable known image focus detection configuration can be used. In any case, in embodiments using the VFL lens 270, the images acquired during the periodic modulation of the effective focal position of the VFL lens 270 (during the scanning of multiple effective focal positions) can be used to determine the image and / or image timing at which a target feature (e.g., including one or more surface points of the workpiece) is in best focus.

[0053] In various embodiments, the illumination control interface 133 controls the image exposure time of the system 100 (e.g., with respect to the phase timing of the periodically modulated effective focus position). More specifically, during image exposure, the illumination control interface 133 can use the effective focus position (Z-height) calibration data and control the illumination configuration LC (e.g., including illumination configuration 230, etc.) to pulse / strobe at corresponding times. For example, the illumination control interface 133 can control the illumination configuration LC to pulse / strobe at corresponding phase timings within the standard imaging resonance frequency period of the VFL lens 270 in order to acquire an image having a specific effective focus position within the scan (periodically modulated) range of the VFL lens 270. It should be understood that the operation of the illumination control interface 133 and other features and elements outlined above can be implemented to control workpiece image acquisition.

[0054] As will be described in more detail below, certain illumination control and illumination optimization features and processes can be provided and implemented in accordance with the principles described herein. In various embodiments, such features and processes can be implemented and / or performed by certain parts of the control system section 120 of Figure 2 For example, in various embodiments, the illumination control interface 133 and / or the illumination optimization section 133lo or 147 can be utilized to implement such features. In various embodiments, such features and processes can include various illumination channels of the system 100, for which corresponding illumination configurations LC can be designated to include specific illumination channels of the represented and controlled system, such as the illumination channels of the illumination configuration 230 and / or the axial illumination channel 240, etc. In addition, the edge detection section 144, the defect detection section 145, and the focus point extraction section 146 can be respectively used to perform various edge detection, defect detection, and / or focus point extraction processes, operations, and / or features, for which more details will be described below.

[0055] In various embodiments, the defect detection section 145 performs various defect detection operations, as will be described in more detail below. In various embodiments, the defect detection section 145 utilizes a model that requires training data (e.g., training images). For example, a set of training images captured using specific imaging, illumination, and workpiece conditions can be used to train the defect detection section 145. In various exemplary embodiments, the model can be a supervised model. In various embodiments, the defect detection section 145 processes image data corresponding to labeled images of defects (e.g., labeled by a user and / or an automated process) to train a classification model.

[0056] In various exemplary embodiments, the defect detection section 145 may implement a defect detection process performed in conjunction with a metrology process (e.g., such as at least partially performed by the edge detection section 144 and / or the take points from focus section 146). In various embodiments, it may be desirable to include the defect detection section 145 in a system configured to perform a metrology process, since the system may generate image data that is input into the defect detection process implemented by the defect detection section 145. Thus, a single machine being configured to perform both a metrology process and a defect detection process can be more advantageous than traditional metrology systems. For example, if a defect is detected in a workpiece during the execution of the defect detection process, there may be no reason to perform a metrology process on the workpiece, which can save time. More specifically, if a defect is detected in a workpiece during the execution of the defect detection process, it may not be necessary to measure a clearly defective part. Therefore, it may be advantageous to perform the defect detection process before starting the metrology process.

[0057] In addition, certain defects may require further metrology or inspection to determine additional defect parameters. For example, a 2D image can quickly identify possible defects, and based thereon certain metrology operations can be performed (e.g., determining the exact XY position of the defect and / or the approximate XY size / area of the defect, such as at least partially performed by the edge detection section 144). If the 3D nature of a potential defect is important, the defect detection section 145 may cause additional processing (e.g., metrology operations) to be performed to determine whether the potential defect is an actual defect. For example, if a scratch on the surface of a workpiece must be deeper than a particular threshold to be considered a defect, the defect detection section 145 may cause a more time-consuming 3D point cloud of the affected area to be obtained (e.g., at least partially utilizing the take points from focus section 146) to see if the depth of the scratch is sufficient to cause the part to be rejected. As will be described in more detail below, in various embodiments, different illumination optimizations can correspond to different types of processes and are determined therefor (e.g., such as the processes performed by each of the edge detection section 144, the defect detection section 145, and the take points from focus section 146).

[0058] In various embodiments, a lighting channel as described herein can be considered the lowest independently controllable element of a light source configuration (e.g., RGB light-emitting diodes (LEDs)). A set of lighting channels can be a collection of lighting channels, such as predefined or user-defined, and can be controlled as a group to share the same settings (e.g., RGB settings). According to various examples herein, a set of lighting channels can correspond to certain shapes, such as circular, pie wedge, asymmetric shapes, etc. These shapes can be considered to correspond to an appearance on a lighting configuration surface. A lighting layout can be considered a collection of multiple sets of lighting channels and / or individual lighting channels. In some embodiments, a lighting layout can be considered a partitioning of some or all of the lighting channels of a lighting configuration into an arrangement (e.g., which can be saved or loaded into a controller of a graphical user interface and used for a lighting optimization process, etc.). As a simplified example, a particular lighting layout can include a collection of circular sets of lighting channels that together include all of the lighting channels of a lighting configuration.

[0059] In various embodiments, various graphical user interface elements and principles are described herein for improving the process of manual, semi-automatic, or automatic lighting adjustment in a system having many (e.g., dozens of) independently controllable lighting channels. An example of a lighting configuration having such characteristics is a ceiling light (e.g., such as having fifty lighting channels). According to certain examples herein, the lighting channels are displayed in a graphical user interface in a geometric arrangement corresponding to the physical arrangement of the lighting channels (e.g., as shown in display area 360 of the light source display of Figure 3A and Figure 3C ), and for the geometric arrangement, the graphical user interface representation can be manipulated and adjusted (e.g., by a user).

[0060] According to the principles described herein, a set of lighting channels can include multiple lighting channels for which each set of lighting channels can be adjusted as a single entity. By including lighting channels in a group and adjusting each set of lighting channels as a single entity, certain processes (such as manual, semi-automatic, or automatic lighting optimization) can be made faster and / or easier by reducing the effective number of entities to be adjusted. In some embodiments, each lighting channel can be included in a single group, or in some embodiments, a lighting channel can be assigned to multiple groups. In various embodiments, when a certain group is determined (e.g., created, selected, etc.), the settings of the group (e.g., current RGB settings) can be displayed in a display area of the graphical user interface (e.g., such as in display area 350 of Figure 3A and Figure 3C ).

[0061] In various embodiments, multiple sets of lighting channels and / or lighting layouts can be stored and recalled, including in some embodiments predefined lighting layouts (e.g., pie segments, rings, etc.), and parameters (e.g., the number of pie segments or rings, etc.) can be adjusted quickly. In various embodiments, certain controls can be provided, such as rotation, expansion, or contraction of elements (e.g., expansion or contraction of a lighting ring), control of certain lighting directions (e.g., using a trackball-like control), or inversion of certain portions (e.g., inversion of the current lighting mode). Such changes and adjustments can be achieved by coordinating changes to the settings of different lighting channels, such as in response to an operation performed in the graphical user interface 300 (e.g., as Figures 3A to 3C shown).

[0062] Figures 3A to 3C FIG. is a diagram of a graphical user interface 300 for lighting control of a system (e.g., such as can be provided on the display 16 of Figure 1 and / or the display device 136 of Figure 2 etc.). Figure 3A includes the complete graphical user interface 300, while Figure 3B and Figure 3C include the left and right halves respectively, with additional labels. The graphical user interface 300 includes display regions 310, 320, 330, 340, 350, 360, 370, 380, 390, and 395. The display region 310 includes a "layout list" and includes selection regions 311 - 316 (i.e., as Figure 3B shown), for which each selection region includes a label indicator in column 310A and a graphical indicator in column 310B. As indicated, the selection regions 311 - 316 currently correspond to layouts 1 - 6, as indicated by the label indicators in column 310A. In column 310B, graphical indicators for each layout are provided, which can provide a graphical representation of the layout. The selection region 310C to the right of the display region 310 includes a scroll region that can be used to scroll to additional layouts not currently shown in the display region 310.

[0063] As a specific example, in the illustrated embodiment, the layout list may include a total of 17 layouts, for which the user can scroll between layouts 1 - 17. Below and adjacent to the display area 310, selection areas 317, 318, and 319 are also included. Selection area 317 provides an option to create a new empty layout. Selection area 318 provides an option to delete the selected layout. Selection area 319 provides an additional quick - view scroll bar that can be used to scroll through the layouts. In selection area 319, it is indicated that the quick - view range spans from layout 1 to layout 17, and for these layouts, the current scroll indicator is at the start of the range, corresponding to layout 1 (i.e., also indicated by the number "1" to the right of the scroll bar). The position of the scroll indicator also coincides with the selected layout 1 in selection area 311, as indicated by the selection indicator SI (i.e., the selection indicator SI in selection area 311 corresponds to the highlighting of the selected layout).

[0064] The display area 320 includes a "group list" and includes selection areas 321 - 326 (i.e., as Figure 3B shown). Column 320A includes label indicators for the currently - displayed set of groups (e.g., the set of groups for the selected layout 1), and column 320B includes a scroll area through which the user can scroll to display different groups in the display area 320. In Figure 3B the illustration, selection areas 321 - 326 are currently shown as corresponding to groups 1 - 6. As a specific example, in the illustrated embodiment, the group list for layout 1 may include twelve groups, including groups 1 - 12. In the illustrated embodiment, the group list in the display area 320 indicates the current selection corresponding to selection area 323, such as corresponding to group 3 (i.e., the current selection for layout 1).

[0065] Below and adjacent to the display area 320 are selection areas 327, 328, and 329. Selection area 327 provides an option to save the current layout as a new layout. Selection area 328 provides an option to delete the selected group. Selection area 329 provides a quick - view scroll option for scrolling between different groups. As indicated by selection area 329, the quick - view range is from group 1 (i.e., indicated at the left - most) to group 12 (i.e., indicated at the end of the right - most range). The current position of the scroll indicator (i.e., having moved approximately 25% within the available range) is at group 3 (i.e., also indicated by the number "3" to the right of the scroll bar). The position of the scroll indicator also coincides with the selected group 3 in selection area 323, as indicated by the selection indicator SI, which is similar to the selection indicator SI on the display area 310.

[0066] The display area 330 includes "layout parameters", and includes selection areas 331 - 333, and may provide additional selection areas for the display area. In the current state, only the selection area 331 is allocated (e.g., dynamically), and is currently indicated in column 330B for multiple segments (i.e., as the first layout parameter of the currently selected layout 1 indicated in the display area 310). The selection area 331 in column 330A indicates that the current number of segments is set to five segments, and includes up and down arrows that can be used to increase or decrease (e.g., in this case, to increase or decrease the number of segments).

[0067] The display area 340 includes a selection area 341 for "channel setting modifier" and a selection area 342 for "shared channel setting behavior". The selection area 341 includes selection areas 341A - 341C, and the selection area 342 includes selection areas 342A - 342E. Each corresponding selection option contains a radio button. In the selection area 341, the selection area 341A is for the "direct" option, the selection area 341B is for the "invert to 0" option, and the selection area 341C is for the "fully invert" option. In the selection area 342, the selection area 342A is for the "average of all groups" option, the selection area 342B is for the "sum of all groups" option, the selection area 342C is for the "product of all groups" option, the selection area 342D is for the "minimum of all groups" option, and the selection area 342E is for the "maximum of all groups" option.

[0068] The display area 350 is for "main settings", and includes selection areas 351 - 354. The selection area 351 is for color setting, including a selection area 351A for red light setting, a selection area 351B for green light setting, and a selection area 351C for blue light setting. As indicated, the color settings can be set within a value range of -1 to +1. Settings within the range between -1 and 0 indicate negative settings of the corresponding light color, and settings between 0 and +1 indicate positive settings of the corresponding light color. The numeric indicators in column 351D indicate the corresponding values for the settings of different light colors, such as the value of the red light setting is -0.50, the value of the green light setting is 0.30, and the value of the blue light setting is 0.15, and these values are also indicated by the positions of the sliders along the range between -1 and +1 in the selection areas 351A, 351B, and 351C.

[0069] The selection area 352 includes a selection option 352A corresponding to an upward-pointing arrow for increasing the current brightness setting, and a selection option 352B corresponding to a downward-pointing arrow for decreasing the current brightness setting. In the selection area 353, the selection option 353A corresponding to a backward-pointing arrow corresponds to an option for undoing a change, and the selection option 353B corresponding to a forward-pointing arrow corresponds to an option for redoing a change (e.g., a change that has been undone by selecting the selection option 353A). The selection area 354 provides an option for making a gang selection.

[0070] The display area 360 includes a "light source display" and includes selection areas 361 - 365. The selection area 361 corresponds to a display area in which a representation of an illumination channel LCH of an illumination configuration LC is displayed and can be selected or deselected. In one embodiment, the representation can be an illumination configuration LC for which two illumination channel rings can correspond to Figure 2 the illumination configuration 230, and a central axial illumination channel corresponds to the light source 240. The selection area 362 corresponds to an "expand" option, the selection area 363 corresponds to a "rotate" option, the selection area 364 corresponds to a "direct" option, and the selection area 365 corresponds to a "forced exclusive group" option. The selection area 366 is located below and adjacent to the display area 360 and includes a selection option 366A for "group display" and a selection option 366B for "settings display", where each selection option is provided with a radio button.

[0071] The display area 370 has certain similarities to the display area 350 and should be understood to operate in a similar manner unless otherwise specified below. The display area 370 includes selection areas 371 - 376. The selection area 371 includes an option for "override selected settings". The selection area 372 includes an option for "other channels off". Similar to the selection area 351, the selection area 373 includes a selection option 373A for red light settings, a selection option 373B for green light settings, and a selection option 373C for blue light settings. The color settings can be made within a value range between -1 and +1. In column 373D, indicators of the corresponding values of the settings are provided, indicating that the value of the red light setting is -0.50, the value of the green light setting is 0.30, and the value of the blue light setting is 0.15. These setting values are also indicated by the position of a slider along each corresponding range (i.e., between -1 and +1), and the user can adjust these setting values to adjust the light settings of different colors in the selection areas 373A, 373B, and 373C.

[0072] In selection region 374, selection option 374A corresponding to the upward-pointing arrow is for increasing the brightness setting, and selection option 374B corresponding to the downward-pointing arrow is for decreasing the current brightness setting. Selection region 375 includes selection option 375A corresponding to the backward-pointing arrow, for undoing changes, and selection option 375B corresponding to the forward-pointing arrow, for redoing changes. Selection region 376 provides an option for "group" selection.

[0073] Display region 380 includes a "workpiece display" (e.g., displaying an image of workpiece 20 illuminated by the current lighting setting). In the illustrated embodiment, display region 380 includes an upper display region 380A and a lower display region 380B. The upper display region 380A shows a portion of the workpiece illuminated by the current lighting setting, and for that workpiece portion, according to example "A", edge EDG on the workpiece is being detected by edge detection tool EDT. The position of edge EDG approximately corresponds to the dashed edge indicator in edge detection tool EDT. In the lower display region 380B, according to example "B", defect DFT on the workpiece is indicated as being detected by defect detection tool DDT.

[0074] Display region 390 includes selection options 391A - 391D. Selection option 391A is for optimizing edge detection, selection option 391B is for optimizing defect detection, selection region 391C is for optimizing taking points from the focus, and selection option 391D is for optimizing an unspecified process of the lighting. Such options can be referred to as lighting optimization modes, as will be described in more detail below. For all selection options 391A - 391D in display region 390, radio buttons are provided for corresponding selection. Display region 395 includes additional selection options 396A - 396D. Selection option 396A is for selecting to optimize brightness, selection option 396B is for selecting to optimize the red light setting, selection option 396C is for selecting to optimize the green light setting, and selection option 396D is for selecting to optimize the blue light setting. For all selection options 396A - 396D in display region 395, checkboxes are provided for corresponding selection. The operation of each display region 310 - 395 will be described in more detail below.

[0075] In various embodiments, each layout in the layout list of the display area 310 includes at least one set of illumination channels (e.g., such as can be created, added, and / or configured, etc.), such as through user interaction in the light source display of the display area 360. In various embodiments, each layout may have a descriptive icon (e.g., as shown in column 310B), and in various embodiments, a descriptive name may be added in an additional column or used to replace the shown generic layout name (e.g., replacing generic layout 1, layout 2, etc. with a descriptive name) to enable the user to understand the purpose of the layout and / or the appearance of the layout, etc. As some examples, in various embodiments, some names that may be used for a particular layout and / or group may include "dark field", "diffuse", "directed", "grazing", "defect detection (scratches / pits)", "concentric rings", "pie segments", etc.

[0076] As indicated above, the quick view sliders in selection areas 319 and 329 allow the user to quickly browse through layouts and groups (e.g., where one or more of the selected groups of the selected layout are immediately displayed in the display area 360 of the light source display). In Figure 3A and Figure 3B the currently shown example, in the display area 310, the layout list is indicated as having a total of 17 layouts, where layout 1 is currently selected with the selection indicator SI (e.g., as indicated in column 310A), and as indicated to the right of the quick view scroll bar in selection area 319, the number 1 is shown for it. As indicated in the example, layout 1 consists of 12 groups of illumination channels (e.g., as indicated by the number 12 at the far right in selection area 329), and group 3 is currently selected (e.g., as indicated by the selection indicator SI in display area 320 and as indicated by the number 3 to the right of selection area 329). Accordingly, group 3 of layout 1 is displayed in the display area 360 (i.e., in the form of a pie wedge), where each illumination channel included in the set of illumination channels corresponding to group 3 is provided with a selection indicator SI (i.e., corresponding to the six illumination channels in the shown example, including three illumination channels in the outer ring of the illumination configuration, two illumination channels in the inner ring of the illumination configuration, and one illumination channel in the center of the illumination configuration, which combine to form a pie wedge).

[0077] In one example, the illumination configuration corresponding to the light source display in the display area 360 may correspond to a certain type of ceiling light. In another example (e.g., with respect to Figure 2 the exemplary configuration), the central illumination channel may correspond to the light source 240 (e.g., the arrangement provides coaxial illumination for it), and the remaining illumination channels may correspond to the illumination channels in the illumination configuration 230 (e.g., which may be an annular illumination configuration, such as an annular light, or a ceiling light without a central illumination channel, etc. in various embodiments).

[0078] In various embodiments, the system can be configured such that more than one group of lighting channels can be selected in the group list of the display area 320 (e.g., the user can perform a special operation such as Shift - click or Control - click after selecting the first group to select one or more additional groups of lighting channels, etc.). Once more than one group (e.g., multiple groups) is selected, the settings of the selected groups can be controlled together. In various embodiments, all selected groups are indicated (e.g., highlighted or provided with another type of selection indicator) in both the display areas 320 and 360 (i.e., in the group list and in the light source display), and for these groups, any RGB or brightness adjustment affects all selected groups (e.g., the adjustment settings of all currently selected groups can be made equal). Additionally, in various embodiments, more than one layout can be selected at a time (e.g., more than one layout from the layout list in the display area 310) (e.g., according to a selection process such as the user performing Shift - click or Control - click to select multiple layouts). In such an embodiment, once multiple layouts are selected, all groups in all layouts can be displayed in the group list of the display area 320, for which the groups can be browsed, combined, edited, deleted, and if needed, new group combinations can be saved as new layouts. In various embodiments, one or more additional constraints (e.g., enforcing mutually exclusive lighting channel groupings, color lighting channel / range limitations, etc.) that limit the operations that can be performed in the graphical user interface can also be defined, stored, and / or loaded in the parameters for the layout (e.g., in the layout definition, etc.).

[0079] In various embodiments, the layout parameters of the display area 330 can enable modification of layout - specific parameters. In various embodiments, the parameters can be dynamically configured based on the information loaded with the selected layout (e.g., the number of segments for pie - segment lighting, the number of rings for ring lighting, the number of lighting channels grouped as "point sources" in directional lighting, for some examples of these multiple groups of lighting channels shown in Figure 4 ). In various embodiments, certain non - geometric parameters can also be represented.

[0080] In various embodiments, the current lighting settings of a layout can be saved and later loaded (e.g., called from memory, etc.). In various embodiments, a user can define and save / load multiple specific lighting settings for a particular layout. In various embodiments, these settings can be stored in different files. For example, the system can be configured such that a user can load a layout and then save / load multiple lighting settings for the same layout. In various embodiments, although some lighting settings (e.g., some default lighting settings) can be saved with the layout (e.g., as part of the same file as the layout), it should be understood that the ability to store and recall many lighting settings for the same layout provides additional flexibility. In various embodiments, the lighting settings can be stored in files that are connected, linked, or otherwise associated with the layout (e.g., with the layout definition file) (e.g., by a naming convention, or included in a directory named after the layout, etc.). As a specific example, for a layout stored in a file named "RingLight", the corresponding lighting settings can be stored in files named "RingLight_scheme001", "RingLight_scheme002", etc. As an alternative to the term "scheme" in such file names and / or for otherwise referring to lighting settings, terms such as "configuration", "pattern", "settings", "set", etc. can be used in such file names and / or otherwise for referring to lighting settings.

[0081] In various embodiments, the display area 340 enables certain logical operations for all lighting channels simultaneously (e.g., to be able to modify / tweak various settings, etc.). In various embodiments, a user can also select the behavior of all channels belonging to more than one single group of lighting channels (e.g., shared between two or more groups of lighting channels). In various embodiments, the direct settings of the selection area 341A are directly applied to the RGB settings selected for each lighting channel. In various embodiments, the "invert to 0" calculation in the selection area 341B calculates 1 - the setting of each RGB component of the lighting channel and clamps the resulting number within the range [0,1]. The "fully invert" in the selection area 341C multiplies the setting of each RGB component of the lighting channel by -1. The following describes the "shared channel setting behavior" of the display area 342. Figure 5

[0082] Briefly turning to Figure 5 , an example related to the display area 342 of the shared channel setting behavior is shown. In certain embodiments, the example can be used to determine how a set of colored light (e.g., RGB) settings affects the lighting channels shared between groups (e.g., in embodiments where such sharing is enabled, belonging to more than one group). For example, asFigure 5 As shown, the lighting channels of the intersection INTS belonging to a group of lighting channels 510 (e.g., corresponding to a ring shape) and a group of lighting channels 520 (e.g., corresponding to a pie-shaped segment) can be set according to the selection of one of the options 342A - 342E. More specifically, the lighting channels corresponding to the intersection INTS (e.g., those corresponding to the lighting channels within the overlapping region between the two groups of lighting channels 510 and 520) can be set according to option 342A (i.e., the average value for the two groups), option 342B (i.e., the sum for the two groups), option 342C (i.e., the product for the two groups), option 342D (i.e., the minimum value for the two groups), or option 342E (i.e., the maximum value for the two groups) of the RGB settings for the two groups of lighting channels 510 and 520. The behavior selected for the shared lighting channels is used by the lighting control system (e.g., during an automatic lighting adjustment process, such as during a lighting optimization process, etc.).

[0083] Return to Figure 3A and Figure 3C , the "main settings" of the display area 350 correspond to RGB lighting control, which displays and allows adjustment of the grouped lighting channels (e.g., or in some cases a single lighting channel) selected in the light source display of the display area 360. In various embodiments, the selection of a single lighting channel in the light source display of the display area 360 may result in the automatic selection of the entire group of lighting channels (if any) to which the single lighting channel belongs. In various embodiments, negative settings (e.g., for RGB lighting control) are enabled and can be displayed (e.g., in the workpiece display of the display area 380). For example, such a display may include: obtaining two images, one with any positive lighting channel weight and the other with the absolute value of any negative lighting channel weight; then subtracting the image illuminated with the negative lighting channel weight from the image illuminated with the positive lighting channel weight; and then displaying the resulting difference image using a specified representation of any negative color weight regions in the image.

[0084] In various embodiments, selecting the undo and redo options 353A and 353B enables the user to switch back and forth between any of the most recent RGB setting changes. In various embodiments, selecting the group option in region 354 sets all three RGB sliders of option 351 to the same setting (e.g., the average setting of the three sliders), and may allow the three sliders to be adjusted together. In various embodiments, selecting the up and down arrows of options 352A and 352B enables brightness adjustment while maintaining color (RGB) balance (e.g., the selected lighting channel or the selected set of lighting channels). In various embodiments, the user selecting (e.g., pressing) one of the arrows multiplies the RGB components by the same factor, and in various embodiments, the result may be clamped within the range [0,1]. In various embodiments, such controls are typically only implementable or otherwise available when all RGB settings are positive. In various embodiments, when certain limits are reached, the brightness adjustment using the arrows may be blocked or otherwise indicated as unavailable, such as not allowing further downward adjustment once any of the RGB values reaches 0, or not allowing further upward adjustment once one of the RGB values reaches 1.

[0085] In various embodiments, the functionality of the selection options in display region 370 may be substantially the same as the selection options in display region 350, unless otherwise specified below. In various embodiments, the selection option 371 in display region 370 may apply a preset "override" setting to multiple groups of lighting channels while making certain types of adjustments. For example, a "maximum brightness" setting may be selected, such as by using the up arrow of selection option 374A, and then a scrolling operation may be performed, such as using the quick preview slider in region 329 to view the effect of the selected setting (e.g., the maximum brightness setting) on the artifacts of different groups of lighting channels (e.g., a circular group or a pie segment group, etc.) through different groups of lighting channels. In various embodiments, when each group of lighting channels is not selected (e.g., when the quick scroll operation moves to the next group in the group list in display region 320), each group of lighting channels returns to its "normal" (e.g., main) setting. In various embodiments, the "other channels off" checkbox option in selection region 372 (if selected) turns off the lighting channels of all currently unselected groups when the "override selected settings" checkbox in selection region 371 has been selected and is active.

[0086] In various embodiments, the "Light Source Display" of the display area 360 provides a visual representation of all lighting channels in a configuration similar to the actual hardware of the lighting configuration LC. In various embodiments, the light source display can display the current settings when the radio button of the "Settings Display" in the selection area 366B is selected, and can display the lighting channel grouping when the radio button of the "Group Display" in the selection area 366A is selected. In various embodiments, when the settings display is selected in the selection area 366B, the true selected RGB color of the selected lighting channel can be displayed. In various embodiments, lighting channels with at least one negative RGB weight can be represented by indicators (e.g., special colors such as black, borders, dynamic patterns such as blinking colors, etc.).

[0087] In various embodiments, the "Group Display" in the selection area 366A can, when selected, highlight the currently selected group or groups of lighting channels (e.g., such as group 3 indicated by the selection indicator SI in the group list of the display area 320). This setting can allow for the selection of lighting channels, the creation and / or deletion of groups (e.g., by making a selection such as a right click after selecting a lighting channel and choosing an action from a menu). In various embodiments, multiple lighting channels can be selected and / or deselected through various operations (e.g., using control click, lasso, bounding box, moving a selector over the channels with a mouse, such as corresponding to path-based selection, etc.). In various embodiments, lighting channels that are not assigned to any group in the currently selected layout can be marked with a special color or pattern to indicate the corresponding status (e.g., to distinguish them, etc.).

[0088] In various embodiments, the "Expand", "Rotate", and "Direct" controls of the selection areas 362, 363, and 364 can be adaptive controls (e.g., which can be enabled / disabled based on the currently selected / loaded layout), and will act on one or more groups of selected lighting channels. In various embodiments, the "Expand" option of the selection area 362 can be used to form multiple groups of lighting channels in concentric rings or other similar geometric structures, for which selecting on the control (e.g., dragging the mouse cursor) can activate smaller and / or larger rings (e.g., in embodiments where the lighting configuration includes all or part of a ceiling light, these functions will effectively move the light ring between the center and the outer edge of the ceiling light). In various embodiments, the "Rotate" option of the selection area 363 can be used for configurations of multiple groups of lighting channels such as forming pie segments or similar geometric arrangements, for which selecting the control (e.g., dragging the mouse cursor onto the control) will "rotate" the segment around the lighting configuration, thus effectively changing the 2D (in-plane) direction of the lighting.

[0089] In various embodiments, the "direct" control of selection region 364 can be used to form multiple sets of illumination channels of a directional light source (e.g., as indicated by examples 442, 443, 444, etc. such as Figure 4 ). In various embodiments, the utilization of the direct control 364 (e.g., by dragging the cursor of a mouse onto the control) can act like a rotating trackball and can change the 3D direction of the illumination (e.g., a focused illumination beam) of a set of illumination channels. In various embodiments, as an alternative or supplement to the direct control 364 (i.e., for determining the illumination direction of a set of illumination channels), an external input device (e.g., Figure 2 's input device 138), such as a physical trackball and / or a joystick, etc., can be utilized to provide similar functions for certain controls (e.g., for rotation and direct control of selection regions 363 and 364, etc.). In various embodiments, the "force exclusive group" checkbox of selection region 365 can prevent illumination channels from being assigned to multiple sets of illumination channels (e.g., such that no illumination channels can be shared between multiple sets of illumination channels in a layout). For some embodiments, such a setting may be desirable.

[0090] In various embodiments, a factory-defined hardware configuration and / or a light source driver can be provided and loaded by the controller for a graphical user interface to establish graphical user interface controls and parameters (e.g., such as the geometric layout of the light source display in display region 360, the range of sliders, and possible adjustment types, such as RGB settings, and / or brightness, regarding the ability to utilize negative RGB settings, etc.). Alternatively or additionally, in some embodiments, a certain type of calibration routine can be utilized to determine illumination configuration options. For example, in one embodiment, a certain type of calibration ball or other calibration object can be placed in the field of view (e.g., below camera 260 and illuminated by the illumination configuration), and each available control of the illumination configuration can be adjusted for it, and for each captured image after each modification, the image can be analyzed and utilized to determine what available options / illumination channels the illumination configuration has (e.g., for determining available illumination channel grouping / optimization, etc.).

[0091] Regarding the workpiece display in the display area 380, in various embodiments, a real-time or synthetic camera view of the workpiece 20 can be displayed. In various embodiments, when using negative RGB weights, a synthetic view can be provided, for which two images can be collected and subtracted, and for which a representation of any negative RGB results can be provided, as described herein. In the example shown, in the upper display area 380A, an example A corresponding to the region of interest of the edge selector of the edge detection tool EDT is provided. This shows an example where it may be desirable to determine lighting optimization to achieve robust edge detection. In the lower display area 380B, as part of example B, a region of interest of the defect area selector is shown (e.g., as part of the defect detection tool DDT for detecting defects DFT). This shows an example where it may be desirable to determine lighting optimization to achieve defect detection.

[0092] In various embodiments, such an edge detection tool EDT or defect detection tool DDT can be contextually displayed depending on the selection made in the display area 390, such as depending on whether the lighting for optimizing one of the following is selected: edge detection (e.g., edge detection in the selected area 391A), defect detection (e.g., defect detection in the selected area 391B), or taking a point from the focus (e.g., taking a point from the focus in the selected area 391C). Generally speaking, it is characterized in that the "optimization" option in the display area 390 is used to select which type of lighting optimization mode the automatic lighting optimization process will be performed for (e.g., regarding which adjustment algorithms, performance metrics, etc. will be used in association with optimizing the lighting for a specific mode).

[0093] In various embodiments, the "optimization" option in the display area 395 can be used to select which parameters (i.e., in addition to the position of the lighting channels, etc.) can be adjusted for optimization or otherwise optimized. Selecting the checkbox for brightness (i.e., in the selected area 396A) automatically selects all three of R, G, and B (i.e., red, green, and blue lighting). When the checkbox for brightness in the selected area 396A is deselected, any one of the R, G, and / or B options in the selected areas 396B, 396C, and 396D can be selected individually or in combination. It should be understood that the examples of configurable parameters provided are not exhaustive, and selection options can be provided for other configurable parameters in the corresponding selection areas of the graphical user interface.

[0094] Figure 4 Examples of different groups of lighting channels are shown (e.g., which can be represented according to certain shapes). It should be understood that Figure 4 the shapes are for illustrative purposes only, and depending on the density of the lighting channels in a given lighting configuration, the shown shapes can be considered to correspond to a set of individual discrete lighting channels that appear within the general area of the shape.Figure 4 Multiple sets of illumination channels therein are arranged into sets, including sets 410, 420, 430, 440, 450, 460, and 470.

[0095] Set 410 includes a set of illumination channels 411, a set of illumination channels 412, a set of illumination channels 413, and a set of illumination channels 414. Set 410 can be characterized as multiple sets of illumination channels corresponding to a "dark field" (e.g., in some embodiments, it can be characterized as being at an angle of 45 degrees to 90 degrees with respect to the optical axis of the lens / camera and may be asymmetric in some cases). Set 420 includes a set of illumination channels 421, a set of illumination channels 422, a set of illumination channels 423, and a set of illumination channels 424. Set 420 can be characterized as multiple sets of illumination channels corresponding to a "bright field" (e.g., in some embodiments, it can be characterized as being at an angle of 0 degrees to 45 degrees with respect to the optical axis and may be asymmetric in some cases).

[0096] Set 430 includes a set of illumination channels 431, a set of illumination channels 432, a set of illumination channels 433, and a set of illumination channels 434. Set 430 can be characterized as multiple sets of illumination channels corresponding to "diffuse" (e.g., it can correspond to a larger set of illumination channels, which can be symmetric or asymmetric). Set 440 includes a set of illumination channels 441, a set of illumination channels 442, a set of illumination channels 443, and a set of illumination channels 444. Set 440 can be characterized as multiple sets of illumination channels that are "directed" (e.g., in some embodiments, it can be an example of a typically asymmetric layout and can include a small and concentrated set of illumination channels).

[0097] Set 450 includes a set of illumination channels 451 and a set of illumination channels 452. Set 450 can be characterized as multiple sets of illumination channels corresponding to "grazing" (e.g., in some embodiments, it can be characterized as corresponding to a dark field at an angle close to 90 degrees with respect to the optical axis of the lens / camera and can be asymmetric / directed in some cases). Set 460 includes a set of illumination channels 461, a set of illumination channels 462, a set of illumination channels 463, and a set of illumination channels 464. Set 460 can be characterized as multiple sets of illumination channels of the "concentric ring" type (e.g., in some embodiments, it can be characterized as substantially including both examples of "dark field" and "bright field").

[0098] The set 470 includes a set of illumination channels 471, a set of illumination channels 472, a set of illumination channels 473, and a set of illumination channels 474. In some embodiments, the set 470 may be characterized as multiple sets of illumination channels in a "pie segment" (e.g., each set of illumination channels forms one or more shapes corresponding to one or more pie segments). It should be understood that certain defect detection processes (e.g., for identifying scratches, pits, etc. in the surface of a workpiece) may utilize "dark field", "grazing", and / or "concentric ring" types of grouped illumination channels in some embodiments (e.g., certain examples of such illumination channels are shown in sets 410, 450, and 460).

[0099] Figure 6 Embodiments are shown that represent a technique for negative color weights (e.g., in the image IMG in the workpiece display 380 of Figure 3A and Figure 3C for displaying the workpiece WP illuminated by the current illumination settings). As described above with respect to display regions 350 and 370, certain color light settings may have negative weights. In the examples shown in the display regions 350 and 370 of Figure 3A and Figure 3C , the red light setting is indicated to have a negative weight corresponding to a value of -0.50. Since the actual negative values of the color light settings cannot be displayed in the image, one technique is to instead display a negative weight indicator region for indicating the regions on the workpiece that have negative color weights. In the example of Figure 6 , four negative weight indicator regions NWIA are shown, each corresponding to a region where at least one component of the color shown therein is negative.

[0100] As described above, in various embodiments, negative color illumination settings (e.g., red, green, and / or blue illumination settings) may be enabled. In various embodiments, when there are negative color illumination channel values in a particular image region, image feedback in the graphical user interface 300 may be provided (e.g., as shown in Figure 6 ). In various embodiments, such a representation may be determined by acquiring two images, including an image illuminated with a "negative" illumination channel weight, which is subtracted from the image illuminated with a positive illumination channel weight. In various embodiments, the resulting difference image including the representation of the negative values in the image may be displayed. For example, appropriate colors, patterns, graphical effects, etc. may be utilized as part of a mapping convention to represent the negative color illumination channel values in the displayed image (e.g., as shown by the negative weight indicator region NWIA in Figure 6 ).

[0101] In various embodiments, various techniques can be utilized to display negative color illumination channel weights. In certain embodiments, pseudo-color techniques can be used, including mapping a series of negative and positive values to a color range that may or may not have an underlying physical representation. This can utilize the concept of a color map (e.g., a look-up table) that maps input values to specific colors. In one embodiment, the mapping can include calculating the grayscale value of all RGB triples in a standard manner (including using absolute values), then negating the grayscale value if at least one of the RGB components is negative, and in this case, the possible range of the grayscale value from -1 to +1 (e.g., scaled to -255 to +255) can be used to index the color map. In various embodiments, different textures (e.g., stripes, checkerboards, etc.) that can be static or dynamic (e.g., moving, flashing, blinking, or using some other form of spatio-temporal modulation) can be utilized to emphasize special negative values in the displayed image. In various embodiments, certain different colors (e.g., including primary colors and / or saturated colors, such as red, green, blue, or different strong combinations of RGB, depending on which channels are negative) can be utilized, and these colors can be static or dynamic (e.g., flashing at a value-dependent frequency or a constant frequency, etc.).

[0102] Generally speaking, it should be understood that any method capable of highlighting the presence of negative RGB values in an image can be utilized. In certain embodiments, it may be desirable to maximize the contrast of the region where negative RGB values occur (e.g., using a color that does not originally exist in the image, or using grayscale if the image is color, or using color if the image is grayscale, etc.). Regarding Figure 6 as an example, if the rest of the image IMG is grayscale, the negative weight indicator area NWIA can be provided in color (e.g., a bright color such as bright blue that can be easily distinguished from grayscale). As another example, if the rest of the image IMG is color, the negative weight indicator area NWIA can be provided in grayscale or other format (e.g., such that it can generally contrast with the rest of the image IMG). In various embodiments, the negative weight indicator area NWIA can indicate that at least one of red, green, or blue has a negative value.

[0103] Figure 7 is a flowchart of a routine 700 for operating a system having multiple sets of illumination channels. At block 710, a representation of multiple sets of illumination channels in a display area is provided. At block 720, it is determined that a set of illumination channels has been selected, where the selected set of illumination channels includes multiple illumination channels. At block 730, the current illumination settings of the selected set of illumination channels are displayed, where adjustments to the illumination settings of a set of illumination channels are applied to all illumination channels in the set.

[0104] In various embodiments, the position of an edge on a workpiece is determined using scan lines and / or one or more edge tools (e.g., via edge detection section 144). The following Figures 8 to 10 will be described in the form of exemplary operations of edge tools (e.g., point and box edge tools), which in certain embodiments can be or can be similar to a "video tool" that can utilize scan lines to determine edge positions. For example, in various embodiments, Figure 2 tools 143a and / or tool 143n, etc. of the video tool section 143 of Figures 8 to 10 can be edge tools, etc., or can include the utilization of features of edge tools, etc. (e.g., which can be utilized by or in combination with the edge detection section 144 in various embodiments). It should be understood that in various embodiments, similar operations (e.g., including using scan lines or similar techniques) can be performed to determine edge positions, which can be automated and in some cases without otherwise displaying and / or utilizing other features of the "video tool" or other tool type features shown and described below with respect to

[0105] Figure 8 is a diagram of an exemplary point tool 810 that is laid on a first edge 805 of a portion of workpiece 20 at the boundary of a darker or shaded region 808 in an image 800 such as acquired by camera 260. In Figure 8 and the figures described below, shaded regions (such as shaded region 808) are shown for illustrative purposes to indicate pixels in the image having a relatively high "gray value" (e.g., or other color value). In various embodiments, depending on the configuration, a higher gray value (e.g., or other color value) can correspond to a pixel of relatively lower intensity or relatively higher intensity.

[0106] As will be described in more detail below, the point tool 810 can be configured to determine the position of edge points on an edge in an image, and similar operations can be the basis for the operation of another type of tool (e.g., a box-type edge tool) for locating multiple edge points on an edge, as will be described in more detail below with reference to Figure 9 In certain embodiments, the point tool 810 can include a body 811, an edge selector 812, and a polarity indicator arrow 814. The arrow 814 can generally point from light to dark or from dark to light across the edge to enhance edge finding reliability in some cases, as explained in the incorporated references. In Figure 8In the illustration, the main body 811 obscures the nominal scan line 820 defined by and nominally coincident with the main body 811. This will be further referred to below Figure 9 The scan lines (820 and 920) are shown and described in more detail.

[0107] In various embodiments, certain operations described below (e.g., determining an edge position using a scan line) may be automatically performed by the machine vision inspection system 100 (e.g., without displaying corresponding video tool features on a display). In operation, the machine vision inspection system 100 may be configured to automatically select and utilize a point tool and / or a corresponding operation (e.g., using a scan line) or the user may select the point tool 810 or a corresponding operation. The system or the user may identify an edge feature to be detected by placing the main body 811 above the edge feature and placing the edge selector 812 at a desired position along the edge as close as possible to the edge. The main body 811 of the point tool may be oriented to define and indicate a desired scan line orientation across the edge (e.g., in the example shown, for simplicity, it is parallel to the x-axis of the shown x-y coordinate system, but in some embodiments, it may alternatively be at an angle with respect to the x and y axes within the x-y coordinate system). In Figure 8 And in other figures herein, the image pixels are arranged in rows along the x coordinate direction and in columns along the y coordinate direction. The arrow 814 points to a reference direction or polarity associated with edge detection. In operation, once the point tool 810 is configured (and / or once the corresponding operation is automatically determined and / or performed), the instructions of the underlying edge point location routine may be executed to perform operations on the analysis and intensity distribution data points (e.g., pixel intensity data) associated with the scan line nominally coincident with the main body 811 of the point tool 810, and perform various operations to detect the edge position of the underlying feature. As described below with respect to Figure 10 As will be described in more detail, in various exemplary embodiments, the edge point location routine of the point tool 810 may determine the edge position based on the magnitude of the gradient along the intensity distribution associated with the scan line.

[0108] Figure 9 is a diagram of an exemplary box tool 910 that is laid over a first edge 905 of a portion of the workpiece 20 at the boundary of the shaded region 908 in the image 900. In various embodiments, Figure 3C The edge detection tool EDT may be as Figure 9The box tool of the shown box tool and / or can operate similarly to such a box tool. In various embodiments, the box tool 910 can include a region of interest (ROI) indicator 911, a polarity indicator arrow 914 along the side, and an edge selector 912 located on the desired edge to be detected. Within the box tool 910, a nominal scan line 920 can be indicated, and the box tool 910 generates the nominal scan line and uses it to determine edge points. Along each nominal scan line 920, the box tool operates to determine the edge points of the underlying feature, as previously described for the scan line 820 of the point tool 810 in Figure 8 as shown. And as Figure 9 shown and will be described in more detail below, a line 925 can be fitted to the set of determined edge points to determine the position and orientation of the edge 905.

[0109] In operation, the box tool 910 is selected and / or otherwise configured to identify the edge feature to be detected. The ROI indicator 911 can be positioned, resized, and rotated (e.g., automatically or by the operator) such that the ROI includes a portion of the edge feature to be detected, and the edge selector 912 can be positioned to more precisely identify the edge to be detected at the desired position along the edge. The ROI indicator 911 can be oriented to define and indicate the desired scan line orientation across the edge. More generally, the orientation of the overall ROI indicator 911, the portion of the ROI indicator including the arrow 914, or the orientation and / or corresponding operation of the nominal scan line 920 can be used to define and / or indicate the scan line orientation. The arrow 914 defines the polarity associated with the edge detection. Once the box tool 910 is configured, the instructions of the underlying edge point localization routine can be executed to perform the operation of analyzing the intensity distribution data to detect the edge points along each scan line and fit a line to the detected edge points, as described in more detail below.

[0110] In summary, generally, edge points can be determined by various tools and / or corresponding operations, and in various embodiments, the tools and / or corresponding operations can fit a geometry to the edge points to determine the position of the corresponding underlying image feature (e.g., as part of a process for determining the exact position of the edge of a feature on a workpiece, etc.). In a traditional method of operating a tool, one or more nominal scan lines are defined or generated according to the type of the tool (e.g., point tool and box tool) (e.g., within the ROI or other locations, such as according to defined tool parameters). For each nominal scan line, a set of intensity distribution data point positions approximating the nominal scan line is determined. The intensity values associated with the data point positions are determined to define the intensity distribution associated with the nominal scan line. Then, in one embodiment, an edge detection algorithm analyzes the gradient along the intensity distribution to find the position corresponding to the maximum gradient magnitude along the intensity distribution, as described below with reference toFigure 10 is described in more detail. The position of the maximum gradient along the intensity distribution is used to determine the position of the edge point in the image associated with the nominal scan line.

[0111] Figure 10 is a diagram of a graphic 1000 showing an exemplary method of determining an edge position based on an intensity distribution 1010. The intensity distribution 1010 includes a set of image pixel intensity values (e.g., such as grayscale or other color) 1025 corresponding to the position of a scan line (e.g., one of the scan lines 820 or 920 as shown respectively in Figure 8 or Figure 9 ). The data points or positions representing the scan line are labeled along the horizontal axis as "number of pixels" from 0 - 35. Starting from data point 0, the pixel intensity values initially indicate a relatively bright region until approximately data point 23, and then a relatively dark region until data point 35.

[0112] It should be understood that Figure 8 , Figure 9 and Figure 10 the values and illustrations in Figure 8 and Figure 9 have been simplified in various aspects with respect to the current example. For example, in Figure 8 and Figure 9 , the determined edge position is shown to be located between a relatively bright region (e.g., close to white) and a darker region, while in some actual implementations, the "brighter region" may have a higher grayscale or other color value. Additionally, although Figure 8 and Figure 9 the edges are shown as relatively "sharp" (i.e., having a relatively rapid transition between the brighter and darker regions), in some actual implementations, the transition of the edge may occur over multiple pixels (e.g., as shown by the transition of the exemplary grayscale or other color values from pixel 23 to pixel 27 in Figure 10 ).

[0113] The gradient magnitude values 1026 are derived from the pixel intensity values 1025 and are also shown in Figure 10 . Various conventional algorithms can be used to find the position along the horizontal axis corresponding to the peak of the gradient magnitude (which is considered to represent the peak "contrast" between pixel intensity values), and this position is identified as the edge position. If there are multiple gradient magnitude peaks, video tool edge selectors and / or orientation indicators (e.g., the edge selector 812 and / or the polarity indicator 814 of the point tool 810) can be used to assist the algorithm in identifying the desired peak. In Figure 10Among them, the maximum gradient criterion (i.e., the peak of the gradient) indicates that the edge is approximately located at data point 23. By using methods that can include curve fitting, centroid determination, etc., the peak of the gradient magnitude can be relatively accurately located between the intensity distribution data points, which generally supports sub-pixel measurement resolution and repeatability when determining the position of the corresponding edge in the image. For example, although Figure 10 in the example shown, the peak of the gradient is approximately located at data point 23 (e.g., corresponding to pixel 23 in this example), it can be seen that another "near peak" appears at data point 24 (pixel 24). This may indicate that a more accurate determination of the edge position may correspond to the position between data points 23 and 24 (pixels 23 and 24), which can be determined by processes such as curve fitting, centroid determination, etc. As a simplified concept, in an example where data points 23 and 24 have the same gradient magnitude, this may correspond to the edge position exactly at 1 / 2 between the two (e.g., the edge position corresponds to the data point / pixel position of 23.5).

[0114] Figures 8 to 10 The example of

[0115] Figure 11A figure showing a metrology operation (e.g., for determining dimensions of a workpiece feature, etc.) being performed on an image of a workpiece including the workpiece feature using one or more video tools. In one example, the workpiece feature 1102 may correspond to a scratch defect or other type of workpiece feature. As shown, for the image 1101 including the workpiece feature 1102, the video frame tool 1106 includes scan lines 1108 (e.g., which may also or alternatively represent a video point tool, etc.), and the scan lines are used to determine the edge position, dimensions, and / or other aspects of the workpiece feature 1102. In various exemplary embodiments, the video frame tool 1106 can be resized, positioned, and rotated until the frame tool 1106 indicates or defines the region of interest (e.g., the region within the frame tool 1106), and the arrows (e.g., representing scan lines, point tools, etc.) shown in Figure 11 can be used to determine the edge of the workpiece feature 1102. In various exemplary embodiments, the video frame tool 1106 can generally use the conventional edge gradients along one or more edges of the workpiece feature 1102 in the region of interest, and can determine the edge of the workpiece feature 1102 based on the local magnitudes of the edge gradients along the various scan lines 1108.

[0116] In various exemplary embodiments, such a metrology operation can also include performing certain morphological filtering or other filtering (e.g., to distinguish the edges of the workpiece feature / scratch from the machining pattern of the workpiece, for which certain types of filtering are described in U.S. Patent No. 7,522,763, which is hereby incorporated by reference in its entirety). As Figure 11 shown, in the display area included in the image, the frame tool 1106 with the scan lines 1108 is used to determine the edge position (e.g., the outer edge or perimeter) of the workpiece feature 1102. Based on such a determination, the video tool and / or other metrology operations can include determining the dimension D1 of the workpiece feature (e.g., corresponding to the length or other dimension of the workpiece feature 1102). In various exemplary embodiments, the frame tool 1106, the scan lines 1108, and / or other video tools and / or metrology operations can be used to determine other dimensions (e.g., width, depth, etc.) of the workpiece feature 1102. For example, as described herein, the take point portion 146 from the focus can perform operations for determining three-dimensional surface profile data, from which the Z-height dimension of the workpiece feature can be determined (e.g., including determining the depth of the workpiece feature / scratch relative to other parts or features of the workpiece surface).

[0117] As described above, in some embodiments, the workpiece feature 1102 may correspond to a scratch defect 1102 on the workpiece. In various embodiments, with respect to the operation of the defect detection section 145 and / or the illumination optimization section 147, the user may provide an image having an example of a scratch defect similar to the scratch defect 1102. As an example, the provided image may be a similar part of the workpiece (e.g., for these parts, the type of machining marks formed on the surface of each part is similar or nominally the same in each image, and the main difference between the images is the characteristic of the corresponding scratch defect). In an exemplary embodiment, such an image may be included as part of a training image set for training.

[0118] In an exemplary embodiment, Figure 11 the image can be an example of an image that can be analyzed by the defect detection section 145 to detect the scratch defect 1102. In various embodiments, the defect detection section 145 may be well trained to correctly detect the scratch defect 1102 (e.g., as has been trained with training images including similar parts of the workpiece and scratch defects having certain similar features to the scratch defect 1102). In various embodiments, certain additional processes may be performed for the exemplary scratch defect 1102. For example, one or more metrology processes may be performed in conjunction with the defect detection process, for which various dimensions or other characteristics of the scratch defect 1102 may be determined.

[0119] As part of the general operation of the defect detection section 145, some detected defects may require further metrology or inspection to determine additional defect parameters. For example, as described above, performing various types of analysis and / or processing (e.g., using the edge detection section 144) on an image including the scratch defect 1102 can determine the XY position and approximate XY area and / or other dimensions of the scratch defect 1102 (e.g., using video tools and / or other operations as described above). If the 3D nature of the potential defect is important (e.g., if the scratch must be deeper than a certain value to be considered a defect), the defect detection section 145 can initiate a focus-picking process (e.g., a focus-picking process from the focus-picking section 146, such as for obtaining a 3D point cloud of the affected area to determine the depth of the scratch defect 1102, etc.). As will be described in more detail below, different illumination optimizations can correspond to each of these different types of processes (e.g., for defect detection, edge detection, focus-picking, etc.)

[0120] Figure 12 is a diagram showing the performance of metrology operations (e.g., for determining the dimensions of a workpiece feature, etc.) on a workpiece feature in an image of a workpiece using one or more video tools. In one example, the workpiece feature 1202 may correspond to a defect, or other types of workpiece features (e.g., similar toFigure 11 Example). As shown in the figure, for the image 1202 including the workpiece feature 1204, the video frame tool 1206 includes scan lines 1208 (e.g., which may also or alternatively represent a video point tool, etc.), and the scan lines are used to determine the edge position, size, and / or other aspects of the workpiece feature 1204. In various exemplary embodiments, the video frame tool 1206 can be resized, positioned, and rotated until the frame tool 1206 indicates or defines the region of interest (e.g., the region within the frame tool 1206), and the arrows shown in Figure 12 (e.g., representing scan lines, point tools, etc.) can be used to determine the edge of the workpiece feature 1204. In various exemplary embodiments, the video frame tool 1206 can generally use the conventional edge gradients along one or more edges of the workpiece feature 1204 in the region of interest, and can determine the edge of the workpiece feature 1204 based on the local magnitudes of the edge gradients along various scan lines 1208, etc.

[0121] Various other processes can also or alternatively be performed, such as threshold processing and / or forming a binary image (e.g., as part of a defect detection process, etc.). As Figure 12 shown, in the display area included in the image, the frame tool 1206 with scan lines 1208 is used to determine the edge position (e.g., the outer edge or perimeter) of the workpiece feature 1204. Based on such determination, the video tool and / or other metrology operations can include determining the size D2 of the workpiece feature (e.g., corresponding to the length or other dimension of the workpiece feature 1204). In various exemplary embodiments, the frame tool 1206, scan lines 1208, and / or other video tools and / or metrology operations can be used to determine other dimensions (e.g., width, depth, etc.) of the workpiece feature 1204. For example, as described herein, the focus taking point portion 146 can perform operations for determining three-dimensional surface profile data, from which the Z-height dimension of the workpiece feature can be determined (e.g., including determining the depth of the workpiece feature / defect relative to other parts or features of the workpiece surface, etc.).

[0122] As described above, similar to the example of Figure 11 , the workpiece feature 1202 may correspond to a defect 1202 on the workpiece in some embodiments. In various embodiments, regarding the operations of the defect detection portion 145 and / or the illumination optimization portion 147, the user can provide an image having examples of defects similar to the defect 1202. As an example, the provided image can be a similar part of the workpiece (e.g., for these parts, the type of the surface of each part is similar or nominally the same in each image, and the main difference between the images is the characteristic of the corresponding defect). In one exemplary embodiment, such images can be included as part of a training image set for training.

[0123] In an exemplary embodiment, Figure 12 The image of may be an example of an image that can be analyzed by the defect detection section 145 to detect the defect 1202. In various embodiments, the defect detection section 145 may have been well trained to correctly detect the defect 1202 (e.g., as trained with training images including similar parts of the workpiece and defects having some similar characteristics to the defect 1202). In various embodiments, certain additional processes may be performed for the exemplary defect 1202. For example, one or more metrology processes may be performed in conjunction with the defect detection process, and for these processes, various dimensions or other characteristics of the defect 1202 may be determined. As will be described in more detail below, different lighting optimizations may correspond to each of these different types of processes (e.g., for defect detection, edge detection, taking points from the focus, etc.).

[0124] Figures 13A to 13C is a diagram showing certain principles related to the defect detection process (e.g., performed by the defect detection section 145). In various embodiments, as part of such a defect detection process, it may be necessary to provide lighting that makes the surface texture visible without underexposing or overexposing any part of the corresponding surface. Figures 13A to 13C Shows different settings of certain exemplary lighting channels, which correspondingly show the ways in which certain surface texture features and / or defects may be more or less visible or detectable. Figures 13A to 13C The example of involves the surface of a machined aluminum plate, which includes ridges.

[0125] Figures 13A to 13C Shows images 1300A - 1300C obtained using different settings of the lighting channels LCH1 and LCH2. In various embodiments, the lighting channel LCH1 may correspond to a coaxial lighting channel (e.g., as provided by the lighting channel 240 in the example of Figure 2 In various embodiments, the lighting channel LCH2 may correspond to a single lighting channel, or may represent a set of lighting channel settings (e.g., the settings of ceiling lights or ring lights, as provided by the lighting configuration 230 according to the example of Figure 2 ). Figure 13A Shows the image 1300A obtained by illuminating the workpiece surface when the setting of the lighting channel LCH1 is 3 and the setting of the lighting channel LCH2 is 7. In the resulting image 1300A, the surface of the workpiece is somewhat dark, but the grooves in the surface can be seen. In Figure 13BIn this case, the setting of illumination channel LCH1 is 5, while the setting of illumination channel LCH2 is 7. This illumination combination for illuminating the surface of the workpiece produces Image 1300B, in which the surface is slightly brighter than the surface in Image 1300A, and the grooves in the workpiece surface have slightly different visibility.

[0126] In Figure 13C the setting of illumination channel LCH1 is 0, while the setting of illumination channel LCH2 is 11. The settings of these illumination channels are combined to illuminate the workpiece, resulting in Image 1300C. As can be seen from Image 1300C, certain features on the workpiece surface are more obvious, while in contrast, these features are less obvious in Images 1300A and 1300B. For example, certain scratches that are usually horizontally oriented are more obvious in Image 1300C, such as the defect DFT shown in the upper left corner of the image. In this example, in order to make the defect relatively more obvious, it is desirable for illumination channel LCH1 to have a relatively low setting (e.g., set to 0), and illumination channel LCH2 to have a relatively high setting (e.g., set to 11). As described above, generally for the defect detection process, it may be desirable to have the visibility of certain surface textures of the workpiece surface (e.g., corresponding to the type of defect being detected) without under - exposing or over - exposing those parts of the workpiece surface.

[0127] Figure 14A and Figure 14B are diagrams showing contrast focus curves, which may be generated by illumination characteristics in corresponding regions of interest and can be used for a focus - taking process (e.g., as performed by the focus - taking section 146). As will be described in more detail below with respect to Figure 14A and Figure 14B In various embodiments (e.g., as part of a focus - taking process), the camera 260 is used to acquire an image stack including multiple images of the workpiece surface with illumination, where each image of the image stack corresponds to a different Z - height. The focus curve data (e.g., for which the visible contrast level may depend on illumination) is determined at least in part based on an analysis of the images of the image stack, where the focus curve data indicates the three - dimensional positions of multiple surface points on the workpiece surface (e.g., including Z - height and relative x - and y - axis positions).

[0128] As Figure 14A and Figure 14BAs shown, the contrast focus curves 1401 and 1402 may be generated by the illumination characteristics of corresponding regions of interest on the workpiece. It is generally desirable for the illumination to produce sufficient contrast in the image such that the peak of the contrast curve can be accurately located and reliably distinguished from the noise of all desired regions of interest. As will be described in more detail below, focus curve data indicating the three-dimensional position of surface points on the surface of the workpiece can be determined by analyzing an image stack (e.g., as part of a pick-from-focus (PFF) process / measurement operation).

[0129] Figure 14A and Figure 14B illustrates how an image stack obtained by the system 100 (e.g., using the illumination provided by the illumination channel of the illumination configuration LC) can be used to determine the Z-height of points on the workpiece surface. In various embodiments, the image stack is obtained by the system 100 (e.g., according to the operation of the pick-from-focus section 146) to determine three-dimensional profile data (e.g., including the Z-height of points on the workpiece surface, etc.). The PFF image stack can be processed to determine or output a Z-height coordinate map (e.g., a point cloud) that quantitatively indicates a set of three-dimensional surface coordinates (e.g., corresponding to the surface shape or profile of the workpiece).

[0130] In the PFF analysis described herein, each of the focus curves 1401 and 1402 (as Figure 14A shown) corresponds to a single point on the workpiece surface. That is, the peak of each focus curve indicates the Z-height of a single point along the direction of the optical axis OA of the vision component portion 200 of the system 100. In various embodiments, the PFF type analysis repeats this process for multiple surface points on the workpiece surface (e.g., each surface point having a corresponding region of interest), such that the overall profile of the workpiece surface can be determined. Generally, the process can be performed for multiple surface points within the field of view (i.e., captured within the images of the image stack), where for each image of the image stack, a specific ROI(i) corresponds to a specific point on the workpiece surface (e.g., the point is located at the center of the ROI).

[0131] Figure 14A and Figure 14B are aligned relative to each other along the Z-height axis shown in the figure. Figure 14A is a representative graph 1400A showing two examples of fitting the focus curves 1401 and 1402, while Figure 14B is a diagram of a variable focus image stack 1400B that includes two different regions of interest ROI(k), specifically ROI(1) and ROI(2), which respectively correspond to and Figure 14AData points fm(1,i) and fm(2,i) corresponding to two different focus curves 1401 and 1402. The region of interest ROI(k) is included in the imaging surface region of the workpiece.

[0132] Regarding the term "region of interest", it should be understood that some "single-point" autofocus tools return a single Z-height corresponding to the entire region of interest. However, known "multi-point" type autofocus tools can return multiple Z-heights corresponding to individual "sub-regions of interest" (such as a grid of sub-regions of interest) within the global region of interest defined by the multi-point type autofocus tool. For example, such sub-regions of interest can be defined manually and / or automatically as centered on each (or most) pixel within the global region of interest. Thus, in some cases, ROI(1) and ROI(2) can be considered representative sub-regions of interest within the global region of interest. However, the key point is that Z-heights can be established for any defined autofocus region of interest (e.g., whether it is the region of interest of a single-point autofocus tool or a sub-region of interest within the global region of interest defined by a multi-point autofocus tool). Therefore, it should be understood that when using the term "region of interest" to establish Z-heights, sub-regions of interest (e.g., within the global region of interest defined by a multi-point autofocus tool) can be included within the meaning of this term. For the sake of simplicity in the current description, the regions of interest ROI(1) and ROI(2) are shown as being relatively small (e.g., 3x3 pixels), but it should be understood that larger regions of interest (e.g., 7x7 pixels, etc.) can be utilized in various embodiments as part of such processes, and so on.

[0133] As Figure 14B shown, each of images (1) - (11) of the image stack image (i) includes a region of interest ROI(1) located at the center, and the focus metric value determined for said region corresponds to the focus metric data point fm(1,i) on the focus curve 1401. Figure 14B The region of interest ROI(1) is schematically indicated in, which includes a relatively high contrast (e.g., in image (6)), corresponding to the relatively large focus metric value shown on the focus curve 1401. Similarly, each of images (1) - (11) of the image stack image (i) includes a region of interest ROI(2) located at the periphery, and the focus metric value determined for said region corresponds to the focus metric data point fm(2,i) on the focus curve 1402. Figure 14B The region of interest ROI(2) is schematically indicated in, which includes a relatively low contrast (e.g., in image (6)), corresponding to the relatively small focus metric value shown on the focus curve 1402.

[0134] As Figure 14AAs shown, each focus metric value fm(1,i) or fm(2,i) can be regarded as sampling the continuous underlying focus data 1401S or 1402S respectively. From Figure 14A it can be seen that the underlying focus data 1401S or 1402S is relatively noisy (e.g., due to the small size of the corresponding region of interest). However, with respect to the focus curve 1401, due to the relatively high contrast of the corresponding region of interest, the focus metric values near the peak of the focus curve (e.g., near Zp1401) are relatively large compared to the magnitude of the "noise component" in the underlying focus data. In contrast, with respect to the focus curve 1402, due to the relatively low contrast of the corresponding region of interest, the focus metric values near the peak of the focus curve (e.g., near Zp1402) are relatively similar compared to the magnitude of the "noise component" in the underlying focus data.

[0135] In a particular example, the relatively high focus metric values indicated in the focus curve 1401 may be at least partially attributed to the illumination provided by the illumination configuration on the surface area in the region of interest ROI(1), resulting in a high contrast of the focused image. In contrast, the relatively low focus metric values indicated in the focus curve 1402 may be at least partially attributed to the illumination provided by the illumination configuration on the surface area in the region of interest ROI(2), resulting in a low contrast of the focused image. In any case, it should be understood that due to the relatively high signal-to-noise ratio associated with the peak of the focus curve 1401 compared to the low "signal-to-noise ratio" associated with the lower peak of the focus curve 1402, the estimated Z height of the focus peak Zp1402 of the focus curve 1402 is more unreliable or more uncertain than the estimated Z height of the focus peak Zp1401 of the focus curve 1401 (e.g., in some cases, the data of the focus curve 1402 may be considered so unreliable and / or uncertain that the determination of the focus peak cannot be reliably performed, which can be regarded as a "gap" in the focus curve data corresponding to the workpiece surface).

[0136] Regarding Figure 14A and Figure 14BBriefly summarized, for the operation of taking a point from the focus, the camera 260 can be moved (e.g., by the motor 294), or the VFL lens 270 can be used to change the focus position of the system by a series of Z-height positions Z(i) along the z-axis (e.g., the focus axis or the optical axis OA), and the camera 260 can be used to capture an image (i) at each of a plurality of corresponding focus positions. For each captured image (i), a focus metric fm(k,i) can be calculated based on a region of interest or sub-region ROI(k) (e.g., a set of pixels) in the image, and associated with the corresponding focus position Z(i) along the Z-axis when the image was captured (e.g., the focus position of the camera or the focus position corresponding to the operation of the VFL lens). This produces focus curve data (e.g., the focus metric fm(k,i) at the position Z(i), which is a type of set of data for determining the focus peak), which can simply be referred to as the "focus curve" or the "auto-focus curve". In one embodiment, the focus metric value can involve the calculation of the contrast or sharpness of the region of interest in the image. In various embodiments, the focus value or curve can be normalized. Various focus metric calculation techniques are described in detail in the incorporated references, and those of ordinary skill in the art will also be aware of various suitable focus metric functions.

[0137] The Z-height corresponding to the peak of the focus curve (e.g., Zp1401 or Zp1402) (which corresponds to the best focus position along the Z-axis) is the Z-height for determining the region of interest of the focus curve. The Z-height corresponding to the peak of the focus curve can be found by fitting a curve (e.g., curve 1401 or 1402) to the focus curve data (e.g., data fm(1,i) or fm(2,i)) and estimating the position of the peak of the fitted curve. It should be understood that although the image stack graph (i) shown for illustrative purposes only includes eleven images, in an actual implementation (e.g., as part of a PFF process or otherwise), a greater number of images (e.g., 100 or 200 or more) can be used. Exemplary techniques for determining and analyzing image stacks and focus curves are taught in U.S. Patent Nos. 6,542,180 and 8,581,162, each of which is hereby incorporated by reference in its entirety. The differences between curves 1401 and 1402 and Z-heights Zp1401 and Zp1402, as well as the accuracy differences for determining the corresponding three-dimensional profile data, illustrate the importance of illumination optimization for determining the desired illumination for a focus picking process (e.g., for a focus picking process, the desired illumination in region of interest ROI(1) results in curve 1401 with Z-height Zp1401, while the less desired illumination in region of interest ROI(2) results in curve 1402 with Z-height Zp1402). It should also be noted that such characteristics / objectives of illumination optimization for focus picking (e.g., which includes providing the desired illumination over a range of focus positions rather than at a single focus position) may be different from the characteristics / objectives of illumination optimization for certain other types of processes (e.g., edge detection and / or defect detection processes, etc.).

[0138] Figures 15A to 15I is a diagram showing the determination of optimized illumination for different illumination optimization modes. Figure 15A shows an inspection system 100', which can be similar to the inspection system 100 described above with respect to Figure 2 and is understood to operate in a similar manner unless otherwise specified below. System 100' is shown as including an illumination configuration LC', a camera configuration 260 (e.g., which can include a lens), and a workpiece stage 210. A workpiece 20 is located on the workpiece stage 210 (i.e., illuminated by the illumination configuration LC' and imaged by the camera 260). In Figure 15A a simplified example, the illumination configuration LC' is shown as including at least illumination channels LX, LY, and LZ.

[0139] Notably, illumination channels LX and LZ provide illumination at an angle (e.g., such as an angle between 45 degrees and 90 degrees with respect to the optical axis (such as the optical axis of the camera 260 and / or a lens in front of the camera 260, etc.)). With respect toFigure 2 Examples of such illumination can come from certain illumination channels of the illumination configuration 230. The illumination channel LY provides axial illumination (e.g., relative to Figure 2 Examples, which may be similar to the illumination provided by the illumination channel 240). In various embodiments, Figure 15A In the illustrations of Figure 15A the illumination channels LX, LY, and / or LZ can represent individual illumination channels, or in some embodiments, each illumination channel can represent a group of illumination channels at the corresponding location, such as

[0140] Regarding the coaxial illumination provided by the illumination channel LY, although for simplicity Figure 15A of the description, the illumination channel LY is shown above the camera 260, in various embodiments, the axial illumination can be provided in a manner similar to that shown in Figure 2 the examples (e.g., where the reflective element 290 reflects the illumination from the illumination channel 240 downward along an axial path towards the workpiece 20, and the correspondingly generated workpiece light 255 travels along the optical path through the element 290 towards the camera 260). As Figure 15A shown, similar to Figure 2 the examples, the workpiece 20 is located on the workpiece table 210 (e.g., the position of the workpiece table 210 can be adjusted), and the workpiece 20 or its desired part is made to be within the field of view (e.g., within the field of view of the camera 260).

[0141] Figure 15B is a figure showing a side view of the workpiece 20 located on the workpiece table 210 similar to Figure 15A that. As Figure 15B shown, the workpiece 20 has inclined surfaces S1 and S3 and a horizontal top surface S2. An edge EA is shown between the surfaces S1 and S2, and an edge EB is shown between the surfaces S2 and S3. Figure 15C is a top view of the workpiece 20 located on the workpiece table 210. As Figure 15C shown, the edge EA is located between the inclined surface S1 and the horizontal top surface S2. The edge EB is located between the horizontal top surface S2 of the workpiece 20 and the inclined surface S3.

[0142] Figure 15D and Figure 15EFIG. is a diagram showing an embodiment in which a lighting optimization process is performed to determine lighting for illuminating workpiece 20 in a selected edge detection lighting optimization mode. More specifically, lighting is determined to enhance edges EA and EB (e.g., relative to determining their exact positions on workpiece 20 such that the edges EA and EB can be best detected, and in one example, the exact distance between the edges can thus be determined). In various embodiments, in order to detect edges EA and EB in an optimal manner for metrology purposes, ideally, the region on at least one side of each edge will have maximized contrast relative to the region on the other side of the edge, and there will not be a significant percentage of saturation of pixel intensities near the edge.

[0143] For such a purpose, lighting can be determined such that surface S1 is mostly dark (e.g., by making lighting channel LX have a low intensity), and surface S2 can be mostly bright but preferably not saturated (e.g., such that some surface texture is still visible) by making lighting channel LY have a medium or medium-high intensity, and surface S3 is mostly dark by making lighting channel LZ have a relatively low intensity. As a specific numerical example related to such a principle, in one embodiment, the lighting intensity settings for lighting channels LX and LZ can be 1%, while the lighting intensity setting for lighting channel LY can be 64%. It is noted that such optimized lighting intensities produce the desired result in which edges EA and EB are enhanced in the acquired result image (e.g., an image corresponding to Figure 15E the top view), thereby improving the detection of the edges.

[0144] As Figure 15E indicated, the images acquired with optimized lighting can be utilized to determine the exact positions of edges EA and EB. In various embodiments, edge detection tools (e.g., as described herein with respect to Figures 8 to 10 can be utilized to determine the exact positions of edges EA and EB. Once the exact positions of edges EA and EB are determined, the distance D3 between edges EA and EB can be determined, such as Figure 15E shown.

[0145] Figure 15F and Figure 15GAn embodiment is shown in which an illumination optimization process is performed to determine the illumination for illuminating the workpiece 20 for a selected defect detection illumination optimization mode. In this particular example, it is desired to detect defects DFT that may be present on the surface S1. In various embodiments, in order to optimally illuminate and detect defects on the surface S1, it may be desirable for the light mainly from the illumination channel LX to have a relatively high intensity to reflect the illumination from the inclined surface S1 back to the camera 260. In various embodiments, one goal is to be able to image the surface texture of the entire surface S1 well without underexposing or overexposing any part of the surface S1.

[0146] Regarding such a goal, the illumination from the illumination channel LZ may have little or less impact on the illumination of the surface S1 and can be optimized at a lower value (e.g., for energy conservation) or other possible values because the illumination from the illumination channel LZ has little impact on defect detection on the surface S1. In various embodiments, a certain portion of the illumination from the illumination channel LY may contribute to illuminating the surface S1 such that the illumination channel LY can have an optimized setting at a certain low to medium value. As some specific numerical examples related to the illustration of Figure 15G the illumination intensity settings of the illumination channels can correspond to 95% for the illumination intensity setting of the illumination channel LX, 10% for the illumination intensity setting of the illumination channel LY, and 0% for the illumination intensity setting of the illumination channel LZ.

[0147] Figure 15H and Figure 15I FIG. is a diagram showing an embodiment in which an illumination optimization process is performed to determine the illumination for illuminating the workpiece 20 for a selected take-point-from-focus illumination optimization mode. In the example shown, for the take-point-from-focus operation, it is desired to determine the three-dimensional contour data of the workpiece 20, such as the relationship with all the surfaces S1, S2, and S3. In various embodiments, regarding such a take-point-from-focus process, it may be desirable to generate illumination that achieves relatively balanced illumination on all the surfaces S1, S2, and S3 (e.g., such that no surface is saturated and no surface is very dark, etc.). Since the surfaces S1 and S3 are angled, as a general principle, more illumination may be required from the illumination channels LX and LZ (e.g., for this, corresponding images can be obtained using illumination that illuminates the surfaces S1 and S3 similarly compared to the surface S2). In various embodiments, it may be desirable to determine the illumination such that the texture on the surfaces S1, S2, and S3 is clearly visible in the corresponding images. According to such a principle, as a specific numerical example related to the illustration of the determined illumination of Figure 15I the illumination channels LX and LZ can have an illumination intensity setting of 75%, while the illumination channel LY can have an illumination intensity setting of 25%.

[0148] It should be understood that as part of the illumination optimization process, the settings of the illumination channels (e.g., the illumination intensity settings in the example corresponding to Figures 15A to 15I ) can be adjusted to determine the optimized illumination intensity. For example, when performing the illumination optimization process in relation to the selected edge detection illumination optimization mode, the process can be configured to make the surface on one side of each edge (e.g., surface S2) relatively bright and unsaturated, and make the surfaces on the other sides of the edge (e.g., surface S1 and surface S3) mostly darker. Regarding these principles, the illumination optimization process can change the settings of the illumination channels LX, LY, and LZ within the range corresponding to these principles, and for these illumination channels, different illumination settings can be used to capture images in order to determine the illumination that best enhances the edges EA and EB. In various embodiments, a quality metric (e.g., related to the enhancement of the edges) can be determined for each image and / or corresponding illumination setting in order to determine the illumination setting and corresponding image with the best metric. This can be done for Figures 15F to 15G (i.e., the defect detection illumination optimization mode) and Figures 15H to 15I (i.e., the take point from focus illumination optimization mode) for the illumination optimization process to perform similar illumination setting changes and utilization of quality metrics.

[0149] Figure 16 is a flowchart of a routine 1600 for determining the illumination optimization of a workpiece (e.g., a new workpiece). In block 1610, the workpiece for which illumination optimization is to be performed is determined. For example, the user may have a new workpiece for which illumination optimization needs to be determined. In various embodiments, the user can place the workpiece within the field of view of the system (e.g., Figure 2 below the camera 260 and lens 250 of

[0150] In block 1630, one or more elements or regions on the workpiece are determined for illumination optimization. As an example, the user may accept or indicate one or more elements or regions on the workpiece to determine illumination optimization. In various embodiments, such elements or regions may be automatically identified, segmented, and / or suggested. As some examples, two or more edges may be indicated for edge detection, or a region that may contain defects may be indicated for surface defect detection. An option to select all or part of these automatically identified regions may be presented to the user. In various embodiments, the elements and / or regions may be manually selected by the user. Some manual selection techniques may include options such as clicking on the perimeter of a feature using a polygon tool, semi-automatically selecting an edge using a live line tool, swiping a finger or drawing with a pen on a touchscreen device, or simply dragging a mouse over a rectangular area. In various embodiments, for an edge detection illumination optimization mode, geometric elements may be automatically suggested or manually identified. For a defect detection illumination optimization mode or a focus taking point illumination optimization mode, regions may be automatically segmented and suggested and / or may be manually identified.

[0151] In block 1640, illumination variables are determined. In various embodiments, a selection may be made (e.g., by the user) to optimize all possible illumination variables, or a selection may be made to optimize illumination to be constrained to fewer variables (e.g., defined by layout, element grouping, and / or other limiting options, such as selected in a graphical user interface, etc.). As an example, multiple sets of illumination channels may be determined, and each set of illumination channels may be controlled as a group so as to share the same settings (e.g., RGB settings), such as as part of the illumination optimization process. By including illumination channels in a group and adjusting each set of illumination channels as a single entity, the illumination optimization process may be made faster and / or easier by reducing the effective number of entities to be adjusted.

[0152] In block 1650, an indication to perform illumination optimization is received, and / or otherwise it is determined to continue determining and providing illumination. In various embodiments, such an indication may be provided by a selection made by the user in the user interface (e.g., pressing a start button) and corresponds to the selection. In various embodiments, a progress indicator may be provided in the user interface (e.g., which indicates to the user that illumination optimization determination / training is in progress).

[0153] At block 1660, one or more determined candidates for illumination are displayed or otherwise provided. For example, in various embodiments, upon completion of training / determination of illumination, one or more candidates for illumination (e.g., having metric target results indicators) may be displayed or otherwise provided (e.g., for user consideration). In various embodiments, if the system determines that there is a clear best result based on the metric target, only that one result may be provided and / or otherwise displayed. In various embodiments, options (e.g., for user selection) may be provided to select a preferred result from two or more options or to accept the single presented best result.

[0154] In various embodiments, the user may choose to obtain the results and further manually adjust the illumination settings to see if different results can be achieved (e.g., may be better for certain aspects) (e.g., and for these results, the displayed metric targets may be updated and displayed based on the corresponding changes made manually). In various embodiments, metric target results may be determined and displayed for each individual region as well as for collections of regions (e.g., all regions). In various embodiments, metric targets may be predefined for each illumination optimization mode. For example, for an edge detection illumination optimization mode, the metric target may be related to how well the determined edges match the profile of an ideal step edge, etc.

[0155] At block 1670, related items are generated and / or saved. The following are some examples of some of the related items that may be generated and / or saved. One example is a trained or optimized illumination model that includes all of the layout / groups / channels and / or other settings (e.g., in some cases including some settings made or selected by the user in a graphical user interface). Another example may include workpiece features (e.g., derived from an image or point cloud according to processing and configuration) for future automatic identification of similar workpieces. Another example may include information about the workpiece's position and orientation. Another example may include an image of the workpiece with optimized illumination (e.g., along with the date of the image, etc.). Another example may include the illumination optimization mode selected for optimization. Another example may include a metric or score associated with the quality of the illumination optimization of the selected illumination optimization mode. Another example may include the regions on the workpiece where illumination optimization is achieved relative to the workpiece's position and orientation (e.g., such that the same regions can be automatically found on new workpieces placed in different positions and orientations within the system's field of view (such as within the field of view of camera 260)). Another example may include an image of the workpiece with neutral illumination (e.g., such as used as a workpiece identification image for subsequent user viewing, etc.).

[0156] At decision block 1680, it is determined whether additional illumination optimization is to be determined for the same workpiece in the same illumination optimization mode. For example, a user may specify one or more different elements or regions on the workpiece for additional illumination optimization. If additional illumination optimization is to be performed, the routine returns to block 1630. At decision block 1690, it is determined whether additional illumination optimization is to be performed for the same workpiece and a different illumination optimization mode. If additional illumination optimization is to be performed, the routine returns to block 1620; otherwise, the routine ends.

[0157] As an example, in various embodiments, a single illumination optimization may not optimally illuminate all surfaces / edges of a workpiece in the manner desired by the selected illumination optimization mode. In such a case, for a single illumination optimization mode selected for illumination optimization, two or more illumination optimizations may be determined for the same workpiece (e.g., based on instructions from a user). For example, in an edge detection illumination optimization mode, there may be a first determined illumination for horizontal edges on the workpiece and a second determined illumination for vertical edges on the workpiece.

[0158] Figure 17 is a flowchart of routine 1700 for providing illumination optimization for a workpiece (e.g., in the case where the workpiece corresponds to a workpiece saved in a database, such as a workpiece for which illumination optimization has been determined by routine 1600 according to Figure 16 ). At block 1710, the workpiece is determined. For example, in various embodiments, a user may place the workpiece in the field of view of the system (e.g., below camera 260), and for the workpiece, the system may automatically sense the workpiece and image it. In various embodiments, such sensing and / or imaging results may be used to automatically determine which workpiece the current workpiece corresponds to (e.g., in an existing set of workpiece databases). In various embodiments, if there is some uncertainty (e.g., matching the current workpiece to a workpiece in the database), a neutral illumination image of the best-matching workpiece in the database (such as presenting it to the user) may be presented so that the user can select the matching workpiece from the database.

[0159] In block 1720, the illumination optimization mode and model for the workpiece are determined. The model is an optimization framework that contains adjustable parameters that assist in optimizing the control of illumination elements. During model optimization, the parameters are changed by an optimization routine based on traditional or machine learning / artificial intelligence methods to achieve illumination of workpiece image characteristics that result in the closest predefined mode-specific optimal goal. In various embodiments, with respect to the current workpiece corresponding to a workpiece saved in the database, if there is only one illumination optimization mode and model for the saved workpiece, the saved illumination optimization mode and model may be automatically loaded / used.

[0160] If more than one illumination optimization mode and model is trained for a saved workpiece, then options may be provided. For example, a user may be provided with the option to select one or more of the illumination optimization modes and models (e.g., for sequential imaging, or a process may be set up for it to acquire images for each illumination optimization mode and model and automatically process the acquired images accordingly). In various embodiments, if a saved workpiece has two different illumination optimization modes and models, such as for edge detection and defect detection, then a first image may be acquired using illumination for edge detection (e.g., which may be sent to or otherwise utilized by an edge detection process, such as a metrology algorithm for edge determination and size / distance measurement, etc.), and illumination for defect detection may be provided to acquire a second image (e.g., which may be sent or otherwise provided to a defect detection process, etc.).

[0161] In block 1730, the workpiece position and orientation are determined. For example, the workpiece position and orientation may be determined (e.g., automatically sensed) and compared to the position and orientation of the corresponding workpiece stored in a database (e.g., based on parameters used during model training). At decision block 1740, a determination is made as to whether adjustments need to be made with respect to the position and orientation of the workpiece. For example, if it is determined that the orientation and position of the current workpiece matches the position and orientation of the workpiece stored in the database, then no adjustments may need to be made, in which case the routine proceeds to block 1760, as described in more detail below.

[0162] If it is determined at decision block 1740 that adjustments are needed, the routine proceeds to block 1750 where adjustments are determined and performed. For example, if the workpiece position and orientation are different than the workpiece position and orientation stored in the database, certain adjustments may be determined and performed. For example, the database and / or process may have a physical model of the lighting and each of its elements in terms of relative positions (e.g., such as Figure 3A and Figure 3C As an example, if the lighting element is about the optical axis (e.g., Figure 2 If the workpiece is cylindrically symmetric about the optical axis of the camera 260 (e.g., the optical axis of the camera 260), the illumination model can be automatically rotated to match the current orientation of the workpiece. This may be sufficient in some cases, such as if it is determined that the workpiece has not shifted from its original center (e.g., corresponding to the center of mass or geometric center, etc.) position. If the workpiece is sensed to have also shifted from the workpiece position saved in the database, suggestions can be provided to the user to make adjustments (e.g., transfer learning of the illumination model, such as starting with a rotated model and then adjusting the illumination through a series of training steps to improve / adjust to account for the workpiece shift). In some embodiments, a re-optimization of the illumination can be determined for a different position and orientation of the current workpiece.

[0163] Once the adjustment is performed at block 1750, or if it is determined that no adjustment is required at block 1740, the routine proceeds to block 1760 where illumination for the workpiece is provided (e.g., to illuminate the workpiece for a corresponding inspection operation, etc.). At block 1770, one or more images of the workpiece are acquired (i.e., images illuminated by the illumination). At block 1780, an inspection operation is performed on the workpiece (e.g., may correspond to the selected illumination optimization mode and may include metrology or inspection operations such as edge detection, defect detection, or taking points from focus, etc.). In various embodiments, such an inspection operation may be performed as part of a corresponding process (e.g., performed by one of parts 144, 145, or 146 of Figure 2 , etc.).

[0164] Figure 18 FIG. 1800 is a representation of certain options available in a system related to illumination optimization. In various embodiments, the options described below may correspond to selection options provided in a user interface, selection options provided to the user in other ways, and / or options that may be automatically performed by the system, etc. As Figure 18 shown, the various options include an illumination channel option 1810, a region of interest option 1820, an optimization method option 1830, a model training option 1840, a training configuration option 1850, and a result / suggestion option 1860. Regarding the illumination channel option 1810, in various embodiments, an option may be provided to the user to select / choose certain desired illumination channels of a lighting configuration. In various embodiments, the user may also have an option to keep the lighting settings in the current state (e.g., as part of a default setting, etc.). In some cases, all illumination channels will be used to determine illumination optimization.

[0165] Regarding the region of interest option 1820, in various embodiments, options corresponding to specific regions of interest on the workpiece are provided (e.g., for the user to select). In various embodiments, an option may also be provided to not select any specific region of the workpiece (e.g., not select any specific region in the image of the workpiece) (e.g., for the user to select). If no specific region is selected, the illumination may be optimized for the entire workpiece or a part of the workpiece (e.g., contained in the image). Regarding the optimization method option 1830, an option may be provided to select from more than one optimization method. For example, certain different optimization methods may utilize different algorithms, different learning, etc. to determine optimization, such as conventional optimization methods or machine learning / artificial intelligence methods.

[0166] Regarding model training option 1840, options can be provided for selecting a pre-trained model (e.g., stored in a database). In various embodiments, options can be provided for altering (e.g., fine-tuning) an existing model (e.g., stored in a database) to attempt to further optimize the illumination of the workpiece. Regarding training configuration option 1850, after an option to alter (e.g., fine-tune an existing model) is selected, options can be provided for selecting certain parameters (e.g., such as the number of training epochs, such as corresponding to training time, etc.). In various embodiments, an option can also be provided to leave these parameters unchanged and retain the current configuration as the default setting.

[0167] Regarding result / suggestion option 1860, in various embodiments, a quality metric can be provided (e.g., displayed in a user interface, such as for user review). In the case where it is determined that the quality metric is below an expected value (e.g., below a specified threshold), certain suggestions can be provided (e.g., displayed in a user interface, such as for user review). For example, the user can be requested to select an area of interest where the user desires to obtain better illumination quality. As another example, the suggestion can indicate that if a pre-trained model is used, fine-tuning the model may yield better results. As another example, the suggestion can indicate that if the model has been fine-tuned, then longer model training or optimization may improve the results.

[0168] Figure 19 is a flowchart showing a routine 1900 for performing an illumination optimization process. At block 1910, options are provided for selecting an illumination optimization mode, which is at least one of an edge detection illumination optimization mode, a defect detection illumination optimization mode, or a take-point-from-focus illumination optimization mode. In various embodiments, there may be a most commonly used or user-selected default mode that is automatically selected (e.g., unless the user or system selects or indicates otherwise). At block 1920, a selection of the illumination optimization mode is received. At block 1930, an illumination optimization process is performed based on the selected illumination optimization mode, where the illumination optimization process determines the illumination for illuminating the workpiece, and the determined illumination includes settings for each illumination channel of the illumination configuration.

[0169] Regarding the edge detection process, it is generally desirable to provide illumination such that any resulting image and the edges of interest have high contrast (e.g., such as having low luminance values on one side and higher luminance values on the other side). Generally, it is desirable for this transition to occur over the minimum number of pixels (e.g., similar to a step function), which results in a large maximum value for the gradient of the scan line passing through the edge (e.g., such as Figure 10as shown by the data 1026). In various embodiments, these conditions can be determined and / or further refined by an optimization objective (e.g., used in the illumination optimization process) that penalizes saturated pixels (i.e., full-brightness pixels) in the region around the edge, such as may result in the gradient curve having more of a Gaussian-type shape because the brightness on one side of the edge is unnaturally clipped at the maximum brightness.

[0170] Regarding the defect detection process, in some embodiments, the requirements for determining optimized illumination may be more variable than the relatively more well-defined requirements of the edge detection process. In some cases, the optimized illumination for defect detection may be at least partially a function of the type of defect being detected. As an example, for a flat surface with raised or recessed defects (e.g., scratches, dents, bumps), the optimized illumination (e.g., the best illumination) can come from the side (e.g., at an angle relative to the optical axis of the system, and in some embodiments can be referred to as dark-field illumination). In contrast, for defects that may not have such features, such as contaminants on the surface, which are of minimal thickness and may not be well illuminated with dark-field illumination, more bright-field illumination may be required in order to be able to view surface texture, brightness, and / or color, etc. In some embodiments, the illumination optimization process can utilize techniques for differentiating such defects and / or can receive user input regarding the type of defect.

[0171] In various embodiments, as part of the illumination optimization process performed in relation to the defect detection process, it may be desirable to receive user input regarding the workpiece surface and / or defect characteristics of the defect to be detected. In some embodiments, the defect detection process can be trained based on examples provided by the user and indicating which pixels in an image of the workpiece represent the defect. For example, the user can indicate whether the defect has certain characteristics related to the workpiece surface (e.g., similar to a bump, or similar to a scratch or pit), or alternatively, whether the defect is more likely related to surface texture or other features. As some other examples, the user can indicate whether the workpiece surface is highly reflective (e.g., polished metal) or less reflective (e.g., plastic, rough machined metal, etc.), and whether the workpiece has a flat or curved surface. In various embodiments, the system can be trained to determine the optimized illumination for defect detection by placing the workpiece in a field of view having the indicated exemplary defect and / or having pixels corresponding to the indicated defect. In such embodiments, the defect detection process can include changing the settings of the illumination channels to determine which settings produce the best contrast and visibility for the specified defect.

[0172] In various embodiments, an indication of which pixels correspond to defects can be provided in various ways (e.g., using an automatic segmentation tool, painting the defective pixels, placing a bounding box around the defect, drawing a contour around the defect, etc.). In some embodiments, the defect detection process can provide feedback to the illumination optimization process. In certain embodiments, such feedback can include running the defect detection process using the illumination suggested by the illumination optimization process and providing feedback thereon that indicates the current ability of the defect detection process to accurately detect defects using the suggested illumination.

[0173] In various embodiments, a user can select an illumination optimization mode and perform an illumination optimization process for the selected illumination optimization mode. Once the illumination optimization mode is selected, the illumination optimization process can automatically determine the optimized illumination of the workpiece within the field of view. In various embodiments, the illumination optimization process can proceed from a random or preset configuration, or from a pre-trained model. In some embodiments, the system can automatically suggest a pre-trained model based on a certain similarity of the workpiece to existing workpieces stored in the database or according to the user's selection. In various embodiments, the illumination optimization process can automatically determine and / or suggest regions of interest for illumination optimization (e.g., for such regions of interest, the illumination optimization process can be configured to automatically find edges and / or automatically find surfaces, etc.). In such embodiments, the system can be configured to enable the user to automatically select the determined elements of interest (e.g., edges, surfaces, etc.), or for these elements, the user can accept all the determined elements. In various embodiments, the user can accept one or more of the elements and then potentially enhance the determined elements by manually entering additional elements or regions of interest.

[0174] In various embodiments, the system can be configured to enable the user to input a region of the workpiece (e.g., an edge, a surface, etc.) for which the illumination optimization process can be applied. In some embodiments, the illumination optimization process can, in at least some cases, attempt to determine the optimized illumination for the entire image or the entire part of the workpiece containing the workpiece for the selected illumination optimization mode. In various embodiments, the system can be configured to learn to identify the workpiece (e.g., based on imaging and / or other sensing, such as 3-D sensing, etc.) and automatically load the optimized illumination model / settings for the corresponding workpiece stored in the database, such as whenever the workpiece is placed within the field of view of the system (e.g., under the camera 260, etc.).

[0175] In various embodiments, the lighting optimization process can be configured to detect the relative rotational orientation of the workpiece and automatically rotate / adjust the optimized lighting model / settings based on the knowledge of the arrangement of the lighting channels in the lighting configuration. In various embodiments, the lighting optimization process and / or system can be configured to automatically determine when the position, orientation, or geometry of the workpiece is likely to have changed somewhat relative to when the original lighting optimization process was performed on the workpiece and saved in the database, and then recommend an automatic adjustment or re-optimization of the optimized lighting model, etc. In various embodiments, transfer learning can be utilized on the optimized lighting model to quickly adjust the lighting for new identical or similar workpieces to accommodate minor changes in the workpiece position, orientation, or geometry. In various embodiments, the lighting optimization process and / or system can display the optimized lighting applied to the workpiece and the indicated optimization score to the user, and for this, can also allow the user to manually adjust the lighting settings while providing an updated optimization score during the adjustment and displaying it to the user.

[0176] In various embodiments, the system can be configured to perform the lighting optimization process for each of multiple lighting optimization modes selected by the user and / or for multiple regions selected for the lighting optimization process, in each case, sequentially imaging the workpiece illuminated by the determined lighting. In various embodiments, each image obtained for a different lighting optimization mode can be automatically sent to an appropriate post-processing section (e.g., an image of lighting optimized for edge detection can be provided to an edge detection process, an image obtained for lighting optimized for defect detection can be provided to a defect detection process, and / or an image obtained for lighting optimized for focus taking can be provided to a focus taking process). In various embodiments, the system can be configured to enable the user to control independent lighting variables for the lighting optimization process. For example, the user can specify or select certain groups of lighting channels, or can limit color adjustment to only include selected RGB lighting options, etc.

[0177] In various embodiments, when the user selects the defect detection lighting optimization mode, the user can provide at least one example of a workpiece having a defect. The user can indicate the area of the defect (e.g., using a bounding box, pixel drawing, indicating the boundary of the defect, using a tool that attempts to automatically segment the defect area after the user selects points within the defect area, etc.). In this way, the lighting optimization process can receive an indication of which pixels in the image constitute the defect, and thus the lighting optimization process can optimize the lighting to maximize the contrast of the area containing the defect on a particular workpiece surface (e.g., for the current workpiece and future similar workpieces). In various embodiments, the user can provide at least some information about the characteristics of the defect to be detected and / or about the characteristics of the workpiece surface.

[0178] In various embodiments, the illumination optimization process can be configured to create surface illumination with maximized texture or darkfield illumination of a surface of interest based on information provided by a user regarding the characteristics of the defect to be detected. In various embodiments, the illumination optimization process can be configured to communicate with a defect detection process that is determining optimized illumination, and to that end, the defect detection process can be configured to provide feedback regarding the proposed illumination. For example, the defect detection process can receive an image of a workpiece with the proposed illumination and can utilize the proposed illumination image to attempt to detect a defect and can provide feedback (e.g., data) indicating the accuracy with which the defect detection process was able to find the defect using the proposed illumination image (e.g., corresponding to a known defect region, etc.).

[0179] In various embodiments, a set of images can be acquired for use by the illumination optimization process, for which each illumination channel, each set of illumination channels, etc. to be used as part of the illumination optimization process is on at least one of the images. In various embodiments, the system can include a light source configuration in which one or more illumination channels can be physically moved via electronic control (e.g., such as with respect to the height or directionality of the illumination channel), and to that end, the illumination optimization process is configured to determine the settings of the illumination channels, which can include determining the height and / or directionality of the illumination channels. In various embodiments, the illumination optimization process can be configured to utilize negative RGB weights, including subtracting one image from another image.

[0180] In various embodiments, when an edge detection illumination optimization mode is selected, the illumination optimization process can be configured to determine illumination that results in a high-contrast step edge transition (e.g., to support an accurate and repeatable determination of the edge position, such as for determining measurements related to the edge position, such as the distance between edges, etc.). In various embodiments, when a take points from focus illumination optimization mode is selected, the illumination optimization process can be configured to determine illumination that results in high texture for all surfaces of interest while minimizing saturation.

[0181] Various exemplary embodiments of the present disclosure are described below, where various features and elements are labeled with reference characters (i.e., reference numbers and reference letters) in the figures described herein. It should be understood that the addition of reference characters is for the purpose of indicating exemplary embodiments, and the features and elements are not limited to the specific embodiments shown in the figures.

[0182] According to one aspect, a system 100 is provided that includes a lens 250, a camera 260, an illumination configuration LC, one or more processors 125, and a memory 140. The lens 250 (e.g., an objective lens) is configured to input image light 255 from a workpiece 20, where the lens 250 is configured to transmit the image light 255 along an imaging optical path OPATH and has an optical axis OA. The camera 260 is configured to receive the image light 255 transmitted along the imaging optical path OPATH and provide an image of the workpiece 20. The illumination configuration LC includes an illumination channel LCH (e.g., including at least five illumination channels, or at least ten illumination channels, or at least twenty illumination channels, etc. in some embodiments) configured to illuminate the workpiece 20 to generate the image light 255.

[0183] The memory 140 is coupled to the one or more processors 125 and stores program instructions that, when executed by the one or more processors 125, cause the one or more processors 125 to at least: provide a representation of multiple sets of illumination channels in a display area; determine that a set of illumination channels has been selected, where the selected set of illumination channels includes a plurality of illumination channels; and display the current illumination settings of the selected set of illumination channels, where an adjustment to the illumination settings of a set of illumination channels is applied to all of the illumination channels in the set.

[0184] As an example of such a feature, in the illustration of Figures 3A to 3C the display area 320 includes a "group list", and the column 320A includes label indicators for the currently displayed set of groups, and for each of these label indicators (e.g., "group 1", "group 2", etc.) represents a corresponding set of illumination channels. As another example, the light source display in the display area 360 may also include a representation of one or more sets of illumination channels. For the selection of a set of illumination channels, in various embodiments, a user can click or otherwise select a display representation of a set of illumination channels (e.g., the user may have clicked or otherwise selected the "group 3" representation in the display area 320), in response to which the system determines that the corresponding set of illumination channels has been selected. Then the settings of the selected set of illumination channels can be displayed (e.g., the system displays the settings of the selected "group 3" in the main settings of the display area 350, both in the column 351D of the corresponding values indicating different light colors and in the slider positions in the selection areas 351A, 351B, and 351C, and as described above, similar settings can be displayed in the display area 370). The sliders in the selection areas 351A, 351B, and 351C can be characterized as adjustment elements provided by the system and enabling adjustment of the illumination settings of the selected group, and such an adjustment to the settings of a set of illumination channels is applied to all of the illumination channels in the set.

[0185] In various embodiments, the program instructions, when executed by one or more processors, may also cause the one or more processors to perform steps such as those described below (and / or may otherwise perform such steps). In various embodiments, the steps described above and / or below (e.g., which are characterized as being performed by one or more processors or otherwise) may also or alternatively be performed as part of a method, and / or the system may be configured to perform these steps.

[0186] In various embodiments, adjustment elements (e.g., sliders in selection regions 351A, 351B, and 351C) may be provided that are configured to be able to adjust the lighting settings of the selected group. In various embodiments, a visual representation of the lighting configuration (including the lighting channels of the lighting configuration) may be displayed (e.g., as provided in display region 360). The lighting channels of the currently selected group may be indicated as being selected in the visual representation of the lighting configuration (e.g., according to the selection indicator SI of lighting channel LCH). Selection of individual lighting channels in the visual representation of the lighting configuration may be enabled, and addition of the selected lighting channels to the currently selected group of lighting channels or deletion of the selected lighting channels from the currently selected group of lighting channels may be enabled (e.g., the selected lighting channel LCH in display region 360 may be added to or deleted from the currently selected group).

[0187] In various embodiments, a representation of a lighting layout may be provided (e.g., the display area 310 includes a "layout list", and the column 321A includes label indicators for the currently displayed set of lighting layouts, each of the label indicators (e.g., "Layout 1", "Layout 2", etc.) being a representation of the corresponding lighting layout), where at least some of the lighting layouts each include multiple sets of lighting channels (e.g., the currently selected "Layout 1" is indicated to include 12 sets of lighting channels). In various embodiments, it may be determined that a lighting layout has been selected (e.g., in various embodiments, a user may click or otherwise select the representation of the displayed lighting layout, such as the user clicking or otherwise selecting the "Layout 1" representation in the display area 310, in response to which the system determines that the corresponding lighting layout has been selected); and a representation of the multiple sets of lighting channels included in the selected lighting layout may be displayed (e.g., the representation of the multiple sets of lighting channels included in the selected "Layout 1" is displayed in column 320A of the display area 320). A selected set of lighting channels of the lighting layout may be deleted from the lighting layout (e.g., the selection area 328 provides an option to delete the selected set). In various embodiments, one or more parameters of the selected lighting layout may be displayed (e.g., in the display area 330), and adjustment of one or more parameters of the selected layout may be enabled (e.g., one such parameter may be the number of segments, and the selection area 331 in column 330A indicates that the current number of segments is set to 5 segments and includes up and down arrows for increasing or decreasing, in this case for increasing or decreasing the number of segments).

[0188] In various embodiments, saving of the lighting settings of a lighting layout may be enabled, and invocation of the saved lighting settings of a lighting layout may be enabled. In various embodiments, the lighting layout and the lighting settings may be saved in different files. As a specific example, for a layout stored in a file named "RingLight", the lighting settings may be stored in files named "RingLight_scheme001", "RingLight_scheme002", etc. In various embodiments, each file (e.g., named "RingLight_scheme001", "RingLight_scheme002", etc.) may include a set of lighting settings for the corresponding layout. From this example, it can be understood that there may be multiple sets of lighting settings for a lighting layout, and in some embodiments, each set of lighting settings may be stored in a different file.

[0189] In various embodiments, adjustment elements are provided that enable at least one of rotation, expansion, contraction, or orientation of the representation of a set of illumination channels (e.g., the "expand", "rotate", or "orient" controls for the respective selection regions 362, 363, or 364). The adjustment may correspond to maintaining the same shape of the set of illumination channels, but when the shape is rotated, expanded, contracted, or oriented, at least some different illumination channels are included in the shape.

[0190] In various embodiments, the illumination settings are configured to be adjustable, including adjustment for one or more positive or negative color illumination settings (e.g., in the example shown in display regions 350 and 370, the red illumination setting is indicated as having a negative weight, corresponding to a value of -0.50). In various embodiments, embodiments of one or more negative color illumination settings include: obtaining two images, including a first image having one or more positive color illumination settings (e.g., having a 0.30 green illumination setting and a 0.15 blue illumination setting) and a second image having the absolute value of one or more negative color illumination settings (e.g., having the absolute value of a red illumination setting or a value of 0.50); and subtracting the second image from the first image. In various embodiments, an image corresponding to the pixel values obtained by subtracting the second image from the first image may be displayed, where pixels corresponding to negative values in the displayed image are displayed in a representation indicating the correspondence to the negative values (e.g., in the Figure 6 example, four negative weight indicator areas NWIA are shown because each area corresponds to an area where at least one component of the displayed color is negative).

[0191] In various embodiments, an image of a workpiece obtained by a camera using the current illumination settings is displayed, where the image is displayed in a workpiece display area (e.g., in display area 380). A representation corresponding to at least one of an edge detection operation or a defect detection operation on a corresponding area of the workpiece may be provided in the workpiece display area (e.g., an edge detection tool EDT or a defect detection tool DDT).

[0192] According to another aspect, a method of operating the system is provided. The method includes: providing a representation of multiple sets of illumination channels in a display area; determining that a set of illumination channels has been selected, where the selected set of illumination channels includes multiple illumination channels; and displaying the current illumination settings of the selected set of illumination channels, where adjustment of the illumination settings for a set of illumination channels is applied to all of the illumination channels in the set.

[0193] According to another aspect, the system is configured to: provide a representation of multiple sets of illumination channels in a display area; determine that a set of illumination channels has been selected, wherein the selected set of illumination channels includes a plurality of illumination channels; and display the current illumination settings of the selected set of illumination channels, wherein an adjustment to the illumination settings of a set of illumination channels is applied to all of the illumination channels in the set.

[0194] According to another aspect, system 100 is provided as including a lens 250, a camera 260, an illumination configuration LC, one or more processors 125, and a memory 140. The memory 140 is coupled to the one or more processors 125 and stores program instructions that, when executed by the one or more processors 125, cause the one or more processors 125 to at least: provide an option for selecting an illumination optimization mode, the illumination optimization mode being at least one of an edge detection illumination optimization mode, a defect detection illumination optimization mode, or a pick-off point from focus illumination optimization mode; receive a selection of the illumination optimization mode; and perform an illumination optimization process based on the selected illumination optimization mode, wherein the illumination optimization process determines the illumination for illuminating the workpiece, and the determined illumination includes settings of the illumination channels of the illumination configuration. In various embodiments, the option may be used to select an illumination optimization mode from a set of illumination optimization modes (e.g., the set includes at least two of an edge detection illumination optimization mode, a defect detection illumination optimization mode, and / or a pick-off point from focus illumination optimization mode, and wherein different modes correspond to different illumination determinations). In various embodiments, the option for selecting the illumination optimization mode may be provided in a user interface (e.g., in the Figure 3C display area 390).

[0195] In various embodiments, an edge detection illumination optimization mode is used to optimize illumination for detecting one or more edges on a surface of a workpiece (e.g., including determining illumination configured to create a high-contrast step edge transition to support accurate and repeatable determination of edge positions, and for such illumination, contrast curve analysis can be utilized to determine edge positions, and for such illumination, the distance between edges can be determined). In various embodiments, a defect detection illumination optimization mode is used to optimize illumination for detecting one or more defects on a surface of a workpiece (e.g., including determining illumination configured to create surface illumination with maximized texture or dark field illumination of a surface of interest, depending on the characteristics of the defect to be detected, etc.). In various embodiments, a focus taking point optimization illumination optimization mode is used to optimize illumination for determining three-dimensional profile data of a surface of a workpiece (e.g., including determining illumination configured to create balanced illumination on different regions / surfaces of the workpiece, and for such illumination, at least one image stack including a plurality of images of the workpiece can be acquired, where each image of the image stack corresponds to a different focus position along the optical axis).

[0196] In various embodiments, program instructions, when executed by one or more processors, can also cause the one or more processors to perform steps such as those described below (and / or can otherwise perform such steps). In various embodiments, the steps described above and / or below (e.g., which are characterized as being executed by one or more processors or otherwise) can also or alternatively be performed as part of a method, and / or the system can be configured to perform these steps.

[0197] In various embodiments, one or more elements or regions on a workpiece can be determined for illumination optimization. In various embodiments, illumination variables can be determined for the illumination optimization process. In various embodiments, the results of the illumination optimization process can be displayed as including one or more determined candidates for the illumination used to illuminate the workpiece.

[0198] In various embodiments, it can be determined to perform additional illumination optimization for the same workpiece and the same illumination optimization mode, but for different one or more elements or regions on the workpiece, and in this regard, the illumination optimization process is correspondingly performed on the different one or more elements or regions on the workpiece. In various embodiments, it can be determined to perform additional illumination optimization for the same workpiece and different illumination optimization modes, and in this regard, the illumination optimization process can be correspondingly performed based on the different illumination optimization modes.

[0199] In various embodiments, an illumination optimization model can be saved for a workpiece, and for the workpiece, the illumination optimization model includes settings (e.g., a set of settings) of illumination channels of an illumination configuration corresponding to a determined illumination. In various embodiments: a second workpiece (e.g., placed in the field of view of the system) can be compared with the workpiece and the similarity of the second workpiece can be determined; it can be determined (e.g., based on the similarity of the second workpiece) that the saved illumination optimization model will be used to illuminate the second workpiece; and the saved illumination optimization model can be invoked to provide illumination for the second workpiece. In various embodiments: the position and orientation of the second workpiece can be determined; it can be determined (e.g., based on the position and / or orientation of the second workpiece) that the illumination needs to be adjusted according to the position and orientation of the second workpiece; and corresponding adjustments to the illumination can be provided based on the position and orientation of the second workpiece.

[0200] In various embodiments, the second workpiece can be illuminated, and one or more images of the second workpiece can be acquired under the illumination. In various embodiments, one or more inspection operations can be performed on the second workpiece, and the one or more inspection operations on the second workpiece correspond to an illumination optimization mode selected for the illumination optimization process. In various embodiments (e.g., where an edge detection illumination optimization mode is selected), one or more inspection operations can include edge detection for determining the positions of one or more edges on the second workpiece (e.g., and the one or more inspection operations on the second workpiece can also include determining the distance between two edges on the second workpiece). In various embodiments (e.g., where a defect detection illumination optimization mode is selected), one or more inspection operations include defect detection for detecting defects on the second workpiece. In various embodiments (e.g., where a focus picking illumination optimization mode is selected), one or more inspection operations include focus picking for determining three-dimensional contour data of the second workpiece.

[0201] In various embodiments, a user is able to control independent illumination variables used as part of the illumination optimization process. For example, the user is able to determine multiple sets of illumination channels, each set of which will be controlled / adjusted as a single entity; determine the range within which color adjustment can be made; limit color adjustment to only selected RGB colors, and so on.

[0202] In various embodiments, when the defect detection illumination optimization mode is selected, information about a workpiece having a defect can be received from a user, where the user indicates the area of the defect, and for the area, the illumination optimization process utilizes the information to provide illumination that maximizes the contrast of the area. For example, the user can provide at least one example of a workpiece having a defect, where the user indicates the area of the defect (e.g., such as by a bounding box, pixel drawing, indicating the defect boundary, or using a tool that attempts to automatically segment the defect area if the user clicks inside the defect area), such that the illumination optimization process can know which pixels in the image constitute the defect and can optimize the illumination to maximize the contrast of that area on the specific workpiece surface and future similar workpieces.

[0203] In various embodiments, when the defect detection illumination optimization mode is selected, information about at least one of the characteristics of a defect or a surface of a workpiece can be received from the user, and the characteristics of the workpiece are utilized by the illumination optimization process. In various embodiments, the characteristics are used to determine whether the optimized illumination corresponds to surface illumination with maximized texture or dark field illumination of at least one surface of interest. In various embodiments, the illumination optimization can be a function of the type of defect to be detected. For example, for a flat surface having raised or recessed defects (e.g., scratches, dents, protrusions, etc.), the optimized illumination may correspond to dark field illumination (e.g., corresponding to illumination from the side, such as at an angle between 45 degrees and 90 degrees with respect to the optical axis OA). In contrast, defects that do not have such characteristics (e.g., contaminants or other defects having a minimum thickness on the surface) may not be well illuminated by dark field illumination and may require surface illumination with maximized texture (e.g., bright field illumination capable of viewing surface texture, brightness, or color, for which such bright field illumination may be at an angle between 0 degrees and 45 degrees with respect to the optical axis OA). In various embodiments, the illumination optimization process is configured to communicate with a defect detection process that evaluates the proposed illumination and provides feedback regarding the accuracy with which the defect detection process can detect a known defect area illuminated by the proposed illumination using the proposed illumination.

[0204] In various embodiments, an image set can be collected for a lighting optimization process, and for the image set, each lighting channel to be optimized is on in at least one of the images. In various embodiments, one or more of the lighting channels of a lighting configuration can be movable lighting channels, and for the movable lighting channels, each movable lighting channel is configured to be controllably movable relative to the other lighting channels of the lighting configuration to adjust at least one of the position or direction of the lighting provided by the movable lighting channel, and for the movable lighting channels, the lighting optimization process includes optimizing at least one of the position or direction of the movable lighting channels. In various embodiments, the lighting optimization process is configured to utilize negative color lighting channels by including a process of subtracting one image from another image.

[0205] According to another aspect, a method for operating the system to perform a lighting optimization process is provided. The method includes: providing an option for selecting a lighting optimization mode, the lighting optimization mode being at least one of an edge detection lighting optimization mode, a defect detection lighting optimization mode, or a taking points from focus lighting optimization mode; receiving a selection of the lighting optimization mode; and performing a lighting optimization process based on the selected lighting optimization mode, wherein the lighting optimization process determines the lighting for illuminating the workpiece, and the determined lighting includes settings of the lighting channels of the lighting configuration.

[0206] According to another aspect, the system is configured to: provide an option for selecting a lighting optimization mode, the lighting optimization mode being at least one of an edge detection lighting optimization mode, a defect detection lighting optimization mode, or a taking points from focus lighting optimization mode; receive a selection of the lighting optimization mode; and perform a lighting optimization process based on the selected lighting optimization mode, wherein the lighting optimization process determines the lighting for illuminating the workpiece, and the determined lighting includes settings of the lighting channels of the lighting configuration.

[0207] Although the preferred embodiments of the present disclosure have been shown and described, based on the present disclosure, those skilled in the art will understand various changes in the arrangement and operation sequence of the shown and described features. Various alternative forms can be adopted to implement the principles disclosed herein. In addition, the above various embodiments can be combined to provide additional embodiments. All U.S. patents and U.S. patent applications mentioned in this specification are hereby incorporated by reference in their entirety. If necessary, aspects of the embodiments can be modified to adopt the concepts of various patents and applications to provide additional embodiments.

[0208] Based on the above detailed description, these and other changes can be made to the embodiments. Generally speaking, in the following claims, the terms used should not be construed as limiting the claims to the specific embodiments disclosed in the specification and the claims, but should be construed to include all possible embodiments and the full scope of equivalents to which these claims are entitled.

Claims

1. A system comprising: a lens configured to input image light from a workpiece, wherein the lens is configured to transmit the image light along an imaging optical path and has an optical axis; a camera configured to receive image light transmitted along the imaging light path and provide an image of the workpiece; an illumination configuration comprising an illumination channel configured to illuminate the workpiece to produce the image light; one or more processors; as well as a memory coupled to the one or more processors and storing program instructions that, when executed by the one or more processors, cause the one or more processors to at least: providing an option for selecting a lighting optimization mode, the lighting optimization mode being at least one of an edge detection lighting optimization mode, a defect detection lighting optimization mode, or a point-from-focus lighting optimization mode; receiving a selection of the lighting optimization mode; as well as An illumination optimization process is performed based on the selected illumination optimization mode, wherein the illumination optimization process determines illumination for illuminating the workpiece, and the determined illumination includes settings of the illumination channels of the illumination configuration.

2. The system of claim 1, wherein the program instructions, when executed by the one or more processors, further cause the one or more processors to determine one or more elements or areas on the workpiece for the lighting optimization.

3. The system of claim 1, wherein the program instructions, when executed by the one or more processors, further cause the one or more processors to determine lighting variables to be used in the lighting optimization process.

4. The system of claim 1 , wherein the program instructions, when executed by the one or more processors, further cause the one or more processors to display results of the lighting optimization process, the results comprising one or more determined candidates for the lighting to illuminate the workpiece.

5. The system of claim 1 , wherein the program instructions, when executed by the one or more processors, further cause the one or more processors to determine at least one of the following: performing additional illumination optimization for the same workpiece and the same illumination optimization mode, but for different one or more elements or regions on the workpiece, and performing the illumination optimization process for the different one or more elements or regions on the workpiece; or Additional lighting optimization is to be performed for the same workpiece and a different lighting optimization mode, and the lighting optimization process is performed based on the different lighting optimization mode.

6. The system of claim 1 , wherein the program instructions, when executed by the one or more processors, further cause the one or more processors to save a lighting optimization model for the workpiece, the lighting optimization model for the workpiece comprising the settings of the lighting channels of the lighting configuration corresponding to the determined lighting.

7. The system of claim 6, wherein the program instructions, when executed by the one or more processors, further cause the one or more processors to: comparing a second workpiece to the workpiece and determining a similarity of the second workpiece; determining to use the stored lighting optimization model for lighting the second workpiece; as well as The saved lighting optimization model is invoked to provide the lighting for the second workpiece.

8. The system of claim 7, wherein the program instructions, when executed by the one or more processors, further cause the one or more processors to: determining a position and orientation of the second workpiece; determining a need to adjust the illumination based on the position and orientation of the second workpiece; and Adjustments are provided to the illumination based on the position and orientation of the second workpiece.

9. The system of claim 7, wherein the program instructions, when executed by the one or more processors, further cause the one or more processors to provide the illumination for the second workpiece and acquire one or more images of the second workpiece illuminated by the illumination.

10. The system of claim 9, wherein the program instructions, when executed by the one or more processors, further cause the one or more processors to perform one or more inspection operations on the second workpiece, the one or more inspection operations on the second workpiece corresponding to the lighting optimization mode selected for the lighting optimization process.

11. The system of claim 10, wherein the one or more inspection operations include: edge detection for determining the location of one or more edges on the second workpiece; as well as A distance between two edges on the second workpiece is determined.

12. The system of claim 10, wherein the one or more inspection operations include defect detection for detecting defects on the second workpiece.

13. The system of claim 10, wherein the one or more inspection operations include taking points from a focal point, the taking points from a focal point being used to determine three-dimensional profile data of the second workpiece.

14. The system of claim 1, wherein the program instructions, when executed by the one or more processors, further cause the one or more processors to enable a user to control independent lighting variables used as part of the lighting optimization process.

15. The system of claim 1, wherein when the defect detection lighting optimization mode is selected, the program instructions, when executed by the one or more processors, further cause the one or more processors to receive information from a user regarding at least one of: a workpiece having a defect, wherein the user indicates an area of ​​the defect, for which area the lighting optimization process utilizes the information to provide lighting that maximizes contrast in the area; or The illumination optimization process utilizes the characteristic of the workpiece to determine whether the optimized illumination corresponds to surface illumination with maximized texture or darkfield illumination of at least one surface of interest.

16. The system of claim 1, wherein when the defect detection lighting optimization mode is selected, the lighting optimization process is configured to communicate with a defect detection process, the defect detection process evaluating the suggested lighting and providing feedback regarding the accuracy with which the defect detection process is able to detect known defect areas illuminated by the suggested lighting using the suggested lighting.

17. The system of claim 1, wherein the program instructions, when executed by the one or more processors, further cause the one or more processors to collect a set of images to be used for the lighting optimization process, and for the set of images, each lighting channel to be optimized is turned on in at least one of the images.

18. A system according to claim 1, wherein one or more of the lighting channels of the lighting configuration are movable lighting channels, and for the movable lighting channels, each movable lighting channel is configured to be controllably movable relative to other lighting channels of the lighting configuration to adjust at least one of a position or a direction of lighting provided by the movable lighting channel, and for the movable lighting channels, the lighting optimization process includes optimizing at least one of the position or the direction of the movable lighting channels.

19. The system of claim 1, wherein the lighting optimization process is configured to utilize a negative color lighting channel by a process comprising subtracting one image from another.

20. A method for operating a system for performing a lighting optimization process, The system comprises: a lens configured to input image light from a workpiece, wherein the lens is configured to transmit the image light along an imaging optical path and has an optical axis; a camera configured to receive image light transmitted along the imaging light path and provide an image of the workpiece; as well as an illumination configuration comprising an illumination channel configured to illuminate the workpiece to produce the image light; The method comprises: providing an option for selecting a lighting optimization mode, the lighting optimization mode being at least one of an edge detection lighting optimization mode, a defect detection lighting optimization mode, or a point-from-focus lighting optimization mode; receiving a selection of the lighting optimization mode; and An illumination optimization process is performed based on the selected illumination optimization mode, wherein the illumination optimization process determines illumination for illuminating the workpiece, and the determined illumination includes settings of the illumination channels of the illumination configuration.

21. A system comprising: a lens configured to input image light from a workpiece, wherein the lens is configured to transmit the image light along an imaging optical path and has an optical axis; a camera configured to receive image light transmitted along the imaging light path and provide an image of the workpiece; an illumination configuration comprising an illumination channel configured to illuminate the workpiece to produce the image light; The system is configured as follows: providing an option for selecting a lighting optimization mode, the lighting optimization mode being at least one of an edge detection lighting optimization mode, a defect detection lighting optimization mode, or a point-from-focus lighting optimization mode; receiving a selection of the lighting optimization mode; as well as An illumination optimization process is performed based on the selected illumination optimization mode, wherein the illumination optimization process determines illumination for illuminating the workpiece, and the determined illumination includes settings of the illumination channels of the illumination configuration.

Images