Optimized multi-channel lighting control system

Abstract

To provide improvements with respect to problems for various types of inspection operations.SOLUTION: A system is provided that includes a lens, a camera, a lighting configuration, one or more processors, and a memory. The lens is configured to: input image light arising from a workpiece; and transmit the image light along an image optical path. The camera is configured to: receive the image light; and provide images of the workpiece. The lighting configuration includes lighting channels configured to illuminate the workpiece for generating the image light. In various embodiments, an option is provided for selecting a lighting optimization mode serving as at least one of an edge detection lighting optimization mode, a defect detection lighting optimization mode, or a focus point group lighting optimization mode. Lighting optimization processing may be implemented based on the selected lighting optimization mode, and determines lighting for illuminating the workpiece including settings for the lighting channels of the lighting configuration.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to inspection systems, and more particularly to precision systems for inspecting workpieces (e.g., for metrology, defect detection).

Background Art

[0002] Inspection systems such as machine vision inspection systems (or simply "vision systems") can be used to acquire images of workpieces for inspection. Such systems can be used for various types of applications (e.g., metrology applications for determining accurate dimensional measurements of workpieces, defect detection applications, general workpiece inspection applications, etc.). Such systems generally include a computer, a camera, and an optical system. In one configuration, a movement mechanism (e.g., a precision stage, a conveyor, etc.) that moves to enable traversal across and inspection of the workpiece may be included. One exemplary prior art machine vision inspection system is the QUICK vision (registered trademark) series of PC-based vision systems and the QVPAK (registered trademark) software available from Mitutoyo America Corporation (MAC) of Aurora, Illinois. The features and operation of the QUICK VISION (registered trademark) series of vision systems and the QVPAK (registered trademark) software are generally described, for example, in the QVPAK 3D CNC VISION Measuring Machine User's Guide issued in January 2003, which is hereby incorporated by reference in its entirety. This type of system uses a microscope type of optical system and moves the stage to provide an inspection image of the workpiece.

[0003] Such systems have typically faced various types of challenges for inspecting workpieces (e.g., due to the type of workpiece to be inspected or surface variations, different types of inspection operations to be performed, changes in inspection conditions, etc.). There is a desire for a system that can provide solutions to such challenges for various types of inspection operations.

Summary of the Invention

Problems to be Solved by the Invention

[0004] This summary is provided to introduce, in a simplified form, a selection of concepts that are further described below in the detailed description of embodiments of the invention. This summary is not intended to identify key features of the subject matter recited in the claims, nor is it intended to be used as an aid in determining the scope of the subject matter recited in the claims.

[0005] According to one aspect, a system is provided that includes a lens, a camera, an illumination configuration, one or more processors, and a memory. The lens (e.g., an objective lens) is configured to input image light originating from a workpiece, the lens is configured to transmit the image light along an image optical path, and has an optical axis. The camera is configured to receive the image light transmitted along the image optical path and provide an image of the workpiece. The illumination configuration includes an illumination channel configured to illuminate the workpiece to generate the image light.

[0006] The memory is coupled to the one or more processors and stores program instructions that, when executed by the one or more processors, cause the one or more processors to perform at least the following processes: provide a display of a group of illumination channels in a display area; determine that a group of illumination channels has been selected, the selected group of illumination channels including a plurality of illumination channels; display a current illumination setting for the selected group of illumination channels, and an adjustment to the illumination setting for the group of illumination channels is applied to all of the illumination channels within the group.

[0007] According to another aspect, a method for operating a system is provided. The method includes the following processes: providing a representation of a group of lighting channels in a display area; determining that a group of lighting channels is selected, wherein the selected group of lighting channels includes a plurality of lighting channels; and displaying a current lighting setting for the selected group of lighting channels, wherein an adjustment to the lighting setting for the group of lighting channels is applied to all of the lighting channels within the group.

[0008] According to another aspect, a system is provided that includes a lens, a camera, an illumination configuration, one or more processors, and a memory. The memory is coupled to the one or more processors and stores program instructions that, when executed by the one or more processors, cause the one or more processors to at least: provide an option for selecting an illumination optimization mode that is at least one of an edge detection illumination optimization mode, a defect detection illumination optimization mode, or a points from focus (PFF) illumination optimization mode; receive a selection of the illumination optimization mode; and execute an illumination optimization process based on the selected illumination optimization mode, wherein the illumination optimization process determines illumination for illuminating a workpiece, and the determined illumination includes settings for the illumination channels of the illumination configuration. In various embodiments, the option can be for selecting the illumination optimization mode from a set of illumination optimization modes.

[0009] Points from focus (PFF) is an optical data collection method for acquiring point data along the Z (optical) axis. The objective lens is moved in the Z direction to acquire a stack of images including a plurality of continuously changing images. Peaks in the contrast of each pixel in the plurality of images are detected, and the images corresponding to the peaks are stitched together to generate an image of the field of view (FOV). PFF is applied to automatic illumination adjustment.

[0010] According to another aspect, a method for operating a system for performing illumination optimization processing is provided. The method includes the following processes: providing an option for selecting at least one illumination optimization mode among an edge detection illumination optimization mode, a defect detection illumination optimization mode, or a focus point group illumination optimization mode; receiving a selection of the illumination optimization mode; and performing an illumination optimization process based on the selected illumination optimization mode, where the illumination optimization process determines illumination for illuminating a workpiece, and the determined illumination includes settings for illumination channels of an illumination configuration.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3A

Figure 3B

Figure 3C

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

[0012] FIG. 1 is a diagram showing various typical components of a general-purpose precision machine vision inspection system 10. The machine vision inspection system 10 includes a vision measurement machine 12 operably connected to exchange data and control signals with a control computer system 14. The control computer system 14 is further operably connected to exchange data and control signals with a monitor or display 16, a printer 18, a joystick 22, a keyboard 24, and a mouse 26. The monitor or display 16 can display a user interface suitable for controlling and / or programming the operation of the machine vision inspection system 10. In various embodiments, it will be understood that a touch screen tablet or the like can be used in place of, and / or provided redundantly with, any or all of the functions of elements 14, 16, 22, 24, and 26.

[0013] One of ordinary skill in the art will understand that the control computer system 14 can generally be implemented using any suitable computing system or device, including, for example, a distributed or networked computing environment. Such a computing system or device can include one or more general-purpose or dedicated processors (e.g., off-the-shelf or custom devices) that execute software to perform the functions described herein. The software can be stored in a memory such as random access memory (RAM), read-only memory (ROM), flash memory, or a combination of such components. Additionally, the software can be stored in one or more storage devices such as optical-based disks, flash memory devices, or any other type of non-volatile storage medium for storing data. The software can include one or more program modules that include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. In a distributed computing environment, the functions of the program modules can be combined or distributed across multiple computing systems or devices in either a wired or wireless configuration and accessed via service calls.

[0014] The vision measuring machine 12 includes a movable workpiece stage 32 and an optical imaging system 34 that may include a zoom lens or an interchangeable objective lens. The zoom lens or the interchangeable objective lens generally provides various magnifications to the image provided by the optical imaging system 34. Various embodiments of the machine vision inspection system 10 are also described in commonly assigned U.S. Patents 7,454,053; 7,324,682; 8,111,905; and 8,111,938. Each of these is incorporated herein by reference in its entirety.

[0015] As will be described in more detail below, for various applications (including those where a variable focal length (VFL) lens such as a variable aperture (TAG) lens is utilized), it may be desirable to provide illumination from multiple directions to better image non-planar workpieces (e.g., workpieces including at least some surfaces that are not orthogonal to the optical axis of the system, such as surfaces that can be inclined / inclined and / or parallel to the optical axis). As will be described in more detail below, an illumination configuration (e.g., including illumination configuration 230, etc.) can include illumination channels that provide such illumination from multiple directions.

[0016] FIG. 2 is a block diagram of a control system unit 120 and a vision component unit 200 of a machine vision inspection system 100 similar to the machine vision inspection system of FIG. 1, including specific features disclosed herein. As will be described in more detail below, the control system unit 120 is utilized to control the vision component unit 200. The vision component unit 200 includes an optical assembly unit 205, light sources 220, 230, 240, and a workpiece stage 210 having a central transparent portion 212. The workpiece stage 210 is controllably movable along X and Y axes in a plane generally parallel to the surface of the stage on which the workpiece 20 can be positioned.

[0017] The optical assembly section 205 includes a camera system 260 (i.e., including a camera), an interchangeable objective lens 250, and a variable focal length (VFL) lens 270. In various embodiments, the VFL lens 270 can be a variable focal length ( "TAG" or "TAGLENS") lens that uses sound waves in a fluid medium to generate a lens effect. The sound waves may be generated by applying an electric field at a resonant frequency to a piezoelectric tube surrounding the fluid medium to generate a time-varying density and refractive index profile within the fluid of the lens, which modulates its refractive power and thereby modulates the focal length (or effective focal position) of the optical system. The TAG lens can be used to rapidly and periodically sweep (i.e., periodically modulate its optical power) a range of focal lengths at a resonant frequency above 30 kHz, or above 70 kHz, or above 100 kHz, or above 400 kHz, for example up to a maximum of 1.0 MHz. Such lenses can be understood in more detail by the teachings of the paper "Fast Variable-Focus Imaging Using a Tunable Acoustic Refractive-Index-Profile Lens" (Optics Letters, Vol. 33, No. 18, September 15, 2008), which is hereby incorporated by reference in its entirety. The TAG (aka TAGLENS) lens and associated controllable signal generator are available, for example, from Mitutoyo Corporation, Kanagawa Prefecture, Japan. As a specific example, a particular TAG lens is capable of periodic modulation with a modulation frequency of up to 1.0 MHz. Various aspects of the operating principles and applications of the TAG lens are described in more detail in U.S. Patents 10,178,321; 10,101,572; 9,930,243; 9,736,355; 9,726,876; 9,143,674; 8,194,307; and 7,627,162. Each of these is hereby incorporated by reference in its entirety into this specification.

[0018] In various implementations, the optical assembly section 205 may further include a turret lens assembly 223 having lenses 226 and 228. As an alternative to the turret lens assembly, in various implementations, a fixed or manually replaceable magnification-changing lens, or a zoom lens configuration, or the like may be included. In various embodiments, the interchangeable objective lens 250 may be selected from a set of fixed magnification objective lenses included as part of a variable magnification lens section (e.g., a set of objective lenses corresponding to magnifications such as 0.5x, 1x, 2x or 2.5x, 5x, 7.5x, 10x, 20x or 25x, 50x, 100x, etc.).

[0019] The optical assembly section 205 is controllably movable along a Z-axis that is generally orthogonal to the X-axis and Y-axis by using a controllable motor 294 that drives an actuator to move the optical assembly section 205 along the Z-axis to change the focus of an image of the workpiece 20. The controllable motor 294 is connected to the input / output interface 130 via a signal line 296. As will be described in more detail below, in order to change the focus of the image over a smaller range or as an alternative to moving the optical assembly portion 205, the VFL (TAG) lens 270 may be controlled via a signal line 234' by a lens control interface 134 to periodically modulate the refractive power of the VFL lens 270 and thus modulate the effective focal position of the optical assembly section 205. The lens control interface 134 can include a VFL lens controller 180, as will be described in more detail below. The workpiece stage 210 has the workpiece 20 placed thereon. The workpiece stage 210 can be controlled to move relative to the optical assembly portion 205 such that the field of view of the interchangeable objective lens 250 moves between positions on the workpiece 20 and / or between a plurality of workpieces 20, etc.

[0020] One or more of the stage light source 220, the illumination configuration 230, and the coaxial light source 240 may each emit source light 222, 232, and / or 242 to illuminate one workpiece 20 or a plurality of workpieces 20. In various exemplary embodiments, pulsed illumination (e.g., stroboscopic) may be used. For example, during image exposure, the illumination configuration 230 can emit stroboscopic source light 232 toward a central volume CV where at least a portion of the workpiece 20 is located. In another example, during image exposure, the coaxial light source 240 can emit stroboscopic source light 242 along a path that includes a beam splitter 290 (e.g., a partial mirror / reflection surface). The source lights 232, 242 are reflected as image light 255, and the image light used for imaging passes through the exchange objective lens 250, the turret lens assembly 223, and the VFL (TAG) lens 270 and is focused onto the camera system 260. A workpiece image exposure including an image of the workpiece 20 is captured by the camera system 260 and output to the control system unit 120 via the signal line 262.

[0021] In various embodiments, the illumination configuration 230 includes a plurality of illumination channels configured to illuminate the workpiece 20 to generate the image light 255, and each illumination channel is configured to direct light toward a central portion CV (e.g., where at least a portion of the workpiece 20 can be disposed). As described above, the objective lens 250 is configured to receive the image light 255 generated from the workpiece 20, and the objective lens 250 is configured to transmit the image light along an image optical path and has an optical axis OA. In the example of FIG. 2, the objective lens 250 transmits the image light to the camera 260 along the image optical path passing through the VFL lens 270. The camera 260 is configured to receive the image light transmitted along the image optical path and provide an image of the workpiece 20. As will be described in more detail below, the focal position corresponding to the focus of the image is configured to be variable within a focal range along the optical axis. In various embodiments, the illumination configuration 230 is controlled via an illumination control interface 133 (e.g., including an optical controller portion for controlling the illumination configuration 230, such as the optical controller section 133n).

[0022] Various light sources (e.g., light sources 220, 230, 240) may be characterized as including illumination channels and may be connected to the illumination control interface 133 of the control system unit 120 via associated signal lines (e.g., buses 221, 231, 241 respectively). The control system unit 120 can control the turret lens assembly 223 to rotate along the axis 224, select the turret lens through the signal line or bus 223‘, and change the image magnification.

[0023] As shown in FIG. 2, in various exemplary embodiments, the control system unit 120 includes a controller 125 (e.g., including one or more processors or operating as part of one or more processors), an input / output interface 130, a memory 140, a workpiece program generator and executor 170, and a power supply unit 190. Each of these components, as well as additional components described below, can be interconnected by one or more data / control buses and / or application programming interfaces, or by direct connections between various elements. The input / output interface 130 includes an image control interface 131, an operation control interface 132, an illumination control interface 133, and a lens control interface 134. The lens control interface 134 may include or be connected to a VFL lens controller 180 that includes circuits and / or routines for controlling various image exposures synchronized with the periodic focus position modulation provided by the VFL (TAG) lens 270. In some embodiments, the lens control interface 134 and the VFL lens controller 180 may be integrated and / or indistinguishable.

[0024] The illumination control interface 133 can include illumination control elements 133a, 133n that control selection, power, on / off switches, and pulse / strobe timing, for example, for various corresponding light sources / illumination channels of the machine vision inspection system 100, if applicable. In various embodiments, an example of strobe illumination can be considered a type of pulse illumination in the terms of this specification. In some embodiments, the illumination control interface 133 can provide a pulse / strobe timing signal to one or more of the illumination control elements 133a, 133n, whereby they provide an image exposure pulse / strobe timing (e.g., this can be synchronized with the desired phase time of the VFL lens focus position modulation according to stored calibration data).

[0025] The memory 140 includes an image file storage unit 141, an edge detection storage unit 140ed, a workpiece program storage unit 142 including one or more sub-programs, etc., and a video tool unit 143. The video tool unit 143 includes a GUI for each corresponding video tool, a video tool unit 143a and other video tool units (e.g., 143n) that determine image processing operations, etc., and a region of interest (ROI) generator 143roi that supports automatic, semi-automatic, and / or manual operations. Various ROIs operable in various video tools included in the video tool unit 143 are defined. Examples of the operations of such video tools for identifying the positions of edge feature points and performing other workpiece feature point inspection operations are described in more detail in some of the previously incorporated references, as well as in U.S. Patent 7,627,162, which is hereby incorporated by reference in its entirety.

[0026] The video tool section 143 includes an autofocus video tool 143af that determines a GUI for focus height (i.e., effective focus position (Z coordinate / Z height)) measurement operations, image processing operations, and the like. In various embodiments, the autofocus video tool 143af may further include a high-speed focus height tool that can be used to measure the focus height at high speed using the hardware shown in FIG. 3, as described in more detail in U.S. Patent 9,143,674 incorporated above. In various embodiments, the high-speed focus height tool may be a special mode of the autofocus video tool 143af that can operate according to conventional methods for the autofocus video tool otherwise, or the operation of the autofocus video tool 143af may include only the operation of the high-speed focus height tool. High-speed autofocus and / or focus determination for an image region or region of interest may be based on analyzing an image to determine corresponding focus characteristic values (e.g., quantitative contrast metric values and / or quantitative focus metric values) for various regions according to known methods. For example, such methods are disclosed in U.S. Patents 8,111,905; 7,570,795; and 7,030,351. Each of these is hereby incorporated by reference in its entirety.

[0027] In the context of the present disclosure, as is known to those skilled in the art, the term "video tool" generally refers to a relatively complex set of automated or programmed operations that a machine vision user can implement through a relatively simple user interface. For example, a video tool may, in certain instances, include a complex pre-programmed set of image processing operations and calculations that are applied and customized by adjusting some variables or parameters that manage the operations and calculations. In addition to the underlying operations and calculations, a video tool includes a user interface that enables the user to adjust these parameters for a particular instance of the video tool. It should be noted that visible user interface functions may sometimes be referred to as video tools, with the underlying operations being implicitly included.

[0028] One or more display devices 136 (e.g., display 16 of FIG. 1) and one or more input devices 138 (e.g., joystick 22, keyboard 24, and mouse 26 of FIG. 1) can be connected to the input / output interface 130. The display device 136 and the input device 138 can be used to execute inspection / measurement operations and / or to create and / or modify part programs, to view images captured by the camera system 260, and / or to directly control the vision component portion 200, and can be used to display a user interface that may include various graphical user interface (GUI) features.

[0029] In various exemplary embodiments, when a user utilizes the machine vision inspection system 100 to create a part program for the workpiece 20, the user generates part program instructions by operating the machine vision inspection system 100 in a learning mode to provide a desired image acquisition training sequence. For example, the training sequence can include positioning specific workpiece features of a representative workpiece within the field of view (FOV), setting the light level, focusing or autofocusing, acquiring an image, and providing an inspection training sequence to be applied to the image (e.g., using one instance of one of the video tools on that workpiece). The learning mode operates such that the sequence is captured or recorded and converted into corresponding part program instructions. These instructions, when the part program is executed, cause the machine vision inspection system to reproduce the learned image acquisition function and cause the inspection operation function to automatically inspect a run-mode workpiece or that specific workpiece feature on the workpiece (i.e., the corresponding feature at the corresponding position) that matches the representative workpiece used when creating the part program.

[0030] In various embodiments, the image optical path OPATH (also referred to herein as the workpiece image optical path) comprises various optical components arranged along the path along which image light 255 is transmitted from the workpiece 20 to the camera 260. The image light typically propagates along the direction of their optical axes OA. In the embodiment shown in FIG. 2, all of the optical axes OA are aligned. However, it will be understood that this embodiment is merely exemplary and not limiting. More generally, the image optical path OPATH can include mirrors and / or other optical elements and can take any form operable to image the workpiece 20 using a camera (e.g., camera 260) according to known principles. In the illustrated embodiment, the image optical path OPATH includes a VFL lens 270 (which can be included in a 4f imaging configuration) and is at least partially utilized to image the workpiece 20 during workpiece image exposure. In various embodiments, the Z-height (e.g., of a surface point on the workpiece) may correspond to and / or alternatively be referred to as the Z-coordinate and / or the focal position, and these terms may be used interchangeably in some examples herein.

[0031] In various embodiments, the VFL lens controller 180 can control the drive signal of the VFL lens 270 to periodically modulate the optical power of the VFL lens 270 over a range of optical powers that occur at each phase timing within the periodic modulation. Alternatively, a controllable motor 294 may be utilized to move the optical assembly portion 205 along the Z-axis to change the effective focal position of the optical assembly portion 205. In either case, the camera 260 (e.g., including an image sensor) receives the light transmitted along the image optical path OPATH during the image exposure and provides a corresponding camera image. The objective lens 250 inputs the image light generated from the workpiece 20 during the image exposure, sends it through the VFL lens 270 along the image optical path OPATH to the camera 260 during the image exposure, and provides a workpiece image to the corresponding camera image. In embodiments that utilize the VFL lens 270, the effective focal position in front of the objective lens 250 during the image exposure corresponds to the refractive power of the VFL lens 270 during that image exposure. The illumination control interface 133 may be configured to control the image exposure timing used for the camera image.

[0032] In various embodiments, the illumination configuration LC (e.g., including the illumination configuration 230, the light source / illumination channel 240, etc.) is configured to emit the source light 232 / 242 (e.g., in some examples, using pulse / strobe illumination) toward a central portion CV including at least a part of the workpiece 20. The objective lens 250 receives the image light 255 (e.g., workpiece light) focused at an effective focal position close to the workpiece 20 and emits the image light 255 along the optical path OPATH.

[0033] In various embodiments, the illumination configuration 230 of FIG. 2 includes a plurality of illumination channels configured to illuminate the workpiece 20 to generate image light 255, and each illumination channel is configured to direct light 232 towards a central portion CV (e.g., at least a portion of the workpiece 220 can be disposed). The objective lens 250 is configured to receive the image light 255 generated from the workpiece 20 and to transmit the image light 255 along the image optical path OPATH, and the objective lens 250 has an optical axis OA. The camera 260 is configured to receive the image light 255 transmitted along the image optical path OPATH and to provide an image of the workpiece 20, and the focal position corresponding to the focus of the image is configured to be variable within a focal range along the optical axis OA.

[0034] In various embodiments, the VFL lens controller 180 is configured to control the VFL lens to periodically modulate the optical power of the VFL lens 270 over a range of optical powers at an operating frequency to vary the focal position of the system over a plurality of positions within the focal range. Alternatively, a controllable motor 294 may be utilized to move the optical assembly portion 205 along the Z-axis to change the effective focal position of the optical assembly portion 205. In either case, in various embodiments, the camera 260 can be utilized to acquire an image stack including a plurality of images of the workpiece 20 (e.g., as will be described in more detail below with respect to FIGS. 14A and 14B), and each image of the image stack corresponds to a different focal position along the optical axis OA (e.g., each corresponding to a different Z-height in the example of FIG. 2). The focus curve data may be determined based at least in part on an analysis of the images of the image stack, and the focus curve data indicates the three-dimensional positions of a plurality of surface points on the workpiece 20 (e.g., as part of the focus point group processing performed by the focus point group unit 146).

[0035] In some embodiments, data or signals from camera 260 can be utilized to determine when an imaged surface region (e.g., including one or more surface points of workpiece 20) is at the effective focus position. For example, a group of images acquired by camera 260 at different effective focus positions (Z heights), such as part of an image stack, can be analyzed using known "maximum contrast" or "best focus image" analysis to determine when the imaged surface region of workpiece 20 is at the corresponding effective focus position (Z height). However, more generally, any other suitable known image focus detection configuration may be used. In any case, in embodiments where VFL lens 270 is utilized, one or more images acquired during the periodic modulation of the effective focus position of VFL lens 270 (during the sweep of multiple effective focus positions) can be used to determine the image and / or image timing at which a target feature (e.g., including one or more surface points of the workpiece) is best focused.

[0036] In various embodiments, illumination control interface 133 controls the image exposure time of system 100 (e.g., with respect to the phase timing of a periodically modulated effective focus position). More specifically, during image exposure, illumination control interface 133 can use effective focus position (Z height) calibration data to control illumination configuration LC (e.g., including illumination configuration 230, etc.) to pulse / strobe at each time. For example, illumination control interface 133 can control illumination configuration LC to pulse / strobe at each phase timing within the period of the standard imaging resonance frequency of VFL lens 270 so as to acquire an image having a specific effective focus position within the sweep (periodic modulation) range of VFL lens 270. It will be understood that the operation of illumination control interface 133 as well as the other features and elements outlined above can be implemented to manage workpiece image acquisition.

[0037] As described in more detail below, certain lighting control and lighting optimization features and processes can be provided and executed in accordance with the principles described herein. In various embodiments, such features and processes may be implemented in and / or executed by some portions of the control system portion 120 of FIG. 2. For example, in various embodiments, the lighting control interface 133 and / or the lighting optimization portions 133lo or 147 may be utilized to implement such features. In various embodiments, such features and processes can include controlling various lighting channels of the system 100, and the corresponding lighting configuration LC can be specified as including specific lighting channels of the system that are represented and controlled such as the lighting channels of lighting configuration 230 and / or the axial lighting channels 240. Additionally, the edge detection unit 144, the defect detection unit 145, and the focus point group unit 146 may each be utilized to perform various edge detection, defect detection, and / or focus point group processing operations and / or features, and such processing, operations, and / or features will also be described in more detail below.

[0038] In various embodiments, the defect detection unit 145 performs various defect detection operations as described in more detail below. In various embodiments, the defect detection unit 145 utilizes a model that requires training data (e.g., training images). For example, the defect detection unit 145 may learn using a set of training images captured using specified imaging conditions, lighting conditions, and workpiece conditions. In various exemplary embodiments, the model may be a supervised model. In various embodiments, the defect detection unit 145 processes image data corresponding to images labeled with defects (e.g., labeled by a user and / or automatically processed) for the classification model to learn.

[0039] In various exemplary embodiments, the defect detection unit 145 can perform defect detection processing that is executed together with measurement processing (e.g., at least partially executed by the edge detection unit 144 and / or the focus point group unit 146). In various embodiments, it may be desirable to include the defect detection unit 145 in a system configured to perform measurement processing in that the system can generate image data input to the defect detection processing performed by the defect detection unit 145. Thus, a single machine can be configured to perform both measurement processing and defect detection processing, which can provide an advantage over conventional measurement systems. For example, if a defect is detected in the workpiece while the defect detection processing is being executed, there is no reason to perform the measurement processing on the workpiece, and time can be saved. That is, if a defect is detected in the workpiece during the defect detection processing, it may not be necessary to measure the clear defect portion. Therefore, it may be advantageous to execute the defect detection processing before starting the measurement processing.

[0040] In addition, some defects may require further measurement or inspection to determine additional defect parameters. For example, a 2D image can enable potential defects to be quickly recognized, based on which specific measurement operations can be performed (e.g., for determining the exact XY position of the defect and / or the approximate XY size / area of the defect, such as at least partially performed by the edge detection unit 144). When the three-dimensional nature of the potential defect is important, the defect detection unit 145 can perform additional processing (e.g., measurement operations) to determine whether the potential defect is an actual defect. For example, if a scratch on the surface of the workpiece has to be deeper than a specific threshold to be considered a defect, the defect detection unit 145 can acquire a more time-consuming 3D point cloud of the affected area (e.g., at least partially utilizing points from the focus point cloud unit 146) to learn whether the depth of the scratch is sufficient to reject the part. As will be described in more detail below, in various embodiments, different lighting optimizations can be determined corresponding to different types of processing (e.g., processing performed by the edge detection unit 144, the defect detection unit 145, the focus point cloud unit 146, etc.)

[0041] In various embodiments, the lighting channels described herein may be considered the lowest independently controllable elements of a light source configuration (e.g., an RGB light emitting diode (LED)). A group of lighting channels may be a set of lighting channels, for example, predefined or defined by a user and controlled as a group to share the same settings (e.g., RGB settings). According to various examples herein, a group of lighting channels may correspond to a particular shape such as a ring, a pie V shape, an asymmetric shape, etc. Such shapes may be considered to correspond to the appearance on the surface of the lighting configuration. The lighting layout can be considered a set of lighting channels and / or groups of individual lighting channels. In some embodiments, the lighting layout may be considered to partition some or all of the lighting channels of the lighting configuration into an array (e.g., stored or loaded in a controller of a graphical user interface and used in lighting optimization processes, etc.). As one simplified example, one particular lighting layout may include a set of ring-shaped groups of lighting channels that together include all of the lighting channels of the lighting configuration.

[0042] In various embodiments, various graphical user interface elements and principles for improving the process of manual, semi-automatic or automatic lighting adjustment in a system having many (e.g., tens of thousands of) independently controllable lighting channels are described herein. An example of a lighting configuration having such features is a dome light (e.g., using 55 lighting channels, etc.). According to the specific examples described herein, the lighting channels correspond to the physical arrangement of the lighting channels, and the graphical user interface representation is displayed in the graphical user interface in a geometric arrangement that can be manipulated and adjusted (e.g., by a user), as shown in the display area 360 of the light source display in FIGS. 3A and 3C, for example.

[0043] According to the principles described herein, a group of lighting channels can include a plurality of lighting channels, and each group of lighting channels can be adjusted as a single entity. Enabling the inclusion of lighting channels in groups and adjusting each group of lighting channels as a single entity can make certain processes, such as manual, semi-automatic, or automatic lighting optimization, faster and / or easier by reducing the effective number of entities to be adjusted. In some embodiments, each lighting channel may be included in a single group, and in some embodiments, the lighting channels may be assigned to multiple groups. In various embodiments, when a group is determined (e.g., created or selected), the settings of that group (e.g., current RGB settings) can be displayed within the display area of a graphical user interface (e.g., such as in the display area 350 of FIGS. 3A and 3C).

[0044] In some embodiments that include a lighting layout (e.g., a pie segment or a ring) having the ability to quickly adjust predefined parameters (e.g., some pie segments or rings), in various embodiments, groups of lighting channels and / or lighting layouts may be stored and recalled. In various embodiments, several controls can be provided, such as rotation of an element, expansion or contraction (e.g., of a lighting ring), control of a specific lighting direction (e.g., control like a trackball), or negation of a specific portion (e.g., of the current lighting pattern). Such changes and adjustments can be achieved by adjusted changes in the settings of different lighting channels, such as in response to operations performed in the graphical user interface 300 (e.g., as shown in FIGS. 3A - 3C).

[0045] Figures 3A - 3C are diagrams of a graphical user interface 300 for lighting control of a system (which may be provided, for example, to the display 16 of FIG. 1 and / or the display device 136 of FIG. 2). FIG. 3A includes the complete graphical user interface 300, and FIGS. 3B and 3C include respective left and right halves with additional labeling. The graphical user interface 300 includes display regions 310, 320, 330, 340, 350, 360, 370, 380, 390, and 395. The display region 310 includes a "layout list" and includes selection regions 311 - 316 (i.e., as shown in FIG. 3B), each of the selection regions including a label indicator in column 310A and a graphical indicator in column 310B. As indicated by the label indicators in column 310A, the selection regions 311 - 316 correspond to current layouts 1 - 6. In column 310B, graphical indicators for each of the layouts are provided, which may provide a graphical representation of the layout. The selection region 310C to the right of the display region 310 includes a scroll region that can be used to scroll to additional layouts not currently shown in the display region 310.

[0046] As a specific example, in the illustrated embodiment, the layout list can include a total of 17 layouts, so that the user can scroll between layouts 1 to 17. Also, selection regions 317, 318, and 319 are also included below and in the vicinity of the display region 310. Selection region 317 provides an option for creating a new empty layout. Selection region 318 provides an option for deleting a selected layout. Selection region 319 provides an additional quick browse scroll bar that can be used to scroll the layout. In selection region 319, the quick browse range is shown to extend from layout 1 to layout 17, and the current scroll indicator is at the beginning of the range to correspond to layout 1 (i.e., also indicated by the number "1" on the right side of the scroll bar). The position of the scroll indicator also coincides with the selected layout 1 in selection region 311 as indicated by the selection indicator SI (i.e., the selection indicator SI within selection region 311 corresponds to the highlighting of the selected layout).

[0047] The display region 320 includes a "group list" and includes selection regions 321, 326 (i.e., as shown in FIG. 3B). Column 320A includes label indicators for the currently displayed set of groups of (e.g., the selected layout 1), and column 320B includes a scroll area, whereby the user can scroll to display different groups in the display region 320. In the diagram of FIG. 3B, the selection areas 321 to 326 are currently shown as corresponding to groups 1 to 6. As a specific example, in the illustrated embodiment, the group list of layout 1 can include 12 groups including groups 1 to 12. In the illustrated embodiment, the group list in the display area 320 shows the current selection corresponding to selection area 323 as corresponding to group 3 (i.e., layout 1).

[0048] Proximate to the bottom of the display area 320, there are selection areas 327, 328, and 329. Selection area 327 provides an option to save the current layout as a new layout. Selection area 328 provides an option to delete the selected group. Selection area 329 provides a quick browse scroll option to scroll between different groups. As shown in selection area 329, the quick browse range is from group 1 (i.e., indicated at the left end) to group 12 (i.e., indicated by the end of the range at the right end). The current position of the scroll indicator (i.e., moving approximately 25 percent across the available range) is at group 3 (i.e., also indicated by the number "3" on the right side of the scroll bar). The position of the scroll indicator also coincides with group 3 selected in selection area 323, as indicated by a selection indicator SI similar to the selection indicator SI in the display area 310.

[0049] The display area 330 includes "layout parameters" and includes selection areas 331 - 333, and additional selection areas may be provided. In the current state, only selection area 331 is (e.g., dynamically) assigned, and in column 330B, it is currently shown to be for several segments (i.e., as the first layout parameter of the currently selected layout 1, shown in the display area 310). The selection area 331 in column 330A indicates that the current number of segments is set to 5 segments and includes up and down arrows that can be used to increase or decrease (e.g., in this case, increase or decrease the number of segments).

[0050] The display area 340 includes a selection area 341 for "channel setting change" and a selection area 342 for "shared channel setting operation". The section area 341 includes selection areas 341A to 341C, and the selection area 342 includes selection areas 342A to 342E. Radio buttons are included in each of the corresponding selection options. In the selection area 341, the selection area 341A is for the "direct" option, the selection area 341B is for the "negate to 0" option, and the selection area 341C is for the "overall negate" option. In the selection area 342, the selection area 342A is for the "average of all groups" option, the selection area 342B is for the "sum of all groups" option, the selection area 342C is for the "product of all groups" option, the selection area 342D is for the "minimum of all groups" option. Also, the selection area 342E is for the "maximum of all groups" option.

[0051] The display area 350 is for "main settings" and includes selection areas 351, 354. The selection area 351 is for color setting and includes a selection area 351A for red light setting, a selection area 351B for green light setting, and a selection area 351C for blue light setting. As shown, the color setting can be set within a range of values from -1 to 1. A setting in the range between -1 and 0 indicates a negative setting for the corresponding light color, and a setting between 0 and 1 indicates a positive setting for the corresponding light color. The numerical indicators in column 351D show the corresponding values of the settings for different light colors, such as a value of 0.50 for red light setting, a value of 0.30 for green light setting, and a value of 0.15 for blue light setting, which are also indicated by the positions of the sliders along the range between -1 and 1 within the selection areas 351A, 351B, and 351C.

[0052] The selection area 352 includes a selection option 352A corresponding to an upward arrow for increasing the current brightness setting and a selection option 352B corresponding to a downward arrow for decreasing the current brightness setting. In the selection area 353, the selection option 353A corresponding to an arrow pointing backward corresponds to an option for canceling the change, and the selection option 353B corresponding to an arrow pointing forward corresponds to an option for re-doing the change (e.g., undone by the undo selection option 353A). The selection area 354 provides an option for performing a gang selection.

[0053] The display area 360 includes "light source display" and selection areas 361 to 365. The selection area 361 corresponds to a display area where the representation of the lighting channel LCH of the lighting configuration LC is displayed and can be selected or deselected. In one embodiment, the representation may be that of the lighting configuration LC, the two rings of the lighting channel may correspond to the lighting configuration 230 in FIG. 2, and the central axis lighting channel may correspond to the light source 240. The selection area 362 corresponds to an "enlarge" option, the selection area 363 corresponds to a "rotate" option, the selection area 364 corresponds to a "direct" option, and the selection area 365 corresponds to a "forced exclusive group" option. The selection area 366 is adjacent to the lower part of the display area 360 and includes a selection option 366A for "group display" and a selection option 366B for "settings display", and radio buttons are provided for each selection option.

[0054] The display area 370 has some similarities with the display area 350 and will be understood to operate similarly, except as otherwise described below. The display area 370 includes selection areas 371, 376. The selection area 371 includes the "Override Settings for Selection" option. The selection area 372 includes the "Other Channels Off" option. The selection area 373 includes, similar to the selection area 351, a red light setting option 373A, a green light setting option 373B, and a blue light setting option 373C. The color settings can be made within a range of values from -1 to 1. In column 373D, indicators of the corresponding values of the settings are provided, showing a value of -0.50 for the red light setting, a value of 0.30 for the green light setting, and a value of 0.15 for the blue light setting. These setting values are also indicated by the positions of sliders along each of the corresponding ranges (i.e., between -1 and 1) so that they can be adjusted by the user to adjust the settings of the different colored lights within the selection areas 373A, 373B, and 373C.

[0055] In the selection area 374, the selection option 374A corresponding to the upward arrow is for increasing the brightness setting, and the selection option 374B corresponding to the downward arrow is for decreasing the current brightness setting. The selection area 375 includes a selection option 375A corresponding to an arrow pointing backward for canceling the change and a selection option 375B corresponding to an arrow pointing forward for redoing the change. The selection area 376 provides an option for "Gang" selection.

[0056] The display area 380 includes "workpiece display" (e.g., displaying an image of the workpiece 20 illuminated by the current lighting settings). In the illustrated embodiment, the display area 380 includes an upper display area 380A and a lower display area 380B. The upper display area 380A shows a workpiece portion illuminated by the current lighting settings. According to example "A", an edge EDG on the workpiece is detected by an edge detection tool EDT. The position of the edge EDG substantially corresponds to a dotted edge indicator in the edge detection tool EDT. In the lower display area 380B, according to example "B", a defect DFT on the workpiece is shown as having been detected by a defect detection tool DDT.

[0057] The display area 390 includes selection options 391A - 391D. The selection option 391A is for optimizing edge detection, the selection option 391B is for optimizing defect detection, the selection area 391C is for optimizing a focus point group, and the selection option 391D is for an unspecified process for optimizing lighting. Such options can be referred to as lighting optimization modes, as will be described in more detail below. For all of the selection options 391A - 391D within the display area 390, radio buttons are provided for making the corresponding selections. The display area 395 includes further selection options 396A - 396D. The selection option 396A is for selecting to optimize brightness, the selection option 396B is for selecting to optimize the red light setting, the selection option 396C is for selecting to optimize the green light setting, and the selection option 396D is for selecting to optimize the blue light setting. For all of the selection options 396A - 396D within the display area 395, check boxes are provided for making the corresponding selections. The operation of each of the display areas 310 - 395 will be described in more detail below.

[0058] In various embodiments, each layout in the layout list of the display area 310 can include at least one group of lighting channels (e.g., be created, added, and / or configured) by user interaction in the light source display of the display area 360. In various embodiments, each layout can have a description icon (e.g., as shown in column 310B), and in various embodiments, the description name can be added to an additional column or substituted with a general layout name (e.g., a description name substituted for general layout 1, layout 2, etc.) to provide the user with ideas such as the purpose the layout serves and / or how the layout looks. As some examples, in various embodiments, some names that can be used for a particular layout and / or group can include "dark field", "diffusion", "direction", "grading", "defect detection (scratch / pit)", "concentric rings", "pie segments", etc.

[0059] As described above, the quick browse sliders within selection regions 319 and 329 enable the user to quickly browse layouts and groups (e.g., the selected group or groups of selected layouts are immediately displayed in the display region 360 for light source display). In the example currently shown in FIGS. 3A and 3B, in display region 310, the layout list is shown as having a total of 17 layouts, and layout 1 of those layouts is currently selected by selection indicator SI (e.g., as shown in column 310A), and is shown to the right of the quick browse scroll bar in selection region 319 where number 1 is displayed. As also shown in this example, layout 1 consists of 12 groups of lighting channels (indicated, for example, by number 12 at the right end of the range within selection region 329), and group 3 (indicated by selection indicator SI within display region 320 and by number 3 to the right of selection region 329) is currently selected. Correspondingly, group 3 of layout 1 is displayed in display region 360 (i.e., in a pie V shape) together with selection indicator SI for each of the lighting channels included in the group of lighting channels corresponding to group 3 (i.e., including 3 lighting channels in the outer ring of the lighting configuration, 2 lighting channels in the inner ring of the lighting configuration, and 1 lighting channel at the center of the lighting configuration, which together form a pie V shape).

[0060] In one example, the lighting configuration corresponding to the light source display within display region 360 may correspond to the type of dome light. In another example, (e.g., in relation to the exemplary configuration of FIG. 2,) the central lighting channel corresponds to light source 240 (e.g., where this configuration provides coaxial lighting), and the remaining lighting channels may correspond to the lighting channels within lighting configuration 230 (which may be, for example, an annular lighting configuration such as a ring light, or a dome light without a central lighting channel in various embodiments).

[0061] In various embodiments, the system can be configured such that multiple groups of lighting channels can be selected in the group list of the display area 320 (e.g., the user may perform a special action such as shift-clicking or control-clicking to select one or more additional groups of lighting channels after the first group has been selected). Once multiple groups are selected (e.g., when multiple groups are selected), the settings of the selected groups can be controlled together. In various embodiments, all of the selected groups are shown (e.g., highlighted or provided with another type of selection indicator) in both the display areas 320 and 360 (i.e., the group list and the light source display), and any RGB or brightness adjustment will affect all of the selected groups (e.g., the adjusted settings can be made equal for all of the currently selected groups). Additionally, in various embodiments, multiple layouts (e.g., the layout list of the display area 310) can be selected at once (e.g., performed according to a selection process such as the user performing a shift-click or control-click to select multiple layouts). In such embodiments, when multiple layouts are selected, all of the groups from all of the layouts may be displayed in the group list of the display area 320, and the groups may be viewed, combined, edited, deleted, and a new combination of groups may be saved as a new layout as needed. In various embodiments, one or more additional constraints that limit what can be done in the graphical user interface (e.g., implementing mutually exclusive lighting channel grouping, color lighting channel / range limiting, etc.) may also be defined, stored, and / or loaded in the parameters of the layout (e.g., the layout definition).

[0062] In various embodiments, the layout parameters of the display area 330 may allow modification of layout-specific parameters. In various embodiments, the parameters may be dynamically configured based on information loaded for a selected layout (e.g., some segments for pie segment illumination, some rings for ring illumination, some illumination channels grouped into “point sources” in directional illumination, some examples of such groups of illumination channels are shown in FIG. 4). In various embodiments, some non-geometric parameters may also be represented.

[0063] In various embodiments, the current lighting settings for a layout may be saved and later loaded (e.g., called from memory or the like). In various embodiments, a user may define, save, and load multiple specific lighting settings for a particular layout. In various embodiments, these settings may be stored in different files. For example, the system may be configured such that a user can load one layout and then save / load multiple lighting settings for the same layout. In various embodiments, some lighting settings (e.g., some default lighting settings) may be saved with the layout (e.g., as part of the same file for the layout), but the ability to store and recall many lighting settings for the same layout provides additional flexibility. In various embodiments, the lighting settings may be stored in a file that is connected, linked, or otherwise associated with the layout (e.g., via a naming convention or by being included in a directory named after the layout) (e.g., together with the layout definition file). As one example, for a layout stored in a file named "RingLight", the corresponding lighting settings are stored in files named "RingLight_scheme001", "RingLight_scheme002". As an alternative to the term "scheme" in such file names, and / or as a reference to the lighting settings, terms such as "configuration", "pattern", "settings", "set", etc. may be used in such file names and / or otherwise used to refer to the lighting settings.

[0064] In various embodiments, the display area 340 enables a certain logical operation for all lighting channels simultaneously (e.g., to enable modification / experimentation with various settings). In various embodiments, the user is also enabled to select the operation of all channels belonging to a plurality of groups of lighting channels (e.g., shared between two or more groups of lighting channels). In various embodiments, the direct setting of the selection area 341A is applied to the RGB settings directly selected for each lighting channel. In various embodiments, the "negate to 0" of the selection area 341B calculates one setting for each RGB component of the lighting channel and clamps the resulting number to [0, 1]. The "complete negate" of the selection area 341C multiplies the setting of each RGB component of the lighting channel by -1. The "shared channel setting operation" of the display area 342 is described below with respect to FIG. 5.

[0065] Referring briefly to FIG. 5, an example is shown in relation to the display area 342 for the shared channel setting operation, which, in one embodiment, can be used to establish how a color site group (e.g., RGB) setting affects the lighting channels shared among groups (e.g., in embodiments where such sharing is enabled, as belonging to multiple groups). For example, as shown in FIG. 5, the lighting channels belonging to the intersection INTS of the group of lighting channels 510 (e.g., corresponding to a ring shape) and the group of lighting channels 520 (e.g., corresponding to a pie segment shape) can be set according to one of the selection options 342A-342E. More specifically, the lighting channels corresponding to the intersection INTS (e.g., which corresponds to the lighting channels within the overlapping area between the groups of lighting channels 510 and 520) can be set according to selection option 342A (i.e., for the average of the two groups), selection option 342B (i.e., for the sum of the two groups), selection option 342C (i.e., for the product of the two groups), selection option 342D (i.e., for the minimum of the two groups), or selection option 342E (i.e., for the maximum of the two groups) with respect to the RGB settings of the two groups of lighting channels 510 and 520. The operation selected for the shared lighting channel is used by the lighting control system (e.g., during automatic lighting adjustment processing, such as during lighting optimization processing, etc.).

[0066] Returning to FIGS. 3A and 3C, the "Main Settings" in the display area 350 corresponds to RGB lighting control that displays and enables adjustment of the group of lighting channels (e.g., in some cases, a single lighting channel) selected in the light source display in the display area 360. In various embodiments, selection of a single lighting channel in the Light Source Display in the display area 360 may, if any, result in automatic selection of the entire group of lighting channels to which the single lighting channel belongs. In various embodiments, negative settings (e.g., for RGB lighting control) may be enabled and displayed (e.g., in the display of the display area 380 in the workpiece). For example, such a display may include obtaining two images, one having an arbitrary positive lighting channel weight and the other having the absolute value of an arbitrary negative lighting channel weight, then subtracting the image illuminated with the negative lighting channel weight from the image illuminated with the positive lighting channel weight, and then displaying the resulting difference image using a designated representation for any negative color weight regions within the image.

[0067] In various embodiments, the Undo and Redo of selection options 353A and 353B enable the user to change back and forth between any recent RGB setting changes. In various embodiments, the Gang selection option of selection area 354 may set all three of the RGB sliders of selection option 351 to the same setting (e.g., the average setting of the three sliders), enabling the three sliders to be adjusted together. In various embodiments, the up and down arrows of selection options 352A and 352B enable brightness adjustment while maintaining the color (RGB) balance (e.g., of the selected lighting channel or group of lighting channels). In various embodiments, the user selection (e.g., pressing) of one of the arrows multiplies the RGB components by the same coefficient, and in various embodiments, the result may be clamped to [0,1]. In various implementations, such controls may generally be implemented or made available only when all of the RGB settings are positive. In various implementations, the brightness adjustment using the arrows may be blocked or indicated as not available when a limit is reached, such as when any further adjustment is not permitted downward when any of the RGB values reaches 0, or when any further adjustment is not permitted upward when one of the RGB values reaches 1.

[0068] In various embodiments, the selection options within display area 370 can function essentially the same as the selection options within display area 350, except as otherwise separately described below. In various embodiments, the selection option 371 of display area 370 can enable applying a preset "override" setting to a group of lighting channels while performing a predetermined type of adjustment. For example, the "maximum brightness" setting may be selected, such as by using the upward arrow of selection option 374A, and then a scroll operation may be performed. For example, the quick browse slider of selection area 329 may be utilized through different groups of lighting channels (such as a ring group or a pie segment group) so as to view the effect of the selected setting (such as the maximum brightness setting) on the workpiece for different groups of lighting channels having the selected setting. In various embodiments, each group of lighting channels returns to its "normal" (e.g., main) setting when not selected (e.g., when a quick scroll operation moves to the next group in the group list of display area 320). In various embodiments, when the "Other channels OFF" checkbox option in selection area 372 is selected, the "Override settings for selected" checkbox in selection area 371 is selected and, when enabled, turns off all currently unselected groups of lighting channels.

[0069] In various embodiments, the "Light Source Display" of the display area 360 provides all visual representations of the lighting channels in a configuration similar to the actual hardware of the lighting configuration LC. In various embodiments, the light source display can display the current settings when the radio button of the "Settings display" in the selection area 366B is selected, and can display the lighting channel grouping when the radio button of the "Group display" in the selection area 366A is selected. In various embodiments, when the settings display is selected in the selection area 366B, the true selected RGB color of the selected lighting channel can be displayed. In various embodiments, lighting channels having at least one negative RGB weight can be indicated by indicators (e.g., special colors such as black, boundaries, dynamic patterns such as blinking colors).

[0070] In various embodiments, when the "Group display" in the selection area 366A is selected, the group of one or more currently selected lighting channels (e.g., group 3 indicated by the selection indicator SI in the group list of the display area 320) can be highlighted. This setting can enable the selection, creation, and / or deletion of lighting channels (e.g., by selecting a lighting channel, selecting an action from a menu, and then making a selection such as a right click). In various embodiments, the selection and / or deselection of multiple lighting channels can be performed by various operations (e.g., moving a selector using the mouse on channels corresponding to control clicks, lasso, using a bounding box, path-based selection, etc.). In various embodiments, lighting channels not assigned to any group within the currently selected layout can be marked with a special color or pattern to indicate the corresponding status (e.g., to distinguish them).

[0071] In various embodiments, the "Expand", "Rotate", "Direct" controls (e.g., UI icons or buttons for controlling the system) for the selection regions 362, 363, and 364 can be adaptive controls (e.g., enabled / disabled depending on the currently selected / loaded layout) and will act on one or more selected groups of lighting channels. In various embodiments, the "Expand" option for the selection area 362 may be utilized for a group of lighting channels forming concentric rings or other similar geometric configurations, and for this purpose, making a selection on this control (e.g., dragging the mouse cursor) can activate smaller and / or larger rings (e.g., in embodiments where the lighting configuration includes a full or partial dome light, these functions effectively move the ring of light between the center and outer edge of the dome light). In various embodiments, the "Rotate" option for the selection region 363 may be utilized for configurations such as a group of lighting channels forming pie segments, or similar geometric arrangements, and for this purpose, selection of this control (e.g., dragging the mouse cursor on the control) "rotates" the segments around the lighting configuration, effectively changing the 2D (in-plane) direction of the lighting.

[0072] "Direct" control of the selection area 364 in various embodiments can be utilized for a group of illumination channels that form a directional light source (such as those shown by examples 442, 443, 444 in FIG. 4). In various embodiments, the use of the direct control 364 (such as by dragging a mouse cursor over this control) can act to rotate a trackball and can change the 3D direction of the illumination (such as a focused illumination beam) of the group of illumination channels. In various implementations, instead of or in addition to the direct control 364 (which is for determining the direction of illumination of a group of illumination channels), an external input device such as a physical trackball and / or joystick (such as the input device 138 in FIG. 2) may be utilized to provide similar functionality for certain controls (such as rotation and direct control of the selection areas 363 and 364). In various embodiments, the "Force exclusive groups" checkbox of the selection area 365 prevents the illumination channels from being assigned to multiple groups of illumination channels (such as preventing the illumination channels from being shared among multiple groups of illumination channels within a layout). Such a setting may be desirable in some embodiments.

[0073] In various embodiments, a hardware configuration and / or a light source driver defined at the factory may be provided and loaded by a controller for a graphical user interface to establish graphical user interface control and parameters (e.g., the geometric layout of the light source display of the display area 360, the range of the slider, and the types of possible adjustments such as RGB settings and / or luminance related to the ability to utilize negative RGB settings). Alternatively or additionally, in some embodiments, a certain type of calibration routine may be utilized to determine lighting configuration options. For example, in one embodiment, a certain type of calibration sphere or other calibration object may be placed within the field of view (e.g., under the camera 260 and illuminated by the lighting configuration), each available control of the lighting configuration may be adjusted, an image may be captured for each modification, and this image may be analyzed to determine the available options / lighting channels of the lighting configuration (e.g., for use in determining available lighting channel grouping / optimization, etc.).

[0074] Regarding the workpiece display in the display area 380, in various implementations, a live or composite camera view of the workpiece 20 may be displayed. In various implementations, the composite view may be provided when negative RGB weights are utilized, in which case two images may be collected and subtracted, and with respect to which, as described herein, a representation is provided for any negative RGB result. In the illustrated example, in the upper display area 380A, an example A is provided corresponding to the attention edge selection area of the edge detection tool EDT. This shows an example where it may be desirable to determine lighting optimization for robust edge detection. In the lower display area 380B, a defect area selection target area (e.g., as part of the defect detection tool DDT to detect the defect DFT) is shown as part of Example B. This shows an example where it may be desirable to determine lighting optimization for defect detection.

[0075] In various embodiments, such an edge detection tool EDT or defect detection tool DDT may be contextually displayed in the display area 390 depending on, for example, whether a selection is made to optimize illumination for one of the following: edge detection (e.g., of the selection area 391A), defect detection (e.g., of the selection area 391B), or focus point group (e.g., of the selection area 391C). Generally, the "Optimize for:" option in the display area 390 may be characterized as being for selecting for which type of illumination optimization mode the automatic illumination optimization process is to be performed (e.g., related to establishing which adjustment algorithms, performance metrics, etc. may be utilized in relation to the optimization of illumination for a particular mode).

[0076] In various embodiments, the "Optimize:" option within the display area 395 may be utilized to select which parameters (i.e., separate from the position of illumination channels, etc.) may be adjusted or optimized for optimization. Selection of the luminance checkbox (i.e., in the selection area 396A) automatically selects all three of R, G, and B (i.e., in the case of red, green, and blue illumination). When the brightness checkbox within the selection area 396A is deselected, any of the R, G, and / or B options within the selection areas 396B, 396C, and 396D may be selected individually or in combination. It is understood that the examples shown as configurable parameters are not intended to be exhaustive, and that other parameters that may be configurable may be provided along with the selection options in the corresponding selection areas of the graphical user interface.

[0077] FIG. 4 shows an example of different groups of illumination channels (e.g., which can be represented according to a particular shape). The shape of FIG. 4 is for illustrative purposes only, and depending on the density of the illumination channels in a given illumination configuration, it will be understood that the illustrated shape can be considered to correspond to individual groups of illumination channels that occur within the general area of the shape. The groups of illumination channels in FIG. 4 are taken as a set including sets 410, 420, 430, 440, 450, 460, and 470.

[0078] Set 410 includes illumination channel group 411, illumination channel group 412, illumination channel group 413, and illumination channel group 414. Set 410 can be characterized as corresponding to an illumination channel group "dark field" (e.g., which in some implementations may be characterized as being at an angle of 45 degrees to 90 degrees from the optical axis of the lens / camera, and in some cases may be asymmetric). Set 420 includes illumination channel group 421, illumination channel group 422, illumination channel group 423, and illumination channel group 424. Set 420 can be characterized as corresponding to an illumination channel group "bright field" (e.g., which in some embodiments may be characterized as being at an angle of 0 degrees to 45 degrees from the optical axis, and in some cases may be asymmetric).

[0079] Set 430 includes lighting channel group 431, lighting channel group 432, lighting channel group 433, and lighting channel group 434. Set 430 can be characterized as corresponding to a lighting channel group "diffusion" (e.g., which can correspond to a larger group of lighting channels that can be symmetric or asymmetric). Set 440 includes lighting channel group 441, lighting channel group 442, lighting channel group 443, and lighting channel group 444. Set 440 can be characterized as a group of lighting channels that is "directional" (e.g., which can be an example of a typically asymmetric layout that can include a small, concentrated group of lighting channels as can be in some embodiments).

[0080] Set 450 includes lighting channel group 451 and lighting channel group 452. Set 450 can be characterized as corresponding to a lighting channel group "grazing" (e.g., which in some embodiments may be characterized as corresponding to a dark field at an angle close to 90 degrees from the optical axis of the lens / camera and which can be asymmetric / directional in some examples). Set 460 includes lighting channel group 461, lighting channel group 462, lighting channel group 463, and lighting channel group 464. Set 460 can be characterized as a "concentric ring" type of lighting channel group (e.g., which in some embodiments can be characterized as essentially including examples of both "dark field" and "bright field").

[0081] Set 470 includes lighting channel group 471, lighting channel group 472, lighting channel group 473, and lighting channel group 474. In some embodiments, set 470 may be characterized as a group of lighting channels "pie segments" (e.g., each group of lighting channels forms a shape corresponding to one or more pie segments). It will be appreciated that certain defect detection processes (e.g., for identifying scratches, pits, etc. on the surface of a workpiece) may utilize, in some embodiments, groups of "dark field", "grazing", and / or "concentric ring" type lighting channels (e.g., some examples are shown in sets 410, 450, and 460).

[0082] FIG. 6 shows an embodiment of a technique for representing negative color weights (e.g., an image IMG of workpiece displays 380 of FIGS. 3A and 3C for displaying workpiece WP illuminated by a current lighting setting). As described above with respect to display areas 350 and 370, a particular color light setting may have a negative weight. In the example shown in display areas 350 and 370 of FIGS. 3A and 3C, the red light setting is shown to have a negative weight corresponding to a value of -0.50. Since the actual negative value of a color light setting cannot be displayed within an image, one technique is to instead display a negative weight indicator area for indicating areas on a workpiece having a negative color weight. In the example of FIG. 6, four negative weight indicator areas NWIA are displayed so as to respectively correspond to areas where at least one component of the displayed color is negative.

[0083] As described above, in various embodiments, a setting of light of a negative color (e.g., with respect to a setting of light of red, green, and / or blue) may be enabled. In various embodiments, image feedback in the graphical user interface 300 may be provided when a negative color illumination channel value is present in a particular image area (e.g., as shown in FIG. 6). In various embodiments, such a representation may be determined by obtaining two images including an image illuminated with the weight of the "negative" illumination channel, the image being subtracted from the image illuminated with the weight of the positive illumination channel. In various embodiments, a resulting difference image including a representation of negative values within the image may be displayed. For example, appropriate colors, patterns, graphic effects, etc. may be utilized as part of a mapping rule for representing a negative color illumination channel value in the displayed image (e.g., as indicated by the negative weight indicator area NWIA in FIG. 6).

[0084] In various embodiments, various techniques can be utilized to display the weights of the negative color illumination channels. In some embodiments, a false color technique can be utilized that includes mapping the ranges of negative and positive values to color ranges that may or may not have a basis in the actual representation. Such may utilize the concept of a color map (e.g., a look-up table) that maps input values to specific colors. In one embodiment, the mapping includes calculating the grayscale value of all RGB triplets in a standard way that includes using the absolute value, and then negating the grayscale value if at least one of the RGB components is negative and the possible range of -1 to 1 of the grayscale value (e.g., scaled from -255 to 255) can be used to index into the color map. In various embodiments, a separate texture (e.g., stripes, checkerboard) that can be static or dynamic (e.g., utilizing some other form of movement, blinking, shimmering, or spatio-temporal modulation) can be utilized to highlight special negative values in the displayed image. In various embodiments, several different colors (e.g., including primary and / or saturation such as red, green, blue, or a distinctively strong combination of RGB depending on which channel is negative) may be utilized, and they can be static or dynamic (e.g., blinking at a value-dependent frequency or a constant frequency).

[0085] It will be understood that generally any method may be utilized to emphasize the presence of negative RGB values within an image. In some embodiments, it may be desirable to maximize the contrast of areas where negative RGB values appear (e.g., using colors not present in the image, using grayscale if the image is colorful, or using colors if the image is grayscale). With respect to the example of FIG. 6, if the remainder of the image IMG is grayscale, the negative weight indicator area NWIA may be provided in a color (e.g., a bright color easily distinguishable from grayscale such as bright blue). As another example, if the remainder of the image IMG is color, the negative weight indicator area NWIA may be provided in grayscale or another format (e.g., generally contrasting with the remainder of the image IMG). In various embodiments, the negative weight indicator area NWIA may indicate that at least one of red, green, or blue has a negative value.

[0086] FIG. 7 is a flow diagram of a routine 700 for operating a system having a group of lighting channels. At block 710, a representation of a group of lighting channels within a display area is provided. At block 720, a determination is made that a group of lighting channels has been selected, the selected group of lighting channels including a plurality of lighting channels. At block 730, a current lighting setting for the selected group of lighting channels is displayed and an adjustment to the lighting setting for the group of lighting channels is applied to all of the lighting channels within the group.

[0087] In various embodiments, a scan line and / or one or more edge tools are utilized to determine the position of an edge on a workpiece (e.g., by edge detector 144). FIGS. 8-10 below are described with respect to exemplary operation of edge tools (e.g., point and box edge tools), which may be “video tools” that can utilize a scan line to determine an edge position in some embodiments and may be similar thereto. For example, in various embodiments, tools 143a and / or 143n of video tool section 143 of FIG. 2 may be edge tools (e.g., which may be utilized by or in combination with edge detector 144 as may be utilized in various embodiments), and may include utilization of features of the edge tools. In various embodiments, similar operations (e.g., including utilization of a scan line or similar techniques) can be performed to determine an edge position, which may, in some cases, be automatically performed without otherwise displaying and / or utilizing other features of the “video tool” or features of other tool types illustrated and described below with respect to the simplified examples of FIGS. 8-10. Various similar video tools are described in U.S. Pat. Nos. 7,003,161; 7,030,351; 7,522,763; 7,567,713; and 7,627,162, each of which is incorporated herein by reference.

[0088] FIG. 8 is a diagram of an exemplary point tool 810 that overlays a first edge 805 of a portion of workpiece 20 at the boundary of a darker or shaded region 808 within image 800 as acquired by camera 260. In FIG. 8 and the figures described below, shaded areas such as shaded area 808 are shown for illustration purposes to indicate pixels having a relatively high “gray value” (e.g., or other color value) within the image. In various embodiments, a higher gray value (e.g., or other color value) may correspond to a pixel of relatively low intensity or relatively high intensity, depending on the configuration.

[0089] As will be described in more detail below, the point tool 810 may be configured to determine the positions of edge points on an edge within an image, and a similar operation may form the basis of the operation of another type of tool (e.g., a box-shaped edge tool) that places a plurality of edge points on an edge, as will be described in more detail below with reference to FIG. 9. In some embodiments, the point tool 810 may include a body 811, an edge selector 812, and an arrow 814 that is a polarity indicator. The arrow 814 may generally point from light to dark or from dark to light across an edge so as to improve edge discovery reliability in a certain situation, as described in the incorporated references. In the illustration of FIG. 8, the body 811 covers a scan line 820 that is defined by the body 811 and nominally coincides with the body 811. The scan lines (820 and 920) will also be illustrated and described in more detail below with reference to FIG. 9.

[0090] In various embodiments, some of the operations described below (e.g., using a scan line to determine an edge position) may be automatically performed by the machine vision inspection system 100 (e.g., without displaying corresponding video tool features on a display). During operation, the machine vision inspection system 100 may be configured to automatically select and utilize a point tool and / or a corresponding operation (e.g., using a scan line), or a user may select the point tool 810 or a corresponding operation. The system or the user can identify the detected edge feature by placing the body 811 of the point tool on the edge feature and placing the edge selector 812 as close as possible to the edge at a desired position along the edge. The body 811 of the point tool defines a desired scan line orientation across an edge portion (e.g., parallel to the x-axis of the x-y coordinate system in the illustrated example for simplicity of explanation, or alternatively, at an angle with respect to the x-axis and y-axis within the x-y coordinate system in some embodiments) and can be oriented as shown. In FIG. 8 and other figures herein, image pixels are arranged in rows along the x-coordinate direction and columns along the y-coordinate direction. Arrow 814 points along a reference direction or polarity associated with edge detection. During operation, when the point tool 810 is configured (and / or when a corresponding operation is automatically determined and / or executed), instructions of a routine for identifying the position of the underlying edge point are executed to perform an operation of analyzing intensity profile data points (e.g., pixel intensity data) associated with a scan line that nominally coincides with the body 811 of the point tool 810, and various operations for detecting the edge position of the underlying feature can be performed. As will be described in more detail below with respect to FIG. 10, in various exemplary embodiments, the edge point position identification routine of the point tool 810 can determine the edge position based on the magnitude of the gradient along the intensity profile associated with the scan line.

[0091] FIG. 9 is a diagram of an exemplary box tool 910 that overlaps a first edge 905 of a portion of workpiece 20 at the boundary of shadow region 908 within image 900. In various embodiments, the edge detection tool EDT of FIG. 3C may be a box tool such as that of FIG. 9 and / or may operate in a manner similar to such a box tool. In various embodiments, box tool 910 may include a region of interest (ROI) indicator 911, a polarity indicator arrow 914 along a side, and an edge selector 912 disposed on a desired edge to be detected. Within box tool 910, a nominal scan line 920 that is generated and used by box tool 910 to determine an edge point may be shown. Along each of nominal scan lines 920, the box tool operates to determine an edge point of a feature below, as described above for scan line 820 of point tool 810 of FIG. 8. As shown in FIG. 9 and described in more detail below, line 925 may be fitted to a set of determined edge points to determine the position and orientation of edge 905.

[0092] During operation, the box tool 910 is selected and / or otherwise configured to identify edge features to be detected. The ROI indicator 911 may be positioned, sized, and rotated (e.g., automatically or by an operator) such that the ROI includes a portion of the edge feature to be detected, and the edge selector 912 may be positioned to more accurately identify the edge to be detected at a desired location along the edge. The ROI indicator 911 may be oriented to define and indicate a desired scan line orientation across the edge portion. More generally, the orientation of the overall ROI indicator 911, the portion of the ROI indicator including the arrow 914, or the orientation and / or corresponding operation of the nominal scan line 920 may each be used to define and / or indicate the scan line orientation. The arrow 914 defines the polarity associated with the edge detection. Once the box tool 910 is configured, the instructions of the underlying edge point identification routine may be executed to analyze the intensity profile data to detect edge points along each of the scan lines and perform the operation of fitting a line to the detected edge points, as described in more detail below.

[0093] In summary, generally, edge points may be determined by various tools and / or corresponding operations, and in various implementations, (e.g., as part of a process for determining the exact location of the edge of a feature on a workpiece,) a geometric shape may be fitted to the edge points to determine the location of the corresponding underlying image features. In one conventional method of operating a tool, one or more nominal scan lines are defined or generated (e.g., within or outside the ROI according to defined tool parameters) depending on the type of tool (e.g., point tool and box tool). For each nominal scan line, a set of intensity profile data point positions approximating the nominal scan line is determined. To define the intensity profile associated with the nominal scan line, intensity values associated with the data point positions are determined. Then, in one embodiment, an edge detection algorithm analyzes the gradient along the intensity profile to find the position along the intensity profile corresponding to the magnitude of the maximum gradient, as will be described in more detail below with reference to FIG. 10. The maximum gradient position along the intensity profile is used to determine the edge point position in the image associated with the nominal scan line.

[0094] FIG. 10 is a diagram of a graph 1000 showing one exemplary method of determining an edge position based on an intensity profile 1010. The intensity profile 1010 includes a set of image pixel intensity values 1025 (e.g., gray or other color) corresponding to the position of a scan line (e.g., one of the scan lines 820 or 920 shown in FIGS. 8 or 9, respectively). The data points or positions representing the scan line are labeled as "pixel numbers" from 0 to 35 along the horizontal axis. Starting from data point 0, the pixel intensity values first show a relatively bright region up to approximately data point 23, followed by a relatively dark region up to data point 35.

[0095] It will be understood that the values and figures of FIGS. 8, 9 and 10 are simplified in various respects with respect to the current example. For example, in FIGS. 8 and 9, the determined edge positions are shown as being between a relatively bright region (e.g., almost white) and a dark region, but in some actual embodiments, the "brighter region" may have a higher gray or other color value. Further, the edges in FIGS. 8 and 9 are shown as being relatively "sharp" (i.e., with a relatively steep transition between the bright and dark regions), but in certain actual embodiments, the edge transition can occur over several pixels (as shown, for example, by the exemplary gray or other color value transition from pixel 23 to pixel 27 in FIG. 10).

[0096] The gradient intensity value 1026 is derived from the pixel intensity value 1025 and is also shown in FIG. 10. Using various conventional algorithms, the position along the horizontal axis corresponding to the peak of the gradient magnitude (which is considered to indicate the peak “contrast” between pixel intensity values) can be found and identified as the edge position. If there are multiple peaks of gradient magnitude, a video tool edge selector and / or orientation indicator (e.g., edge selector 812 and / or polarity indicator 814 of point tool 810) can be used to assist the algorithm in identifying the desired peak. In FIG. 10, the maximum gradient criterion (i.e., the gradient peak) indicates that the edge is located approximately at data point 23. By using methods such as curve fitting and centroid determination, the peak of the gradient magnitude can be located relatively accurately between the intensity profile data points, which generally supports sub-pixel measurement resolution and reproducibility when determining the position of the corresponding edge within the image. For example, in the example of FIG. 10, the gradient peak is shown to be approximately at data point 23 (e.g., corresponding to pixel 23 in this example), but it can be seen that another “near peak” occurs at data point 24 (pixel 24). This indicates that a more accurate determination of the edge position can be determined by processes such as curve fitting and centroid determination and can correspond to a position between data points 23 and 24 (pixels 23 and 24). As a simplified concept, in an example where data points 23 and 24 have the same gradient magnitude, this can correspond to an edge position exactly halfway between the two (e.g., an edge position corresponding to a data point / pixel position of 23.5).

[0097] The examples of FIGS. 8-10 are shown and described in connection with the preferred illumination provided by the illumination configuration LC (using illumination as determined, for example, by the illumination optimization process as described herein). It will be understood that if less desirable / less optimal illumination is provided, it can result in significantly smaller and / or less distinct transitions of the data points as corresponding to the edges. For example, the pixel values on different sides of the edge transition (e.g., pixels 0-23 as compared to pixels 24-35) may have a much smaller difference, and the gradient peak may be much lower, which can result in, for example, a less accurate detection / determination of the position of the edge. Thus, for illumination optimization performed in connection with edge detection, ideally, at least one side region of the edge has a maximized contrast with the region on the other side of the edge, and it may be desirable to result in a significant proportion of the pixels near the edge not being saturated. Note that such characteristics / goals of illumination optimization for edge detection may be different from the characteristics / goals of illumination optimization for certain other types of processing (e.g., defect detection and / or points from focus point group processing).

[0098] FIG. 11 shows the use of one or more video tools to perform a measurement operation on an image of a workpiece that includes workpiece features (e.g., to determine dimensions of workpiece features). In one example, workpiece feature 1102 may correspond to a scratch defect or other type of workpiece feature. As shown, for image 1101 that includes workpiece feature 1102, video box tool 1106 includes scan lines 1108 (e.g., which may alternatively or additionally represent a video point tool) that are used to determine the edge position, dimensions, and / or other aspects of workpiece feature 1102. In various exemplary embodiments, video box tool 1106 may be sized, positioned, and rotated until box tool 1106 indicates or defines a region of interest (e.g., a region within box tool 1106), and the arrows shown in FIG. 11 (e.g., scan lines, point tools, etc.) may be used to determine the edges of workpiece feature 1102. In various exemplary embodiments, video box tool 1106 may generally use one or more conventional edge gradients along the edges of workpiece feature 1102 within the region of interest, and the edges of workpiece feature 1102 may be determined based on, for example, the local magnitude of the edge gradient along various scan lines 1108.

[0099] In various exemplary embodiments, such measurement operations may also include performing some morphological filtering or other filtering (e.g., a particular type of such filtering for distinguishing workpiece features / scratch edges from the machining pattern of the workpiece is described in U.S. Patent 7,522,763, which is hereby incorporated by reference in its entirety). As shown in FIG. 11, in the display area included in the image, the box tool 1106 having the scan line 1108 is utilized to determine the edge position (e.g., outer edge or outer perimeter) of the workpiece feature 1102. Based on such determination, the video tool and / or other measurement operations may include determining the dimension D1 of the workpiece feature (e.g., corresponding to the length or other dimension of the workpiece feature 1102). In each embodiment, the box tool 1106, the scan line 1108, and / or other dimensional workpiece features 1102 (e.g., width, depth) of the workpiece that can be utilized in the determination of other video tools and length measurement operations may be used. For example, the focus point group unit 146 can perform operations for determining the feature of the dimension of the height in the Z direction of the workpiece from the three-dimensional surface shape data (e.g., including determining the depth of the workpiece feature / scratch relative to other parts or features of the workpiece surface, etc.).

[0100] As described above, the workpiece feature 1102 may, in some embodiments, correspond to a scratch defect 1102 on the workpiece. In various embodiments, with respect to the operations of the defect detection unit 145 and / or the illumination optimization unit 147, the user can provide an image with examples of scratch defects similar to the scratch defect 1102. As an example, the provided image may be of a similar part of the workpiece (e.g., the type of machining marks formed on the surface of the part is similar or nominally the same in each image, and the main difference between the images is the characteristic of each scratch defect). In one exemplary embodiment, such an image may be included as part of a learning set of images for learning.

[0101] In one exemplary embodiment, the image of FIG. 11 can be an example of an image that can be analyzed by the defect detection unit 145 to detect the scratch defect 1102. In various embodiments, the defect detection unit 145 may be sufficiently trained to be able to appropriately detect the scratch defect 1102 (for example, trained using training images including similar portions of the workpiece and scratch defects having specific similar characteristics to the scratch defect 1102). In various embodiments, some additional processing may be performed with respect to the exemplary scratch defect 1102. For example, one or more measurement processes may be performed in conjunction with the defect detection process in which various dimensions or other characteristics of the scratch defect 1102 can be determined.

[0102] As part of the general operation of the defect detection unit 145, some of the detected defects may warrant further measurement or inspection to determine additional defect parameters. For example, as described above, various types of analysis and / or processing of an image including the scratch defect 1102 (for example, by utilizing the edge detection unit 144) can enable the determination of the XY position and the approximation of the XY area and / or other dimensions of the scratch defect 1102 (for example, by utilizing video tools and / or other operations as described above). If the three-dimensional nature of the possible defects is important (for example, if the scratch must be deeper than some value considered a defect), the defect detection unit 145 may initiate focus point cloud processing (for example, by the focus point cloud unit 146 that acquires the 3D point cloud of the affected area to determine the depth of the scratch defect 1102). Details will be described later, but different lighting optimizations may be adapted to these different types of processing (for example, defect detection, edge detection, focus point cloud).

[0103] FIG. 12 shows the use of one or more video tools to perform a measurement operation on a workpiece feature within an image of a workpiece (e.g., to determine dimensions of the workpiece feature). In one example, workpiece feature 1202 may correspond to a defect or other type of workpiece feature (e.g., similar to the example of FIG. 11). As shown, for image 1202 including workpiece feature 1204, video box tool 1206 includes scan line 1208 that is utilized to determine the edge position, dimensions, and / or other aspects of workpiece feature 1204 (e.g., this may represent, additionally or alternatively, a video point tool, etc.). In various exemplary embodiments, video box tool 1206 may be sized, positioned, and rotated until box tool 1206 indicates or defines a region of interest (e.g., a region within box tool 1206), and the arrows shown in FIG. 12 (e.g., representing scan lines, point tools, etc.) may be utilized to determine the edges of workpiece feature 1204. In various exemplary embodiments, video box tool 1206 may generally use one or more conventional edge gradients along the edges of workpiece feature 1204 within the region of interest, and the edges of workpiece feature 1204 may be determined based on, for example, the local magnitude of the edge gradient along various scan lines 1208.

[0104] Additionally or alternatively, various other processes may be performed, such as threshold processing and / or binary image formation (e.g., as part of the defect detection process). As shown in FIG. 12, in the display area included in the image, the box tool 1206 having the scan line 1208 is utilized to determine the edge position (e.g., outer edge or outer perimeter) of the workpiece feature 1204. Based on such determination, video tools and / or other measurement operations may include determining the dimension D2 of the workpiece feature (e.g., corresponding to the length or other dimension of the workpiece feature 1204). In each embodiment, the box tool 1206, the scan wiring 1208, and / or other video tools and / or measurement operations may be used to determine the dimensions (e.g., width, depth) of the workpiece feature 1204. For example, the focus point group section 146 can determine the features of the dimension of the height in the Z direction of the workpiece from the three-dimensional surface shape data (e.g., including determining the depth of the workpiece feature / defect relative to other parts or features of the workpiece surface).

[0105] As described above, similar to the example of FIG. 11, the workpiece feature 1202 may correspond to a defect 1202 on the workpiece in some embodiments. In various embodiments, with respect to the operation of the defect detection unit 145 and / or the illumination optimization unit 147, the user can provide an image with an example of a defect similar to the defect 1202. As an example, the provided image may be a similar part of the workpiece (e.g., the type of the surface of the part is similar or nominally the same in each image, and the main difference between the images is the characteristics of each defect). In one exemplary embodiment, such an image may be included as part of a learning set of images for learning.

[0106] In one exemplary embodiment, the image of FIG. 12 can be an example of an image that can be analyzed by the defect detection unit 145 to detect the defect 1202. In various embodiments, the defect detection unit 145 may be trained to appropriately detect the defect 1202 (e.g., trained with training images including similar portions of the workpiece and defects having specific similar characteristics as the defect 1202). In various embodiments, some additional processing can be performed with respect to the exemplary defect 1202. For example, one or more measurement processes can be performed in combination with the defect detection process, and for this purpose, various dimensions or other characteristics of the defect 1202 can be determined. As will be described in more detail below, different illumination optimizations can correspond to each of these different types of processes (e.g., defect detection, edge detection, focus point cloud).

[0107] FIGS. 13A-13C are diagrams showing some principles regarding a defect detection process (e.g., performed by the defect detection unit 145). In various embodiments, as part of such a defect detection process, it may be desirable to provide illumination that enables visualization of the surface texture without exposing any portion of the corresponding surface downward or excessively. FIGS. 13A-13C illustrate different settings of an exemplary illumination channel and, correspondingly, illustrate how certain surface texture features and / or defects may appear more or less visible or otherwise detectable. The examples of FIGS. 13A-13C relate to the surface of a machined aluminum plate including raised portions.

[0108] Figures 13A-13C illustrate images 1300A-1300C obtained with different settings for illumination channels LCH1 and LCH2. In various embodiments, illumination channel LCH1 may correspond to a coaxial illumination channel (e.g., provided by illumination channel 240 in the example of FIG. 2). In various embodiments, illumination channel LCH2 may correspond to a single illumination channel or may represent the settings of a group of illumination channels (e.g., a dome light or a ring light as provided according to illumination configuration 230 of the example of FIG. 2). FIG. 13A shows an image 1300A obtained by illuminating a workpiece surface provided with illumination channel LCH1 having a setting of 3 and illumination channel LCH2 having a setting of 7. In the resulting image 1300A, the surface of the workpiece is somewhat dark and has some visibility of the grooves within the surface. In FIG. 13B, the setting of illumination channel LCH1 is 5 and the setting of illumination channel LCH2 is 7. This combination of illumination for illuminating the surface of the workpiece generates an image 1300B, where the surface is somewhat brighter than in image 1300A and the grooves within the workpiece surface have some different levels of visibility.

[0109] In FIG. 13C, the setting of the illumination channel LCH1 is 0, and the setting of the illumination channel LCH2 is 11. These settings for the combined illumination channels illuminate the workpiece so that the image 1300C is generated. In the image 1300C, there are certain features on the workpiece surface that are more visible compared to the images 1300A and 1300B, and it can be seen that the visibility of such features is low in the images 1300A and 1300B. For example, a certain scratch that is generally horizontally oriented, such as the defect DFT shown in the upper left corner of the image, is more visible in the image 1300C. In this example, in order to make the defect relatively more visible, it is desirable for the illumination channel LCH1 to have a relatively lower setting (e.g., a setting of 0) and the illumination channel LCH2 to have a relatively higher setting (e.g., a setting of 11). As described above, in general, in the defect detection process, it may be desirable to have the visibility of a specific surface texture of the workpiece surface (e.g., corresponding to the type of defect being detected) without insufficient or excessive exposure of these portions of the workpiece surface.

[0110] FIGS. 14A and 14B are diagrams showing contrast focus curves that can be utilized for a focus point group (e.g., executed by points from the focus point group portion 146) that can result from the characteristics of illumination in corresponding regions of interest. As will be described in more detail below with respect to FIGS. 14A and 14B, in various implementations (e.g., as part of the focus point group processing), the camera 260 is utilized to acquire an image stack comprising a plurality of images of the workpiece surface with illumination, and each image of the image stack corresponds to a different Z height. Focus curve data (e.g., the visibility level of contrast can be dependent on illumination) is determined based at least in part on the analysis of the images of the image stack, and the focus curve data indicates the three-dimensional positions (e.g., including the Z height and the relative x-axis and y-axis positions) of a plurality of surface points on the workpiece surface.

[0111] As shown in FIGS. 14A and 14B, the contrast focus curves 1401 and 1402 can result from the characteristics of the illumination in the corresponding regions of interest on the workpiece. Generally, it is desirable for the illumination to provide sufficient contrast in the image such that the position of the peak of the contrast curve is accurately grasped and can be reliably distinguished from noise for all desired regions of interest. As will be described in more detail below, the focus curve data can be determined from the analysis of an image stack (e.g., as part of a focus point cloud (PFF) processing / measurement operation) that indicates the three-dimensional positions of surface points on the surface of the workpiece.

[0112] FIGS. 14A and 14B show how an image stack (e.g., having illumination provided by an illumination channel of an illumination configuration LC) acquired by the system 100 can be utilized to determine the Z-height of points on the workpiece surface. In various embodiments, the image stack is acquired by the system 100 (e.g., in accordance with operations from the focus point cloud unit 146) to determine three-dimensional profile data (e.g., including the Z-height of points on the workpiece surface, etc.). The PFF image stack may be processed to determine or output a Z-height coordinate map (e.g., a point cloud) that quantitatively represents a set of three-dimensional surface coordinates (e.g., corresponding to the surface shape or profile of the workpiece).

[0113] In the PFF analysis described herein, each of the focus curves 1401 and 1402 (shown in FIG. 14A) corresponds to a single point on the workpiece surface. That is, the peak of each focus curve indicates the Z height of a single point along the direction of the optical axis OA of the vision component part 200 of the system 100. In various embodiments, the PFF-type analysis repeats this process for a plurality of surface points (e.g., each having a corresponding region of interest) across the workpiece surface so as to be able to determine the overall profile of the workpiece surface. Generally, this process may be performed on a plurality of surface points within the field of view (i.e., captured within the images of the image stack), and for each image of the image stack, a particular ROI(i) corresponds to a particular point (e.g., the center point of the ROI) on the workpiece surface.

[0114] FIGS. 14A and 14B are aligned with respect to each other along the Z-height axis shown in the figures. FIG. 14A is a representative graph 1400A showing two examples of the conforming focus curves 1401 and 1402, and FIG. 14B is a view of a variable focus image stack 1400B including two different regions of interest ROI(k), specifically ROI(1) and ROI(2). This corresponds to the data points fm(1,i) and fm(2,i) corresponding to the two different focus curves 1401 and 1402 of FIG. 14A, respectively. The region of interest ROI(k) is included in the imaged surface area of the workpiece.

[0115] Regarding the term "region of interest", it will be understood that some "single-point" autofocus tools output a single Z-height corresponding to the entire region of interest. However, known "multi-point" type autofocus tools can return multiple Z-heights corresponding to individual "sub-regions of interest" (e.g., a grid of sub-regions of interest) within the global region of interest defined by the multi-point type autofocus tool. For example, such sub-regions of interest can be defined manually and / or automatically as centered on each (or most) pixel within the global region of interest. Thus, in some cases, ROI(1) and ROI(2) can be considered representative sub-regions of interest within the global region of interest. However, the essential point is that the Z-height can be established for any defined region of autofocus interest (e.g., the region of interest of a single-point autofocus tool or a sub-region of interest within the global region of interest defined by a multi-point autofocus tool). Thus, it will be understood that when the term region of interest is used in connection with the establishment of the Z-height, such sub-regions of interest (e.g., within the global region of interest defined by a multi-point autofocus tool) can be subsumed within the meaning of the term. For the sake of simplicity of the present description, the regions of interest ROI(1) and ROI(2) are shown as being relatively small (e.g., 3×3 pixels), but it will be understood that larger regions of interest (e.g., 7×7 pixels, etc.) can be utilized in various embodiments as part of such processing and the like.

[0116] As shown in FIG. 14B, each of the images image(1) to image(11) of the image stack image (i) includes a region of interest ROI(1) located at the center where the determined focus metric value corresponds to the focus metric data point fm(1,i) on the focus curve 1401. The region of interest ROI(1) is schematically shown in FIG. 14B as including a relatively high level of contrast (e.g., image (6)) corresponding to a relatively large focus metric value shown on the focus curve 1401. Similarly, each of the images image(1) to image(11) of the image stack image (i) includes a region of interest ROI(2) located peripherally where the determined focus metric value corresponds to the focus metric data point fm(2,i) on the focus curve 1402. The region of interest ROI(2) is schematically shown in FIG. 14B as including a relatively low level of contrast (e.g., image (6)) corresponding to a relatively lower focus metric value shown on the focus curve 1402.

[0117] As shown in FIG. 14A, each focus metric value fm(1,i) or fm(2,i) can be regarded as sampling the continuous basic focus data 1401S or 1402S, respectively. In FIG. 14A, it can be seen that the underlying focus data 1401S or 1402S is relatively noisy (e.g., due to the small size of the corresponding region of interest). However, in the case of the focus curve 1401, due to the higher contrast in the corresponding region of interest, the focus metric values in the vicinity of the focus curve peak (e.g., in the vicinity of Zp1401) are relatively large compared to the size of the "noise component" in the underlying focus data. In contrast, in the case of the focus curve 1402, due to the low contrast in the corresponding region of interest, the focus metric values in the vicinity of the focus curve peak (e.g., near Zp1402) are relatively similar to the size of the "noise component" in the underlying focus data.

[0118] In one particular example, the higher focus metric value shown by focus curve 1401 is provided, at least in part, by the illumination configuration on the surface area of the region of interest ROI(1), and may be due to the illumination that results in high contrast in the focused image. In comparison, the lower focus metric value shown by focus curve 1402 is provided, at least in part, by the illumination configuration on the surface area of the region of interest ROI(2), and may be due to the illumination that results in little contrast in the focused image. In any case, due to the lower "signal-to-noise" associated with the lower peak of focus curve 1402 compared to the relatively high signal-to-noise associated with the peak of focus curve 1401, it will be understood that the estimated Z height of the focus peak Zp1402 of focus curve 1402 is less reliable or more uncertain than the estimated Z height of the focus peak Zp1401 of focus curve 1401 (e.g., in some cases, the data of focus curve 1402 may be considered to correspond to a "gap" in the focus curve data of the workpiece surface, such that the determination of the focus peak cannot be made reliably enough, and / or may be considered to be uncertain).

[0119] Briefly summarized with reference to FIGS. 14A and 14B, for the focus point group, the camera 260 may move (e.g., moved by the motor 294), or the VFL lens 270 may be utilized. To vary the focus position of the system through a range of Z-height positions Z(i) along the Z-axis (e.g., the focusing axis or the optical axis OA), the camera 260 may be utilized to capture an image (i) at each of a plurality of corresponding focus positions. For each captured image (i), a focus metric fm(k,i) may be calculated based on a region of interest or sub-region of interest ROI(k) (e.g., a set of pixels) within the image, and is associated with the corresponding focus position Z(i) along the Z-axis at which the image was captured (e.g., corresponding to the operation of the camera, or the VFL lens). As a result, focus curve data (e.g., the focus metric fm(k,i) at the position Z(i), which is a type of focus peak determination data set) is obtained, and this may simply be referred to as the "focus curve" or the "autofocus curve". In one embodiment, the focus metric value may include the calculation of the contrast or sharpness of the region of interest within the image. In various embodiments, the focus value or curve may be normalized. Various focus metric calculation techniques are described in detail in the incorporated references, and various suitable focus metric functions are also known to those skilled in the art.

[0120] The Z height (e.g., Zp1401 or Zp1402) corresponding to the peak of the focus curve corresponding to the best focus position along the Z axis is the Z height of the region of interest used to determine the focus curve. The Z height corresponding to the peak of the focus curve can be found by fitting a curve (e.g., curve 1401 or 1402) to the focus curve data (e.g., data fm(1,i) or fm(2,i)) and estimating the position of the peak of the fitted curve. The image stack image (i) is shown for illustrative purposes as including only 11 images, but it will be understood that in actual embodiments (e.g., as part of PFF processing or otherwise), a greater number of images (e.g., 100 or more, or 200 or more) may be utilized. Exemplary techniques for determining and analyzing the image stack and the focus curve are taught in U.S. Patents 6,542,180 and 8,581,162, each of which is incorporated herein by reference in its entirety. The differences between curves 1401 and 1402 and Z heights Zp1401 and Zp1402, as well as the noted accuracy differences for the determination of the corresponding 3D profile data, indicate the importance of illumination optimization for determining the desired illumination for focus point group processing (e.g., the desired illumination in region of interest ROI(1) results in curve 1401 having Z height Zp1401, and the less desirable illumination in region of interest ROI(2) results in curve 1402 having Z height Zp1402). Also note that such characteristics / goals of illumination optimization for a focus point group (e.g., which includes providing the desired illumination over a range of focus positions as opposed to a single focus position) can differ from the characteristics / goals of illumination optimization for certain other types of processing (e.g., for edge detection and / or defect detection processing, etc.).

[0121] Figures 15A to 15I are diagrams showing the determination of optimized illumination for different illumination optimization modes. Figure 15A shows an inspection system 100', which may be similar to the inspection system 100 described above with respect to Figure 2 and will be understood to operate similarly unless otherwise described below. The system 100' is shown as including an illumination configuration LC', a camera configuration 260 (which may include a lens, for example), and a workpiece stage 210. The workpiece 20 is placed on the stage 210 (i.e., illuminated by the illumination configuration LC' and imaged by the camera 260). In the simplified example of Figure 15A, the illumination configuration LC' is shown as including at least illumination channels LX, LY, and LZ.

[0122] Note that the illumination channels LX and LZ provide illumination at an angle (e.g., an angle of 45 degrees to 90 degrees from the optical axis of the camera 260 and / or the optical axis of a lens in front of the camera 260). In connection with the example of Figure 2, such illumination may be derived from a specific illumination channel of the illumination configuration 230. The illumination channel LY provides axial illumination (e.g., in connection with the example of Figure 2, this may be similar to the illumination provided by the illumination channel 240). In various embodiments, the illumination channels LX, LY, and / or LZ in the diagram of Figure 15A can represent individual illumination channels, or in some embodiments, each can represent a group of illumination channels at corresponding positions such as those shown in Figure 15A.

[0123] Regarding coaxial illumination as provided by the illumination channel LY, for the sake of simplicity in the illustration of FIG. 15A, the illumination channel LY is shown as being above the camera 260. However, in various embodiments, axial illumination can be provided in a manner similar to that shown in the example of FIG. 2 (for example, the reflective element 290 reflects the illumination from the illumination channel 240 downward in an axial path towards the workpiece 20, and the workpiece light 255 generated correspondingly passes through the element 290 along the optical path towards the camera 260). As shown in FIG. 15A, similar to the example of FIG. 2, the workpiece 20 is placed on the workpiece stage 210 (for example, the position of the stage 210 may be adjustable) such that the workpiece 20 or its desired portion is within the field of view (e.g., of the camera 260).

[0124] FIG. 15B is a side view similar to FIG. 15A of the workpiece 20 placed on the stage 210. As shown in FIG. 15B, the workpiece 20 has inclined surfaces S1 and S3 and a horizontal upper surface S2. Edge EA is shown between surface S1 and surface S2, and edge EB is shown between surface S2 and surface S3. FIG. 15C is a top view of the workpiece 20 placed on the stage 210. As shown in FIG. 15C, edge EA is between the inclined surface S1 and the horizontal upper surface S2. Edge EB is between the horizontal upper surface S2 of the workpiece 20 and the inclined surface S3.

[0125] Figures 15D and 15E are diagrams showing an embodiment in which an illumination optimization process is executed to determine illumination for illuminating workpiece 20 for a selected edge detection illumination optimization mode. More specifically, illumination is determined to emphasize edges EA and EB (e.g., to best detect edges EA and EB in connection with determining the exact positions on workpiece 20, and in one example, to be able to determine the exact distance between the edges). In various embodiments, to best detect edges EA and EB for measurement purposes, ideally, the area on at least one side of each edge has maximum contrast with the area on the other side of the edge, and a significant percentage of the pixels near the edge do not experience intensity saturation.

[0126] For such purposes, the surface S1 can be made almost dark and the surface S2 can be made almost bright (e.g., by reducing the intensity of illumination channel LX), but preferably non-saturating (e.g., making some of the surface texture visible) illumination may be determined by having a medium or medium-high intensity with respect to illumination channel LY, and to make surface S3 almost dark, illumination channel LZ may be determined to have a relatively low intensity. As a specific numerical example regarding such a principle, in one embodiment, the illumination intensity settings of illumination channels LX and LZ may be 1 percent, and the illumination intensity setting of illumination channel LY may be 64 percent. Such optimized illumination intensity produces the desired result, and edges EA and EB are emphasized in the acquired image (e.g., the image corresponding to the top view of Figure 15E) such that edge detection can be improved.

[0127] As shown in FIG. 15E, the image obtained using the optimized illumination can be utilized to determine the exact positions of edges EA and EB. In various embodiments, edge detection tools (e.g., as described herein in connection with FIGS. 8 - 10) can be utilized to determine the exact positions of edges EA and EB. Once the exact positions of edges EA and EB are determined, the distance D3 between edges EA and EB can be determined, as shown in FIG. 15E.

[0128] FIGS. 15F and 15G show embodiments in which an illumination optimization process is performed to determine the illumination for illuminating the workpiece 20 for a selected defect detection illumination optimization mode. In this particular example, it is desirable to detect a defect DFT that may be present on surface S1. In various embodiments, in order to best illuminate and find defects on surface S1, it may be desirable for the light from the main illumination channel LX to be of relatively high intensity and reflect back from the angled surface S1 to the camera 260. In various embodiments, one goal would be to be able to have good imaging of the surface texture for all of surface S1 without underexposing or overexposing any part of surface S1.

[0129] With respect to such an objective, the illumination from the illumination channel LZ has little or no effect on the illumination of surface S1, and since the illumination from the illumination channel LZ has little effect on defect detection on surface S1, it can be optimized with a low value (e.g., to conserve energy), or other possible values. In various embodiments, some portions of the illumination from the illumination channel LY can be useful for illuminating surface S1 such that the illumination channel LY can have settings optimized from some low values to intermediate values. As some specific numerical examples with respect to the figure of FIG. 15G, the settings of the illumination channels (e.g., corresponding to the illumination intensities in these examples) can correspond to an illumination intensity setting of 95 percent for illumination channel LX, 10 percent for illumination channel LY, and 0 percent for illumination channel LZ.

[0130] FIG. 15H and FIG. 15I are diagrams showing an embodiment in which an illumination optimization process is executed to determine illumination for illuminating the workpiece 20 for a selected focus point group illumination optimization mode. In the illustrated example, for the operation of the focus point group, it is desirable to determine the three-dimensional profile data of the workpiece 20 based on all the relationships of the surfaces S1, S2, and S3. In various embodiments, for such focus point group operation, it may be desirable to generate illumination that achieves relatively balanced illumination on all of the surfaces S1, S2, and S3 (e.g., such that none of the surfaces are saturated and none of the surfaces are very dark). Due to the fact that the surfaces S1 and S3 are angled, as a general principle, more illumination may be required from the illumination channels LX and LZ (e.g., corresponding images may be obtained using illumination that illuminates the surfaces S1 and S3 similarly compared to the surface S2). In various embodiments, it may be desirable for the determined illumination to enable the texture on the surfaces S1, S2, and S3 to be clearly visible in the corresponding images. According to such a principle, as one specific numerical example related to the diagram of FIG. 15I for the determined illumination, the illumination channels LX and LZ can have an illumination intensity setting of 75%, while the illumination channel LY can have an illumination intensity setting of 25%.

[0131] As part of the illumination optimization process, it will be understood that the settings of the illumination channels (e.g., corresponding to the illumination intensity settings in the examples of FIGS. 15A-15I) may be adjusted for the determination of the optimized illumination intensity. For example, when the illumination optimization process is executed in relation to a selected edge detection illumination optimization mode, the process may be configured such that the surface on one side of each edge (e.g., surface S2) is relatively bright without saturation, and the surfaces on the opposite side of the edge (e.g., surface S1 and surface S3) are mostly dark. In connection with these principles, the illumination optimization process can change the settings of the illumination channels LX, LY, and LZ within the range corresponding to these principles to determine the illumination that most strongly emphasizes the edges EA and EB, and images therefor can be captured with different illumination settings. In various embodiments, (e.g., in relation to edge enhancement,) quality metrics may be determined for each image and / or corresponding illumination setting to determine the illumination setting and corresponding image having the best metric. Similar changes in illumination settings and utilization of quality metrics can be performed with respect to the illumination optimization processes of FIGS. 15F-15G (i.e., in the case of the defect detection illumination optimization mode) and FIGS. 15H-15I (i.e., for the focus point group illumination optimization mode).

[0132] FIG. 16 is a flowchart of routine 1600 for determining illumination optimization for a workpiece (e.g., a new workpiece). In block 1610, a workpiece is determined for illumination optimization. For example, the user can have a new workpiece for which illumination optimization is determined. In various implementations, the user may place the workpiece within the field of view of the system (e.g., under camera 260 and lens 250 of FIG. 2), and the workpiece may be illuminated by the illumination configuration LC of the system. In various embodiments, the system is configured to recognize a new workpiece (e.g., which may not match any workpiece already in the workpiece database), and the system can propose settings for the new workpiece to the user. In various embodiments, the user can also select settings for an existing workpiece for the new workpiece (e.g., to add a new illumination optimization mode in a mode where illumination optimization was previously determined, or to add a new area for illumination optimization). In block 1620, the selected illumination optimization mode (e.g., edge detection, defect detection, focus point cloud) is determined. For example, the user can select an illumination optimization mode.

[0133] In block 1630, one or more elements or regions on the workpiece are determined for lighting optimization. As an example, the user can accept or indicate one or more elements or regions on the workpiece for lighting optimization decisions. Such elements or regions can be automatically identified, segmented, and / or proposed in various embodiments. As some examples, two or more edges may be shown for edge detection, or regions that may contain defects may be shown for surface defect detection. The user may be presented with the option to select all or some of these automatically identified regions. In various implementations, the elements and / or regions may be manually selected by the user. Some manual selection techniques may include options such as a polygon tool for clicking around features, a live wire tool for semi-automatically selecting edges, a finger swipe or pen drag on a touch screen device, or a simple mouse drag over a rectangular area. In various embodiments, in the case of edge detection lighting optimization mode, geometric elements can be automatically proposed or manually identified. In the case of defect detection lighting optimization mode or focus point group lighting optimization mode, the regions may be automatically segmented and proposed and / or manually identified.

[0134] In block 1640, lighting variables are determined. In various embodiments, the selection may be made (e.g., by a user) to optimize all possible lighting variables, or the selection may be made to optimize lighting subject to fewer variables (e.g., defined by other constraint options such as those selected in a layout, grouping of elements, and / or graphical user interface). By way of example, a group of lighting channels may be determined, and each group may be controlled as a group to share the same settings (e.g., RGB settings) such as part of a lighting optimization process. By making it possible to include lighting channels in groups and by adjusting each group of lighting channels as a single entity, the number of entities to be adjusted can be reduced, making the lighting optimization process faster and / or easier.

[0135] In block 1650, receive an indication that lighting optimization should proceed and / or proceed with the determination and provision of lighting. In various embodiments, such an indication may be provided by a user making a selection in a user interface (e.g., pressing a start button) and may correspond thereto. In various embodiments, a progress indicator (e.g., indicating to the user that the determination / learning of lighting optimization is in progress) may be provided to the user interface.

[0136] In block 1660, one or more determined candidates for illumination are displayed or otherwise provided. For example, in various embodiments, upon completion of illumination learning / determination, one or more candidates for illumination can be displayed (e.g., for user consideration) (e.g., using a metric target result indicator) or otherwise provided. In various embodiments, if the system determines that there is one clear best result based on the metric target, only that one result can be provided and / or displayed. In various embodiments, options can be provided for selecting a preferred result from two or more options (e.g., to enable user selection) or for accepting a single presented best result.

[0137] In various embodiments, the user can select to obtain the result and further manually adjust the lighting settings to check whether different results (which may be better in some aspects) can be achieved (e.g., the displayed metric target can be updated and displayed according to the corresponding manually made changes). In various embodiments, the metric target results can be determined and displayed for both each individual region and sets of regions (e.g., all of the regions). In various embodiments, the metric target can be predefined for each illumination optimization mode. For example, in the case of edge detection illumination optimization mode, the metric target can relate to how well the determined edge matches the profile of an ideal step edge.

[0138] In block 1670, related items are generated and / or saved. The following are some examples of some of the related items that can be generated and / or saved. One example is a trained or optimized lighting model that includes all of the layout / group / channel and / or other settings (e.g., described as including what has been created or selected by the user in the graphical user interface in some cases). Another example can include workpiece features (e.g., derived from an image or point cloud depending on the processing and configuration) for automatically identifying similar workpieces in the future. Another example can include information regarding the position and orientation of the workpiece. Another example can include an image of the workpiece with optimized lighting (e.g., displayed along with the date of the image, etc.). Another example can include the lighting optimization mode selected for optimization. Another example can include a metric or score related to the quality of the lighting optimization for the selected lighting optimization mode. Another example can include an area on the workpiece where lighting optimization has been achieved for the position and orientation of the workpiece (e.g., such that the same area can be automatically found on a new workpiece placed at different positions and orientations within the field of view of the system, such as within the field of view of camera 260). Another example can include an image of the workpiece with neutral lighting (e.g., utilized as a workpiece identification image for subsequent user viewing).

[0139] In decision block 1680, a decision is made as to whether additional lighting optimization is to be determined for the same workpiece in the same lighting optimization mode. For example, the user can specify one or more different elements or areas on the workpiece for additional lighting optimization. If additional lighting optimization is to be performed, the routine returns to block 1630. In decision block 1690, a decision is made as to whether additional lighting optimization is to be performed for the same workpiece and different lighting optimization modes. If additional lighting optimization is to be performed, the routine returns to block 1620; otherwise, the routine ends.

[0140] As an example, in various embodiments, a single lighting optimization may not be able to optimally illuminate all of the surfaces / edges of the workpiece in the desired manner for the selected lighting optimization mode. In such a situation, for a single lighting optimization mode selected for lighting optimization, two or more lighting optimizations can be determined for the same workpiece (e.g., in accordance with an instruction from the user). For example, in an edge detection lighting optimization mode, there may be a first determined lighting for horizontal edges on the workpiece and a second determined lighting for vertical edges.

[0141] FIG. 17 is a flowchart of a routine 1700 that provides lighting optimization for a workpiece (e.g., where the workpiece corresponds to a stored workpiece in a database such as a workpiece for which lighting optimization has already been determined according to routine 1600 of FIG. 16). In block 1710, the workpiece is identified. For example, in various embodiments, the user can place the workpiece within the field of view of the system (e.g., under camera 260), and the system can automatically sense and image the workpiece. In various embodiments, such sensing and / or imaging results can be used to automatically determine which workpiece the current workpiece corresponds to (e.g., in an existing setup workpiece database). In various embodiments, if there is some uncertainty, a neutral lighting image of the most matching workpiece in the database can be presented (e.g., to the user) for the user to select a matching workpiece from the database (e.g., to match the current workpiece to a workpiece in the database).

[0142] In block 1720, an illumination optimization mode and model for the workpiece are determined. The model is an optimization framework that includes adjustable parameters contributing to the control of the illumination elements to be optimized. During model optimization, the parameters are changed by an optimization routine based on conventional or machine learning / artificial intelligence methods to achieve illumination that results in workpiece image characteristics closest to the optimization goals specific to the default mode. In various embodiments, for the current workpiece corresponding to the stored workpiece in the database, if there is only one illumination optimization mode and model for the stored workpiece, the stored illumination optimization mode and model can be automatically loaded / used.

[0143] If multiple illumination optimization modes and models have been learned for the stored workpiece, options may be provided. For example, the user may be provided with an option to select one or more of the multiple illumination optimization modes and models (e.g., for continuously imaging or for acquiring an image for each illumination optimization mode and model and having the processing set to automatically process the acquired images accordingly). In various embodiments, if the stored workpiece has two different illumination optimization modes and models, for example, for edge detection and defect detection, the illumination for edge detection may be used to acquire a first image (e.g., to be sent to an edge detection process such as a measurement algorithm for edge determination and dimension / distance measurement, or to be used in another manner), and the illumination for defect detection may be provided to acquire a second image (e.g., to be sent to a defect detection process or provided in another way).

[0144] In block 1730, the position and orientation of the workpiece are specified. For example, the position and orientation of the workpiece are specified (e.g., automatically detected) and can be compared with the corresponding stored position and orientation of the workpiece in a database (e.g., corresponding to the parameters used during model learning). In decision block 1740, a decision is made as to whether an adjustment is necessary in relation to the position and orientation of the workpiece. For example, if it is determined that the current orientation and position of the workpiece match the stored position and orientation of the workpiece in the database, an adjustment may not be necessary, and as will be described in more detail below, the routine proceeds to block 1760.

[0145] If it is determined in decision block 1740 that an adjustment is necessary, the routine proceeds to block 1750 where the adjustment is determined and executed. For example, if the position and orientation of the workpiece are different from the stored position and orientation of the workpiece in the database, a specific adjustment can be determined and executed. For example, the database and / or the process can have a physical model for each of the illumination and its elements with respect to the relative position (e.g., within the display area 360 of the light source display as shown in FIGS. 3A and 3C). As an example, if the illumination element is cylindrically symmetric with respect to the optical axis (e.g., of the camera 260 in FIG. 2), the illumination model can be automatically rotated to fit the current orientation of the workpiece. This may be sufficient in some cases, such as when it is determined that the workpiece has not been displaced from its original central (e.g., corresponding to the center of mass or geometric center, etc.) position. If it is detected that the workpiece has also been translated from the position of the workpiece stored in the database, a recommendation can be provided to the user to make an adjustment (e.g., start from the rotation model and then perform transfer learning on the illumination model to adapt the illumination by a series of learning steps to account for the workpiece displacement, etc.). In some embodiments, re-optimization of the illumination can be determined for different positions and orientations of the current workpiece.

[0146] When adjustment is performed in block 1750 or when it is determined in block 1740 that adjustment is not necessary, the routine proceeds to block 1760 and illumination is provided to the workpiece (e.g., to illuminate the workpiece for a corresponding inspection operation). In block 1770, one or more images of the workpiece are acquired (i.e., illuminated by the illumination). In block 1780, a measurement operation is performed on the workpiece (e.g., corresponding to a selected illumination optimization mode including edge detection, defect detection, or measurement of a focus point group or inspection operation). In various embodiments, such inspection operations may be performed as part of a corresponding process (e.g., performed by one of parts 144, 145, or 146 of FIG. 2).

[0147] FIG. 18 is a diagram of a representation 1800 of specific options that may be provided in the system in connection with illumination optimization. In various embodiments, the options as described below may correspond to selectable options such as those provided in a user interface, provided to the user in another way, and / or automatically performed by the system. As shown in FIG. 18, the various options include an illumination channel option 1810, a region of interest option 1820, an optimization method option 1830, a model learning option 1840, a learning setting option 1850, and a result / recommendation option 1860. In connection with the illumination channel option 1810, in various embodiments, the user may be provided with the option to select a particular desired illumination channel of the illumination configuration. In various embodiments, the user may also have the option to leave the illumination settings in their current state (e.g., as part of the default settings). In some such cases, all of the illumination channels are utilized for the determination of illumination optimization.

[0148] In connection with the region of interest option 1820, in various embodiments, options corresponding to specific regions of interest on the workpiece are provided (e.g., for selection by the user). In various embodiments, an option may also be provided not to select any specific region of the workpiece (e.g., not to select any specific region within the image of the workpiece) (e.g., for selection by the user). If no specific region is selected, the illumination can be optimized for the entire workpiece or a portion of the workpiece (e.g., included in the image). With respect to the optimization method option 1830, options can be provided for selecting from among a plurality of optimization methods. For example, several different optimization methods can utilize different algorithms, different learning, etc. to determine optimization such as conventional optimization methods or machine learning / artificial intelligence methods.

[0149] Regarding the model learning option 1840, an option for selecting a pre-trained model (e.g., as stored in a database) may be provided. In various embodiments, an option can be provided for modifying (e.g., fine-tuning) an existing model (e.g., stored in a database) in an attempt to further optimize the illumination of the workpiece. Regarding the learning configuration option 1850, when an option for modification (e.g., fine-tuning of an existing model) is selected, an option can be provided for selecting specific parameters (e.g., the number of training epochs corresponding to learning time, etc.). In various embodiments, an option may also be provided to leave the current configuration as the default setting without changing such parameters.

[0150] In connection with result / recommendation option 1860, in various embodiments, a quality metric may be provided (e.g., displayed in a user interface for review by a user). If the quality metric is determined to be lower than desired (e.g., lower than a specified threshold), certain recommendations may be provided (e.g., displayed in a user interface for review by a user). For example, a request may be made to the user to select an area of interest where the user desires to achieve better lighting quality. As another example, if a pre-trained model is used, the recommendation can indicate that fine-tuning of the model may achieve better results. As another example, the recommendation can indicate that longer learning or optimization of the model may improve the results if the model is being fine-tuned.

[0151] FIG. 19 is a flowchart showing a routine 1900 for performing a lighting optimization process. At block 1910, an option is provided for selecting a lighting optimization mode that is at least one of an edge detection lighting optimization mode, a defect detection lighting optimization mode, or a focus point group optimization mode. In various embodiments, there may be a most commonly used or user-selected default mode that is automatically selected (e.g., unless the user or system selects or indicates otherwise). At block 1920, the selection of the lighting optimization mode is received. At block 1930, a lighting optimization process is performed based on the selected lighting optimization mode, and the lighting optimization process determines lighting for illuminating a workpiece, and the determined lighting includes settings for each of the lighting channels of the lighting configuration.

[0152] Regarding edge detection processing, generally, it may be desirable to provide illumination such that the resulting image and the edges of the object have high contrast (e.g., having low luminance values on one side and higher luminance values on the other side). Generally, such a transition preferably occurs over a minimum number of pixels (e.g., similar to a step function) such that the gradient of the scan line across the edge has a large maximum value (e.g., as shown by data 1026 in FIG. 10). In various embodiments, such conditions may be determined and / or further refined by an optimization goal (e.g., utilized in an illumination optimization process), and the optimization goal (e.g., utilized in an illumination optimization process) penalizes having saturated pixels (i.e., full luminance pixels) in the region around the edge so as to have more Gaussian-shaped gradient curves since the luminance on one side of the edge is unnaturally clipped at the maximum luminance.

[0153] Regarding defect detection processing, in some embodiments, the requirements for determining optimized illumination may be more changeable than the relatively more defined requirements for edge detection processing. In some cases, the optimized illumination for defect detection may be at least partially a function of the type of defect being detected. As an example, in the case of a flat surface having raised or sunken defects (e.g., scratches, dents, bumps), the optimized illumination (e.g., the best illumination) may be from the side (e.g., at a particular angle with respect to the optical axis of the system and may be referred to as dark field illumination in some embodiments). In contrast, for defects that may not have such characteristics, such as contaminants on a surface having a minimal thickness and that may not be well illuminated by dark field illumination, such defects may require more bright field illumination that enables viewing the surface texture, luminance, and / or color, etc. In some embodiments, the illumination optimization process can utilize techniques for distinguishing such types of defects and / or can receive user input regarding the type of defect.

[0154] In various embodiments, it may be desirable to receive user input regarding information about the workpiece surface and / or the defect characteristics of the defect to be detected as part of an illumination optimization process to be performed in connection with a defect detection process. In some embodiments, the defect detection process may learn based on the user providing an example indicating which pixels in an image of the workpiece represent a defect. For example, the user may indicate whether the defect has a certain characteristic (e.g., similar to a bump, or similar to a scratch or pit) with respect to the surface of the workpiece, or alternatively, whether the defect is likely to be related to the surface texture or other characteristics. As some other examples, the user may indicate whether the workpiece surface is highly reflective (e.g., polished metal) or lowly reflective (e.g., plastic, coarsely machined metal, etc.), and whether the workpiece has a flat surface or a curved surface. In various embodiments, the system may learn to determine optimized illumination for defect detection by placing the workpiece within a field of view in which an exemplary defect is shown and / or within a field of view in which pixels corresponding to the defect are shown. In such embodiments, the defect detection process may include changing the settings of the illumination channels to determine which settings produce the best contrast and visibility for the specified defect.

[0155] In various embodiments, an indication of which pixels correspond to a defect may be provided in various ways (e.g., using an automatic segmentation tool, filling in the defect pixels, placing a bounding box around the defect, drawing a contour around the defect, etc.). In some embodiments, the defect detection process may be able to provide feedback to the illumination optimization process. In some embodiments, such feedback may include a defect detection process performed using the illumination proposed by the illumination optimization process, and thus, be able to provide feedback indicating the current ability of the defect detection process to accurately detect the defect using the proposed illumination.

[0156] In various embodiments, a user can select an illumination optimization mode, and an illumination optimization process is executed based on the selected illumination optimization mode. When an illumination optimization mode is selected, the illumination optimization process can automatically determine optimized illumination for a workpiece within the field of view. In various embodiments, the illumination optimization process may proceed from a random or preset configuration, or from a pre-trained model. In some embodiments, a pre-trained model can be automatically proposed by the system based on some similarity of the workpiece to existing workpieces stored in a database, or according to a user's selection. In various embodiments, the illumination optimization process can automatically determine and / or propose a region of interest for illumination optimization (e.g., the illumination optimization process can be configured to automatically find edges and / or automatically find surfaces). In such embodiments, the system can be configured to allow the user to select elements of interest (e.g., edges, surfaces) that are automatically determined, or to accept all of the determined elements. In various embodiments, the user can potentially enhance the determined elements by accepting one or more of the elements and then manually entering additional elements or regions of interest.

[0157] In various embodiments, the system can be configured to allow the user to input a workpiece region (e.g., an edge, a surface, etc.) to which the illumination optimization process can be applied. In some embodiments, the illumination optimization process may attempt to determine optimized illumination for a complete image or a complete portion including the workpiece, at least in some instances, with respect to the selected illumination optimization mode. In various embodiments, the system can be configured to recognize a workpiece (e.g., based on other sensing such as imaging and / or 3D sensing), and to automatically load an optimized illumination model / setting for the corresponding workpiece stored in a database whenever the workpiece is placed within the system's field of view (e.g., under camera 260).

[0158] In various embodiments, the illumination optimization process may be configured to detect the relative rotational orientation of the workpiece and automatically rotate / align an optimized illumination model / setting based on knowledge of the placement of illumination channels in the illumination configuration. In various embodiments, the illumination optimization process and / or system may automatically determine when the position, orientation or geometric shape of the workpiece may have some change relative to when the original illumination optimization process was performed on the workpiece stored in the database, and then be configured to propose that the optimized illumination model be automatically adjusted or re-optimized. In various embodiments, transfer learning may be utilized on the optimized illumination model to quickly adjust the illumination for smaller changes in the workpiece position, orientation, or geometric shape of a new identical or similar workpiece. In various embodiments, the illumination optimization process and / or system may show the user an optimized illumination as applied to the workpiece along with an indicated optimization score, and the user may further be able to manually adjust the illumination settings, with an updated optimization score being provided and displayed to the user during the adjustment.

[0159] In various embodiments, the system may be configured to perform an illumination optimization process for each of a plurality of illumination optimization modes selected by a user and / or for a plurality of regions selected for the illumination optimization process, in each case involving continuous imaging of the workpiece illuminated by the determined illumination. In various embodiments, each image acquired for different illumination optimization modes may be automatically sent to an appropriate post-processing part (e.g., an image having illumination optimized for edge detection may be provided to an edge detection process, an image acquired using illumination optimized for defect detection may be provided to a defect detection process, and / or an image acquired using illumination optimized for a focus point cloud may be provided for a focus point cloud process). In various forms, the system may be configured such that the user can control independent lighting variables for the illumination optimization process. For example, the user may identify or select a particular group of lighting channels and may be limited to adjusting only the selected RGB lighting options.

[0160] In various embodiments, when the user selects a defect detection illumination optimization mode, the user can provide at least one example of a workpiece having a defect. The user can indicate the region of the defect (e.g., by using a bounding box, pixel painting, indication of the defect boundary, use of a tool that attempts to automatically segment the defect region after the user selects points within the defect region, etc.). In this way, the illumination optimization process can receive an indication of which pixels in the image constitute a defect, and thus the illumination optimization process can optimize the illumination to maximize the contrast of the region containing the defect on a particular workpiece surface (e.g., for both the current workpiece and future similar workpieces). In various embodiments, the user can provide at least some information regarding the characteristics of the defect to be detected and / or the characteristics of the workpiece surface.

[0161] In various embodiments, the illumination optimization process may be configured to create surface illumination having either a maximized texture of the surface of interest or darkfield illumination, depending on information supplied by the user regarding the characteristics of the defect to be detected. In various embodiments, the illumination optimization process may be configured to communicate with a defect detection process for which the optimized illumination is being determined, and the defect detection process may be configured to provide feedback regarding the proposed illumination. For example, the defect detection process may be able to receive a workpiece image with the proposed illumination, utilize the illumination image proposed to attempt to detect a defect, and provide feedback (e.g., data) indicating the accuracy with which the defect detection process was able to find a defect (e.g., corresponding to a known defect area, etc.).

[0162] In various embodiments, a set of images utilized by the illumination optimization process may be acquired, in which case each illumination channel, group of illumination channels, etc. to be utilized as part of the illumination optimization process is on in at least one of the images. In various embodiments, the system may include a light source configuration configured to determine a setting of an illumination channel, including where one or more illumination channels are physically moved when electronically controlled (e.g., with respect to the height or directionality of the illumination channel), and the illumination optimization process may include determining the height and / or directionality of the illumination channel. In various embodiments, the illumination optimization process may be configured to utilize negative RGB weights including subtracting one image from another image.

[0163] In various embodiments, when an edge detection illumination optimization mode is selected, the illumination optimization process may be configured to determine illumination that results in a high contrast step edge transition (e.g., to support an accurate and repeatable determination of an edge position, such as for determining a measurement (such as a distance between edges) regarding the edge position). In various embodiments, when a focus point group illumination optimization mode is selected, the illumination optimization process may be configured to determine illumination that results in a high texture on all surfaces of the object while minimizing saturation.

[0164] The following describes various exemplary embodiments of the present disclosure together with various features and elements annotated with reference characters (i.e., reference numbers and reference characters) found in the figures described herein. It should be understood that the reference characters are added to illustrate the exemplary embodiments and the features and elements are not limited to the specific embodiments shown in the figures.

[0165] According to one aspect, a system 100 is provided that includes a lens 250, a camera 260, an illumination configuration LC, one or more processors 125, and a memory 140. The lens 250 (e.g., an objective lens) is configured to receive image light 255 originating from the workpiece 20, and the lens 250 is configured to transmit the image light 255 along an image optical path OPATH and has an optical axis OA. The camera 260 is configured to receive the image light 255 transmitted along the image optical path OPATH and provide an image of the workpiece 20. The illumination configuration LC includes an illumination channel LCH (e.g., in some embodiments, including at least 5 illumination channels, or at least 10 illumination channels, or at least 20 illumination channels, etc.) configured to illuminate the workpiece 20 to generate the image light 255.

[0166] The memory 140 is coupled to the one or more processors 125 and stores program instructions that, when executed by the one or more processors 125, cause the one or more processors 125 to at least: provide a representation of a group of illumination channels in a display area, determine that a group of illumination channels has been selected, the selected group of illumination channels includes a plurality of illumination channels, display a current illumination setting for the selected group of illumination channels, and an adjustment to the illumination setting for the group of illumination channels is applied to all of the illumination channels within the group.

[0167] As an example of such a feature, in the diagrams of FIGS. 3A - 3C, the display area 320 includes a "group list", column 320A includes label indicators for the set of currently displayed groups, and each label indicator (e.g., "Group 1", "Group 2", etc.) is a representation of a corresponding group of lighting channels. As another example, the light source display within the display area 360 may also include a representation of one or more groups of lighting channels. For the selection of a group of lighting channels, in various embodiments, a user can click or otherwise select the displayed representation of the group of lighting channels (e.g., the user may click or otherwise select the "Group 3" display within the display area 320), and the system determines that the corresponding group of lighting channels has been selected. Then, the settings for the selected group of lighting channels can be displayed (e.g., the system can display the settings for the selected "Group 3" both in column 351D showing the corresponding values for settings regarding different light colors and in the positions of the sliders within the selection areas 351A, 351B, and 351C within the main settings of the display area 350, and regarding those, as described above, similar settings can be displayed within the display area 370). The sliders within the selection areas 351A, 351B, and 351C are provided by the system and allow the lighting settings of the selected group to be adjusted, and the adjustment to the settings for the group of lighting channels can be characterized as an adjustment factor that is applied to all of the lighting channels within the group.

[0168] In various embodiments, when program instructions are executed by one or more processors, the one or more processors may be further caused to execute steps such as those described below (and / or such steps may be executed in other ways). In various embodiments, the above and / or below steps may be executed, additionally or alternatively, as part of a method, and / or the system may be configured to execute the steps (e.g., characterized as being executed by one or more processors or otherwise being executed).

[0169] In various embodiments, an adjustment element (e.g., a slider for selection regions 351A, 351B, 351C) may be provided that is configured to enable adjustment of the lighting settings of a selected group. In various embodiments, a visual representation of the lighting configuration, including the lighting channels of the lighting configuration (e.g., provided to the display region 360), may be displayed. The lighting channels of the currently selected group may be shown as selected in the visual representation of the lighting configuration (e.g., according to the selection indicator SI for the lighting channel LCH). Selection of individual lighting channels in the visual representation of the lighting configuration may be enabled, and it may be enabled to add the selected lighting channel to the group of currently selected lighting channels or to delete the selected lighting channel from the group of currently selected lighting channels (e.g., the selected lighting channel LCH within the display region 360 may be added to or deleted from the currently selected group).

[0170] In various embodiments, a representation of a lighting layout (e.g., the display area 310 includes a "layout list", column 321A includes a label indicator for the currently displayed set of lighting layouts, and each label indicator (e.g., "layout 1", "layout 2", etc.) is a representation of the corresponding lighting layout) may be provided, and at least some of the lighting layouts each comprise a plurality of groups of lighting channels (e.g., the currently selected "layout 1" is shown to include 12 groups of lighting channels). In various embodiments, a determination may be made that a lighting layout has been selected (e.g., in various embodiments, the user can click or otherwise select the display representation of the lighting layout, such as clicking on the "layout 1" display within the display area 310, and the system determines that the corresponding lighting layout has been selected). And a representation of the groups of lighting channels included in the selected lighting layout may be displayed (e.g., a representation of the groups of lighting channels included in the selected "layout 1" is displayed in column 320A of the display area 320). The selected groups of lighting channels of the lighting layout may be enabled to be deleted from the lighting layout (e.g., the selection area 328 provides an option to delete the selected group). In various embodiments, one or more parameters of the selected lighting layout can be displayed (e.g., in the display area 330), and adjustment of one or more parameters of the selected layout (e.g., if one such parameter can be the number of segments, the selection area 331 within column 330A indicates that the current number of segments is set to 5 segments and includes up and down arrows that can be utilized to increase or decrease the number of segments in this case) can be enabled.

[0171] In various embodiments, it is possible to save lighting settings for a lighting layout and to be able to call up the saved lighting settings for the lighting layout. In various embodiments, the lighting layout and lighting settings can be stored in different files. As one specific example, for a layout stored in a file named "RingLight", the lighting settings can be stored in files named "RingLight_scheme001", "RingLight_scheme002", etc. In various embodiments, each of the files (e.g., named "RingLight_scheme001", "RingLight_scheme002", etc.) can include a set of lighting settings for the corresponding layout. From this example, it will be understood that there may be multiple sets of lighting settings for a lighting layout, and in some embodiments, each set of lighting settings can be stored in a different file.

[0172] In various embodiments, adjustment elements (e.g., "zoom", "rotate", or "direct" control of respective selection regions 362, 363, or 364) are provided that enable the representation of a group of lighting channels to be rotated, zoomed, shrunk, or oriented in at least one of these ways. The adjustment can correspond to maintaining the same shape of the group of lighting channels, but when the shape is rotated, zoomed, shrunk, or oriented, at least some different lighting channels are included in the shape.

[0173] In various embodiments, the lighting settings are configured to allow adjustments that include lighting settings of one or more positive or negative colors (e.g., in the example shown in display regions 350 and 370, the red light setting is shown to have a negative weight corresponding to a value of -0.50). In various embodiments, the implementation of one or more negative color lighting settings includes obtaining two images, a first image having one or more positive color lighting settings (e.g., using a green light setting of 0.30 and a blue light setting of 0.15), and a second image having an absolute value of one or more negative color lighting settings (e.g., using an absolute value of the red light setting or a value of 0.50), and subtracting the second image from the first image. In various embodiments, an image corresponding to the pixel values resulting from the subtraction of the second image from the first image may be displayed, and pixels in the displayed image corresponding to negative values are displayed with a representation indicating the correspondence to the negative values (e.g., in the example of FIG. 6, four negative weight indicator regions NWIA are displayed, each corresponding to a region where at least one component of the displayed color is negative).

[0174] In various embodiments, an image of the workpiece acquired by the camera with the current lighting settings is displayed, and the image is displayed in a workpiece display region (e.g., display region 380). Representations (e.g., edge detection tool EDT or defect detection tool DDT) may be provided within the workpiece display region corresponding to at least one of an edge detection operation or a defect detection operation on the corresponding region of the workpiece.

[0175] According to another aspect, a method for operating the system is provided. The method includes providing a representation of a group of lighting channels in a display region, determining that a group of lighting channels has been selected, the selected group of lighting channels including a plurality of lighting channels, and displaying a current lighting setting for the selected group of lighting channels, wherein an adjustment to the lighting setting for the group of lighting channels is applied to all of the lighting channels within the group.

[0176] According to another aspect, the system provides a display of a group of lighting channels in a display area, determines that a group of lighting channels has been selected, the selected group of lighting channels includes a plurality of lighting channels, displays a current lighting setting for the selected group of lighting channels, and an adjustment to the lighting setting for the group of lighting channels is configured to be applied to all of the lighting channels within the group.

[0177] According to another aspect, a system 100 is provided that includes a lens 250, a camera 260, a lighting configuration LC, one or more processors 125, and a memory 140. The memory 140 is coupled to the one or more processors 125 and stores program instructions that, when executed by the one or more processors 125, cause the one or more processors 125 to at least perform the following: provide an option for selecting a lighting optimization mode that is at least one of an edge detection lighting optimization mode, a defect detection lighting optimization mode, or a focus point group lighting optimization mode; receive a selection of the lighting optimization mode; perform a lighting optimization process based on the selected lighting optimization mode, the lighting optimization process determining lighting for illuminating a workpiece, the determined lighting including settings for lighting channels of the lighting configuration. In various embodiments, the option can be for selecting a lighting optimization mode from a set of lighting optimization modes (e.g., a set that includes at least two of an edge detection lighting optimization mode, a defect detection lighting optimization mode, and / or a focus point group lighting optimization mode, where different modes correspond to different lighting determinations). In various embodiments, the option for selecting a lighting optimization mode can be provided in a user interface (e.g., in the display area 390 of FIG. 3C).

[0178] In various embodiments, the edge detection illumination optimization mode is for optimizing illumination for detecting one or more edges on the surface of the workpiece (e.g., may include determining illumination configured to generate a high contrast step edge transition to support an accurate and repeatable determination of the edge position, contrast curve analysis may be utilized to determine the edge position, and the distance between edges may be determined). In various embodiments, the defect detection illumination optimization mode is for optimizing illumination for detecting one or more defects on the surface of the workpiece (e.g., includes determining illumination configured to generate either surface illumination or dark field illumination that maximizes the texture of the surface of interest depending on, for example, the characteristics of the defect to be detected), and in various embodiments, the focus point group illumination optimization mode is for optimizing illumination for determining three-dimensional profile data of the surface of the workpiece (e.g., a step of determining illumination configured to generate balanced illumination on different regions / surfaces of the workpiece, where at least one image stack may be acquired as including a plurality of images of the workpiece, and each image of the image stack corresponds to a different focus position along the optical axis).

[0179] In various embodiments, when the program instructions are executed by one or more processors, the one or more processors may be further caused to execute steps such as those described below (and / or such steps may be executed in other ways). In various embodiments, the above and / or below steps (e.g., characterized as being executed by one or more processors or other means) may alternatively or additionally be executed as part of a method, and / or the system may be configured to execute the steps.

[0180] In various embodiments, one or more elements or regions on the workpiece can be determined for lighting optimization. In various embodiments, lighting variables may be determined for use in the lighting optimization process. In various embodiments, the result of the lighting optimization process may be displayed as including one or more determined candidates for lighting to illuminate the workpiece.

[0181] In various embodiments, for the same workpiece and the same lighting optimization mode, additional lighting optimization should be performed for one or more different elements or regions on the workpiece, and a decision may be made that the lighting optimization process should be correspondingly performed for one or more different elements or regions on the workpiece. In various embodiments, a decision may be made that additional lighting optimization should be performed for the same workpiece and different lighting optimization modes, and the lighting optimization process may be correspondingly performed based on different lighting optimization modes.

[0182] In various embodiments, a lighting optimization model for the workpiece can be saved that includes settings (e.g., a set of settings) for the lighting channels of the lighting configuration corresponding to the determined lighting. In various embodiments, a second workpiece (e.g., disposed within the view of the system) can be compared to the workpiece to determine its similarity, and a decision may be made (e.g., based on the similarity of the second workpiece) that the saved lighting optimization model is to be used for the lighting of the second workpiece, and the saved lighting optimization model can be called to provide lighting for the second workpiece. In various embodiments, the position and orientation of the second workpiece can be determined, and a decision may be made (e.g., based on the position and / or orientation of the second workpiece) that an adjustment to the lighting is required with respect to the position and orientation of the second workpiece, and corresponding adjustments may be provided to the lighting based on the position and orientation of the second workpiece.

[0183] In various embodiments, lighting can be provided to a second workpiece, and one or more images of the second workpiece illuminated by the lighting device can be obtained. In various embodiments, one or more inspection operations can be performed on the second workpiece, and the one or more inspection operations therefor correspond to an illumination optimization mode selected for the illumination optimization process. In various embodiments (for example, when an edge detection illumination optimization mode is selected), the one or more inspection operations can include edge detection utilized to determine the position of one or more edges on the second workpiece (for example, the one or more inspection operations can further include determining the distance between two edges on the second workpiece). In various embodiments (for example, when a defect detection illumination optimization mode is selected), the one or more inspection operations include defect detection utilized to detect defects on the second workpiece. In various embodiments (for example, when a focus point group illumination optimization mode is selected), the one or more inspection operations include a focus point group utilized to determine three-dimensional profile data of the second workpiece.

[0184] In various embodiments, the user can control independent lighting variables utilized as part of the illumination optimization process. For example, it may be made possible for the user to determine a group of lighting channels where each group is controlled / regulated as a single entity, determine the range within which color adjustment can be performed, limit that color adjustment is only performed for the selected RGB color, etc.

[0185] In various embodiments, when a defect detection illumination optimization mode is selected, information regarding a workpiece having a defect in the area indicated by the user may be received from the user, and the illumination optimization process may utilize that information to provide illumination that maximizes the contrast of the area. For example, the user may provide at least one example of a workpiece having a defect, and the user may indicate the area of the defect (e.g., by using a bounding box, pixel painting, indication of the defect boundary, or use of a tool that attempts to automatically segment the defect area when the user clicks within the defect area), so that the illumination optimization process can know which pixels in the image constitute the defect and can optimize the illumination to maximize the contrast of a specific area on the surface of this workpiece and future similar workpieces.

[0186] In various embodiments, when a defect detection illumination optimization mode is selected, information regarding at least one characteristic of either the defect or the surface of the workpiece that is utilized by the illumination optimization process may be received from the user. In various embodiments, the characteristic is utilized to determine whether the optimized illumination corresponds to surface illumination or dark field illumination having a maximized texture of at least one surface of interest. In various embodiments, the illumination optimization may be a function of the type of defect to be detected. For example, in the case of a flat surface having raised or recessed defects (e.g., scratches, dents, protrusions, etc.), the optimized illumination may correspond to dark field illumination (e.g., corresponding to illumination from the side at an angle of 45 degrees to 90 degrees from the optical axis OA). In contrast, defects that do not have such characteristics (e.g., contaminants or other defects on a surface having a minimum thickness) may not be illuminated well by dark field illumination and may require surface illumination having a maximized texture (e.g., bright field illumination that enables visualization of surface texture, luminance, or color, where such bright field illumination may be at an angle between 0 degrees and 45 degrees from the optical axis OA). In various embodiments, the illumination optimization process is configured to communicate with a defect detection process that evaluates the proposed illumination and provides feedback regarding the accuracy with which the defect detection process can detect a known defect area illuminated by the proposed illumination.

[0187] In various embodiments, a set of images in which each lighting channel to be optimized, which is utilized in the lighting optimization process, is on can be collected in at least one of the images. In various embodiments, one or more of the lighting channels of the lighting configuration may be movable lighting channels, and each movable lighting channel is configured to be controllable to move relative to other lighting channels of the lighting configuration to adjust at least one of the position or direction of the lighting provided by the movable lighting channel. The lighting optimization process includes optimizing at least one of the position or direction of the movable lighting channel. In various embodiments, the lighting optimization process is configured to utilize negative color lighting channels through a process that includes subtracting one image from another.

[0188] According to another aspect, a method for operating a system for performing a lighting optimization process is provided. The method includes: providing an option for selecting a lighting optimization mode that is at least one of an edge detection lighting optimization mode, a defect detection lighting optimization mode, or a focus point group lighting optimization mode; receiving a selection of the lighting optimization mode; and performing a lighting optimization process based on the selected lighting optimization mode, the lighting optimization process determining lighting for illuminating a workpiece, the determined lighting including settings for lighting channels of a lighting configuration.

[0189] According to another aspect, the system provides an option for selecting a lighting optimization mode that is at least one of an edge detection lighting optimization mode, a defect detection lighting optimization mode, or a focus point group lighting optimization mode, receives a selection of the lighting optimization mode, and performs a lighting optimization process based on the selected lighting optimization mode, the lighting optimization process determining lighting for illuminating a workpiece, the determined lighting including settings for lighting channels of a lighting configuration.

[0190] Preferred embodiments of the present disclosure have been illustrated and described, but numerous variations in the illustrated and described configurations and sequences of operations of the features will be apparent to those skilled in the art based on the present disclosure. Various alternative forms may be used to implement the principles disclosed herein. In addition, the various implementations described above can be combined to provide further implementations. All U.S. patents and U.S. patent applications referred to herein are hereby incorporated by reference in their entirety. Aspects of the embodiments can be modified, as necessary, to employ concepts from various patents and applications to provide further embodiments.

[0191] In light of the above detailed description, these and other changes can be made to the embodiments. Generally, in the following claims, the terms used should not be construed as limiting the claims to the specific embodiments disclosed herein and in the claims, but rather the claims should be construed to include all possible embodiments along with the full scope of equivalents to which such claims are entitled.

Claims

1. A system comprising: a lens configured to receive image light generated from a workpiece, the lens configured to transmit the image light along an image optical path and having an optical axis; a camera configured to receive the image light transmitted along the image optical path and provide an image of the workpiece; an illumination configuration including an illumination channel configured to illuminate the workpiece to generate the image light; one or more processors; coupled to the one or more processors, and when executed by the one or more processors, causing the one or more processors to at least: provide an option for selecting at least one illumination optimization mode from among an edge detection illumination optimization mode, a defect detection illumination optimization mode, or a focus point group illumination optimization mode; receive a selection of the illumination optimization mode; perform an illumination optimization process based on the selected illumination optimization mode, the illumination optimization process determining illumination for illuminating the workpiece including settings for the illumination channel of the illumination configuration; a memory storing program instructions for causing the above to be executed; A system having the above.

2. When executed by the one or more processors, the program instructions further cause the one or more processors to determine one or more elements or regions on the workpiece for the illumination optimization process. The system according to claim 1.

3. When executed by the one or more processors, the program instructions further cause the one or more processors to determine illumination variables to be utilized in the illumination optimization process. The system according to claim 1.

4. When executed by the one or more processors, the program instructions further cause the one or more processors to display the result of the illumination optimization process including one or more illumination candidates determined for illuminating the workpiece. The system according to claim 1.

5. When executed by the one or more processors, the program instructions cause the one or more processors to: perform additional illumination optimization for the same workpiece and the same illumination optimization mode but for different one or more elements or regions on the workpiece, or perform the illumination optimization process for different one or more elements or regions on the workpiece; or Cause additional lighting optimization to be performed on the same workpiece and different lighting optimization modes, and determine at least one of additional lighting optimization for performing lighting optimization processing based on the different lighting optimization modes. The system according to claim 1. **Claim 6** When the program instructions are executed by the one or more processors, cause the one or more processors to save a lighting optimization model for the workpiece, including the settings for the lighting channels of the lighting configuration corresponding to the determined lighting. The system according to claim 1. **Claim 7** When the program instructions are executed by the one or more processors, further cause the one or more processors to compare a second workpiece with the workpiece to determine the similarity of the second workpiece, determine that the saved lighting optimization model is used for the lighting of the second workpiece, cause the saved lighting optimization model to be called to provide lighting for the second workpiece. The system according to claim 6. **Claim 8** When the program instructions are executed by the one or more processors, further cause the one or more processors to determine the position and orientation of the second workpiece, determine that lighting adjustment is required for the position and orientation of the second workpiece, cause lighting adjustment to be provided based on the position and orientation of the second workpiece. The system according to claim 7. **Claim 9** When the program instructions are executed by the one or more processors, cause the one or more processors to provide the lighting for the second workpiece and acquire one or more images of the second workpiece when illuminated by the lighting. The system according to claim 7. **Claim 10** When the program instructions are executed by the one or more processors, cause the one or more processors to further perform one or more inspection operations on the second workpiece, and the one or more inspection operations correspond to the lighting optimization mode selected for the lighting optimization processing. The system according to claim 9. **Claim 11** The one or more inspection operations are edge detection used to determine the position of one or more edges on the second workpiece, and determining the distance between two edges on the second workpiece, and include. The system according to claim 10.

12. The one or more inspection operations include defect detection utilized to detect defects on the second workpiece. The system according to claim 10.

13. The one or more inspection operations include focus point cloud operations utilized to determine three-dimensional profile data of the second workpiece. The system according to claim 10.

14. When the program instructions are executed by the one or more processors, the one or more processors are further enabled to allow a user to control independent illumination variables utilized as part of the illumination optimization process. The system according to claim 1.

15. When the program instructions are executed by the one or more processors, when the defect detection illumination optimization mode is selected, the one or more processors are further caused to receive from the user information regarding at least one of the following: The information is: A workpiece having a defect, where the user indicates the area having the defect so that the illumination optimization process utilizes the information to provide illumination that maximizes the contrast of the area; or At least one characteristic of the defect or the surface of the workpiece, where the characteristic is utilized by the illumination optimization process to determine whether the optimized illumination corresponds to surface illumination or dark field illumination having a maximized texture of at least one surface of interest. The system according to claim 1.

16. When the defect detection illumination optimization mode is selected, the illumination optimization process is configured to communicate with a defect detection process that evaluates the proposed illumination and provides feedback regarding the accuracy with which a known defect area illuminated by the proposed illumination can be detected. The system according to claim 1.

17. When the program instructions are executed by the one or more processors, the one or more processors are caused to collect a set of a plurality of images utilized for the illumination optimization process, and in at least one of the plurality of images, each illumination channel to be optimized is on. The system according to claim 1.

18. One or more of the lighting channels of the lighting configuration are movable lighting channels, and each of the movable lighting channels is configured to be controllable to move relative to other lighting channels of the lighting configuration in order to adjust at least one of the position or direction of the lighting provided by the movable lighting channel, and the lighting optimization process includes optimizing at least one of the position or direction of the movable lighting channel. The system according to claim 1.

19. The system according to claim 1, wherein the lighting optimization process is configured to utilize a negative color lighting channel through a process including subtracting one image from another image.

20. A method for operating a system for performing a lighting optimization process, wherein the system includes a lens configured to input image light generated from a workpiece, the lens being configured to transmit the image light along an image optical path and having an optical axis; a camera configured to receive the image light transmitted along the image optical path and provide an image of the workpiece; a lighting configuration including a lighting channel configured to illuminate the workpiece to generate the image light; and has the method includes providing an option for selecting a lighting optimization mode that is at least one of an edge detection lighting optimization mode, a defect detection lighting optimization mode, or a focus point group lighting optimization mode; receiving the selection of the lighting optimization mode; performing a lighting optimization process based on the selected lighting optimization mode, the lighting optimization process including determining lighting for illuminating the workpiece including settings for the lighting channels of the lighting configuration. and the method includes.

21. A system, wherein a lens configured to input image light generated from a workpiece, the lens being configured to transmit the image light along an image optical path and having an optical axis; a camera configured to receive the image light transmitted along the image optical path and provide an image of the workpiece; a lighting configuration including a lighting channel configured to illuminate the workpiece to generate the image light; and has, and the system includes Provide an option for selecting an illumination optimization mode that is at least one of an edge detection illumination optimization mode, a defect detection illumination optimization mode, or a focus point group illumination optimization mode, Receive the selection of the illumination optimization mode, Execute an illumination optimization process based on the selected illumination optimization mode, the illumination optimization process determining illumination for illuminating the workpiece including settings for illumination channels of an illumination configuration, a system.

Images