Workpiece inspection and defect detection system including monitoring of workpiece images - Patents.com

Abstract

To provide a precision machine vision system for inspecting a workpiece to detect a defect.SOLUTION: A precision machine vision system 10 includes: light sources; lenses for inputting image light emitted from a surface of a workpiece; and a camera 34 for receiving imaging light transmitted along an imaging optical path. The system uses images of the workpiece acquired by the camera as training images and trains a defect detection section so as to detect a defect image including a defective workpiece. The system determines an anomaly detection classification index on the basis of elements of the training images. The system acquires an execution mode image of the workpiece by the camera and determines whether the image of the workpiece is classified as an anomaly using the anomaly detection classification index on the basis of the elements determined from the images. The defect detection section further can determine whether the image is the defect image including the defective workpiece and can perform an additional operation (for example, a measurement operation for measuring dimensions or the like of the defect).SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to workpiece inspection systems, and more particularly, to precision machine vision systems for inspecting workpieces and detecting defects. [Background technology]

[0002] Precision non-contact workpiece inspection systems, such as machine vision inspection systems (or "vision systems" for short), can be used to acquire images of workpieces for inspection. Such systems can be used for various types of applications, such as general workpiece inspection, metrology applications to determine precise dimensional measurements of the workpiece, etc. Such systems typically include a computer, a camera, and an optical system. In some configurations, a moving mechanism (e.g., precision stage, conveyor, etc.) may be included that moves to enable traversal and inspection of the workpiece. One exemplary prior art machine vision inspection system is the QUICK VISION® series of PC-based vision systems and QVPAK® software available from Mitutoyo America Corporation (MAC), located in Aurora, Illinois. The functionality and operation of the QUICK VISION® series of vision systems and QVPAK® software are generally described, for example, in the QVPAK 3D CNC Vision Measuring Machine User's Guide published in January 2003. This type of system utilizes a microscope-type optical system and a moving stage to provide an inspection image of the workpiece. Summary of the Invention [Problem to be solved by the invention]

[0003] Such machine vision inspection systems typically face various types of challenges in inspecting workpieces (e.g., due to variations in the type or surface of the workpiece being inspected, changes in inspection conditions, etc.) Systems that can provide improvements over such challenges in some types of inspection operations (e.g., for workpiece defect detection and / or to improve the accuracy of defect detection) are desired.

[0004] This summary is provided to introduce some concepts in a simplified form that are further described below in the Detailed Description. This summary is not intended to identify key features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. [Means for solving the problem]

[0005] A workpiece inspection and defect detection system is provided that includes a light source, a lens, a camera, one or more processors, and a memory. The lens receives imaging light emanating from a surface of the workpiece illuminated by the light source and transmits the imaging light along an imaging optical path. The camera receives the imaging light transmitted along the imaging optical path and provides an image of the workpiece.

[0006] The memory is coupled to the one or more processors and stores program instructions that, when executed by the one or more processors, cause the one or more processors to at least: acquire at least some of a plurality of training images used to train the defect detector, determine a plurality of training image elements from the plurality of training images, determine a plurality of anomaly detector classification characteristics based at least in part on the determined training image elements, acquire a first run mode image of the workpiece using a camera, determine the plurality of elements from the first run mode image of the workpiece, and determine whether the first run mode image of the workpiece is classified as anomaly using the anomaly detection classifiers based at least in part on the determined elements from the first run mode image of the workpiece.

[0007] In various embodiments, the anomaly detection classifier can be used to determine that the second run mode image of the workpiece is classified as non-anomalous, and the defect detector can be used to analyze the second run mode image to determine whether the second run mode image of the workpiece is classified as a defect image that includes a defect in the workpiece.

[0008] In various embodiments, the first run mode image of the workpiece can be classified as anomalous using the anomaly detection classifier. Further, a message can be generated indicating that there is a problem with at least one of the imaging conditions, the lighting conditions, or the workpiece conditions based at least in part on the classification of the first run mode image of the workpiece as anomalous. Further, a reason why the first run mode image of the workpiece was classified as anomalous can be determined. Also, a message can be generated indicating why the first run mode image of the workpiece is classified as anomalous. In various embodiments, after determining why the first run mode image is classified as anomalous, at least one of the brightness of the light source, or the position of the light source, or the position of the lens or lens element relative to at least one of the camera or the workpiece can be adjusted.

[0009] In various embodiments, a synthetic data set can be generated in which a number of training image elements are modified to simulate anomalous images and are not modified to simulate non-anomalous images, and a two-class classifier can be trained in the synthetic data set that can distinguish between anomalous and non-anomalous images.

[0010] In various embodiments, a method (e.g., a computer-implemented method operating under the control of one or more computing systems configured with executable instructions) for operating a workpiece inspection and defect detection system can be provided that includes using a light source, a lens, and a camera of the workpiece inspection and defect detection system to acquire a plurality of training images for training a defect detector, determining a plurality of training image elements from the training images, determining a plurality of anomaly detection classifiers based at least in part on the training image elements, acquiring a first run mode image of the workpiece, determining the plurality of elements from the first run mode image of the workpiece, and determining whether the first run mode image of the workpiece is classified as anomaly using the anomaly detection classifiers based at least in part on the elements determined from the first run mode image of the workpiece.

[0011] In various embodiments, a non-transitory computer readable medium may be provided that stores program instructions that, when executed by one or more processors, cause the one or more processors to at least: use a light source, a lens, and a camera of a workpiece inspection and defect detection system to acquire at least some of the plurality of training images, determine a plurality of training image elements from the plurality of training images, determine a plurality of anomaly detection classifiers based at least in part on the determined training image elements, acquire a first run mode image of the workpiece with the camera, determine the plurality of elements from the first run mode image of the workpiece, and determine whether the first run mode image of the workpiece is classified as anomaly using the anomaly detection classifiers based at least in part on the determined elements from the first run mode image of the workpiece. [Brief description of the drawings]

[0012] The foregoing aspects and many of the attendant advantages of this invention will become more readily appreciated as the same become better understood by reference to the following detailed description, when taken in conjunction with the accompanying drawings, in which:

[0013] [Figure 1] FIG. 1 illustrates various typical components of a general purpose precision machine vision inspection system. [Diagram 2] FIG. 2 is a block diagram of a control system and vision components portion of a machine vision inspection system similar to that of FIG. 1 and including certain features disclosed in the present invention. [Figure 3A] 2 is an example of an image of a defect-free workpiece portion that may be acquired using a machine vision inspection system similar to that of FIG. 1. [Figure 3B] 2 is an example of an image of a defect-free workpiece portion that may be acquired using a machine vision inspection system similar to that of FIG. 1. [Figure 3C] 2 is an example of an image of a defect-free workpiece portion that may be acquired using a machine vision inspection system similar to that of FIG. 1. [Figure 3D] 2 is an example of an image of a defect-free workpiece portion that may be acquired using a machine vision inspection system similar to that of FIG. 1. [Figure 3E] 2 is an example of an image of a defect-free workpiece portion that may be acquired using a machine vision inspection system similar to that of FIG. 1. [Figure 3F] 2 is an example of an image of a defect-free workpiece portion that may be acquired using a machine vision inspection system similar to that of FIG. 1. [Figure 4A] 2 is an example of an image of a workpiece portion including a defect that may be acquired using a machine vision inspection system similar to that of FIG. 1. [Figure 4B] 2 is an example of an image of a workpiece portion including a defect that may be acquired using a machine vision inspection system similar to that of FIG. 1. [Figure 4C] 2 is an example of an image of a workpiece portion including a defect that may be acquired using a machine vision inspection system similar to that of FIG. 1. [Figure 4D] 2 is an example of an image of a workpiece portion including a defect that may be acquired using a machine vision inspection system similar to that of FIG. 1. [Figure 4E] 2 is an example of an image of a workpiece portion including a defect that may be acquired using a machine vision inspection system similar to that of FIG. 1. [Figure 5A] 5B through 5D, taken together, are defect-free examples showing images of a workpiece portion that may be acquired using a machine vision inspection system similar to that of FIG. [Figure 5B] 5A, 5C, and 5D, taken together, are examples showing images of a workpiece portion with defects that may be acquired using a machine vision inspection system similar to that of FIG. [Figure 5C] 5A, 5B, and 5D, taken together, are examples of images of a workpiece portion with defects that may be acquired using a machine vision inspection system similar to that of FIG. [Figure 5D]5A-5C together show examples of images of a workpiece portion with defects that may be acquired using a machine vision inspection system similar to that of FIG. [Figure 6] FIG. 1 illustrates the use of one or more video tools to perform metrology operations on an image of a workpiece containing a defect. [Figure 7A] 7B through 7J, taken together, are defect-free examples showing images of a workpiece portion that may be acquired by a machine vision inspection system similar to that of FIG. 1 at various brightness levels. [Figure 7B] FIG. 7A, along with FIGS. 7C through 7J, are defect-free examples showing images of a workpiece portion that may be acquired by a machine vision inspection system similar to that of FIG. 1 at various brightness levels. [Figure 7C] 7A, 7B, and 7D-7J, along with other defect-free examples, show images of a workpiece portion that may be acquired by a machine vision inspection system similar to that of FIG. 1 at various brightness levels. [Figure 7D] 7A-7C, along with 7E-7J, are defect-free examples showing images of a workpiece portion that may be acquired by a machine vision inspection system similar to that of FIG. 1 at various brightness levels. [Figure 7E] 7A-7D, along with 7F-7J, are defect-free examples showing images of a workpiece portion that may be acquired by a machine vision inspection system similar to that of FIG. 1 at various brightness levels. [Figure 7F] 7A-7E, along with 7G-7J, are examples showing images of a workpiece portion with defects that may be acquired by a machine vision inspection system similar to that of FIG. 1 at various brightness levels. [Figure 7G] 7A-7F, along with 7H-7J, are examples showing images of a workpiece portion with defects that may be acquired by a machine vision inspection system similar to that of FIG. 1 at various brightness levels. [Figure 7H]7A through 7G, along with 7I and 7J, are examples showing images of a workpiece portion with defects that may be acquired by a machine vision inspection system similar to that of FIG. 1 at various brightness levels. [Figure 7I] 7A through 7H, along with FIG. 7J, are examples showing images of a workpiece portion with defects that may be acquired by a machine vision inspection system similar to that of FIG. 1 at various brightness levels. [Figure 7J] 7A-7I, taken together, are examples of defects showing images of a workpiece portion that may be acquired by a machine vision inspection system similar to that of FIG. 1 at various brightness levels. [Figure 8A] 8B through 8F, together with a defect-free example, show images of a workpiece portion that may be acquired by a machine vision inspection system similar to that of FIG. 1 at various focus levels. [Figure 8B] FIG. 8A, along with FIGS. 8C-8F, are defect-free examples showing images of a workpiece portion that may be acquired by a machine vision inspection system similar to that of FIG. 1 at various focus levels. [Figure 8C] 8A, 8B, and 8C through 8F are defect-free examples showing images of a workpiece portion that may be acquired by a machine vision inspection system similar to that of FIG. 1 at various focus levels. [Figure 8D] 8A-8C, along with 8E-8F, are examples showing images of a workpiece portion with defects that may be acquired by a machine vision inspection system similar to that of FIG. 1 at various levels of focus. [Figure 8E] 8A-8D, along with FIG. 8F, are examples showing images of a workpiece portion with defects that may be acquired by a machine vision inspection system similar to that of FIG. 1 at various levels of focus. [Figure 8F] 8A-8E, taken together, are examples of defects showing images of a workpiece portion that may be acquired by a machine vision inspection system similar to that of FIG. 1 at various levels of focus. [Figure 9]1 is an example of an image augmentation user interface according to the present disclosure. [Figure 10] 2 shows a flow diagram of a method for operating a machine vision inspection system similar to that of FIG. 1 . [Figure 11] 2 shows a flow diagram of a method for operating a machine vision inspection system similar to that of FIG. 1 . DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] 1 is a block diagram of one exemplary machine vision inspection system 10 that can be used as an imaging system in accordance with the methods described herein. The machine vision inspection system 10 includes a vision inspection machine 12 operably connected to exchange data and control signals with a control computer system 14. The control computer system 14 is further operably connected to exchange data and control signals with a monitor or display 16, a printer 18, a joystick 22, a keyboard 24, and a mouse 26. The monitor or display 16 can display a user interface suitable for controlling and / or programming the operation of the machine vision inspection system 10. It will be appreciated that in various exemplary embodiments, the functionality of any or all of the elements 14, 16, 22, 24, and 26 can be substituted and / or provided redundantly, such as by a touch screen tablet and / or similar device.

[0015] Those skilled in the art will appreciate that the control computer system 14 and / or other control systems described herein may generally be implemented using any suitable computing system or device, including distributed or networked computing environments, etc. Such computing systems or devices may include one or more general-purpose or special-purpose processors (e.g., non-custom or custom devices) that execute software to achieve the functions described herein. The software may be stored in memory, such as random access memory (RAM), read-only memory (ROM), flash memory, or a combination of such components. The software may also be stored in one or more storage devices, such as optical disks, flash memory devices, or any other type of non-volatile storage medium for storing data. The software may include one or more program modules, including routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. In a distributed computing environment, the functionality of the program modules may be combined or distributed across multiple computing systems or devices, in either a wired or wireless configuration, and accessed via service calls.

[0016] The vision inspection machine 12 includes a movable workpiece stage 32 and an optical imaging system 34, which may include a zoom lens or interchangeable lenses that generally provide different magnifications for the images acquired by the optical imaging system 34.

[0017] Figure 2 is a block diagram of a control system portion 120 and a vision components portion 200 of a machine vision inspection system 100 that includes several features described herein similar to the machine vision inspection system of Figure 1. The control system portion 120 is used to control the vision components portion 200, as described in more detail below. The control system portion 120 can be configured to exchange data and control signals with the vision components portion 200. The vision components portion 200 includes an optical assembly portion 205, light sources 220, 230, 240, 300, and a workpiece stage 210 having a central transparent portion 212. The workpiece stage 210 can be controllably moved along x- and y-axes that lie in a plane generally parallel to a surface of the stage on which a workpiece 20 can be placed.

[0018] The optical assembly portion 205 includes a camera system 260 and an interchangeable objective lens 250. It may optionally include a variable focal length (VFL) lens, such as a tunable acoustic gradient (TAG) lens.

[0019] In various exemplary embodiments, the optical assembly portion 205 may further include a turret lens assembly 280 having lenses 286 and 288. Instead of a turret lens assembly, various exemplary embodiments may include fixed or manually interchangeable magnification-altering lenses, zoom lens configurations, etc. In various exemplary embodiments, the interchangeable objective lens 250 may be selected from a set of fixed magnification objective lenses (e.g., a set of objective lenses corresponding to magnifications of 0.5x, 1x, 2x or 2.5x, 5x, 10x, 20x or 25x, 50x, 100x, etc.) included as part of the variable magnification lens portion.

[0020] The optical assembly portion 205 can be controllably moved along a z-axis, generally orthogonal to the x-axis and y-axis, using a controllable motor 294. The controllable motor 294 drives an actuator to move the optical assembly portion 205 along the z-axis to vary the focus of an image of the workpiece 20. The controllable motor 294 is connected to the input / output interface 130 via a signal line 296 to vary the focus of the image over a particular range. The workpiece 20 can be positioned on a workpiece stage 210. The workpiece stage 210 is controlled to move relative to the optical assembly portion 205 such that the field of view of the interchangeable objective lens 250 can be moved between multiple positions on the workpiece 20 and / or between multiple workpieces 20.

[0021] One or more of the transmitted illumination source 220, the epi-illumination source 230, and the oblique illumination source 240 (e.g., a ring illumination) may emit source light 222, 232, and / or 242, respectively, to illuminate one or more workpieces 20. For example, during an image exposure, the epi-illumination source 230 may emit source light 232 along a path that includes a beam splitter 290 (e.g., a partial mirror). The source light 232 is reflected or transmitted as workpiece image light 255, which is used for imaging, passes through an interchangeable objective lens 250 and a turret lens assembly 280, and is collected by a camera system 260. The workpiece image exposure, which includes an image of one or more workpieces 20, is captured by the camera system 260 and output on a signal line 262 to the control system portion 120.

[0022] The various light sources (e.g., light sources 220, 230, 240, 300) can be connected to the lighting control interface 133 of the control system portion 120 via associated signal lines (e.g., buses 221, 231, 241, 331, respectively). The control system portion 120 can control the selection of one of the turret lenses by rotating the turret lens assembly 280 along axis 284 via signal line or bus 281 to change the magnification of the image.

[0023] 2, in various exemplary embodiments, the control system section 120 includes a controller 125, an input / output interface 130, a memory 140, a workpiece program generator and executor 170, and a power supply section 190. Each of these components, as well as additional components described below, may be interconnected by one or more data / control buses and / or application programming interfaces, or by direct connections between the various elements. The input / output interface 130 includes an imaging control interface 131, a motion control interface 132, a lighting control interface 133, and a lens control interface 134.

[0024] The illumination control interface 133 can include illumination control elements 133a-133n that control, for example, the selection, power, and on / off switching of various corresponding light sources of the machine vision inspection system 100. The illumination control interface 133 also includes an illumination control element 133sip, which in the illustrated embodiment operates in conjunction with a structured illumination pattern (SIP) generator 300 to provide structured illumination during image acquisition. In various embodiments, a projection pattern can be output from the SIP generator 300 and input to a beam splitter 290 that is directed as epi-illumination light through the objective lens 250 to provide SIP structured light 232' that illuminates the field of view.

[0025] Memory 140 may include an image file memory portion 141, a defect detection portion 140dp, a workpiece program memory portion 142 that may include one or more part programs, and a video tool portion 143. Video tool portion 143 includes a video tool portion 143a and other video tool portions (e.g., 143n) that determine the GUI, image processing operations, etc. for each corresponding video tool, as well as a region of interest (ROI) generator 143roi. Region of interest generator 143roi supports automatic, semi-automatic, and / or manual operation of defining various ROIs operable in the various video tools included within video tool portion 143.

[0026] The video tools section 143 also includes an autofocus video tool 143af that determines GUIs for focus height measurement operations, image processing operations, and the like. In various exemplary embodiments, the autofocus video tool 143af may further include a fast focus height tool that may be utilized to measure focus heights at high speed using hardware, as described in more detail in U.S. Pat. No. 9,143,674, which is incorporated herein by reference in its entirety. In various exemplary embodiments, the fast focus height tool may be a special mode of the autofocus video tool 143af that otherwise operates according to conventional methods for autofocus video tools, or the operation of the autofocus video tool 143af may include only the operation of the fast focus height tool. Fast autofocus and / or focus position determination for one or more image regions of interest may be based on analyzing the image to determine corresponding quantitative contrast measures of the various regions according to known methods.

[0027] In the context of this disclosure, as known to those skilled in the art, the term "video tool" generally refers to a relatively complex automated or programmed set of operations that can be performed by a machine vision user through a relatively simple user interface. For example, a video tool may include a complex set of pre-programmed image processing operations and calculations that can be adapted and customized in a particular instance by adjusting a small number of variables or parameters that define the operations and calculations. In addition to the underlying operations and calculations, a video tool also includes a user interface that allows a user to adjust those parameters for a particular instance of the video tool. It should be noted that in some cases, visible user interface features are referred to as video tools and the underlying operations are implicitly included.

[0028] One or more display devices 136 (e.g., display 16 of FIG. 1 ) and one or more input devices 138 (e.g., joystick 22, keyboard 24, and mouse 26 of FIG. 1 ) may be connected to input / output interface 130. Display device 136 and input device 138 may be used to display a user interface, which may include various graphical user interface (GUI) functions that may be used to perform inspection operations and / or create and / or modify part programs, view images captured by camera system 260, and / or directly control vision components portion 200.

[0029] In various exemplary embodiments, when a user uses the machine vision inspection system 100 to generate a part program for a workpiece 20, the user generates the part program instructions by operating the machine vision inspection system 100 in a learn mode to provide a desired image acquisition training sequence. For example, the training sequence may include placing a particular workpiece feature of a representative workpiece within a field of view (FOV), setting illumination levels, focusing or auto-focusing, acquiring an image, and providing an inspection training sequence to be applied to the image (e.g., using an instance of one of the video tools for that workpiece feature). The learn mode operates such that this sequence or sequences are captured or recorded and converted into corresponding part program instructions. When the part program is executed, these instructions cause the machine vision inspection system to replicate the trained image acquisition and perform inspection operations to automatically inspect the particular workpiece features (i.e., corresponding features at corresponding locations) on one or more workpieces in the run mode that match the representative workpieces used in generating the part program. In various illustrative embodiments, several types of training modes may additionally or alternatively be utilized (e.g., a training mode to train the defect detector to detect defects, a training mode to train the anomaly detector to detect anomaly images that are not acceptable to the defect detection process, etc.).

[0030] The video tools portion 143 also includes a Z-height measurement tools portion 143z that provides various operations and functions related to Z-height measurement operations. In one embodiment, the Z-height measurement tools portion 143z can include a Z-height tool 143zt. The Z-height tools 143zt can include, for example, an autofocus tool 143af and a multipoint autofocus video tool 143maf. The Z-height tools 143zt can manage some aspects of the image stack acquisition and associated structured light pattern generation operations in conjunction with a Z-height tool configured in a mode to determine a best focus height and / or Z-height measurement value. In general, the Z-height measurement tools portion 143z can perform at least some operations similar to known Z-height measurement tools, such as performing a learn mode and / or a run mode or other modes of operation to generate all or a portion of a focus curve and finding its peak as the best focus position.

[0031] The defect detector 140dp performs various defect detection operations as described in detail below. Before performing such defect detection operations, the defect detector 140dp is trained using a training image set captured using specific (e.g., current) imaging, lighting, and workpiece conditions. These conditions may change (e.g., drift) significantly when the defect detector 140dp is used for defect detection in an online or offline environment over time. If the appearance of the images used during defect detection differs significantly from the appearance of the images in the training image set, the performance of the trained defect detector 140dp may degrade. Thus, the defect detector 140dp may include a monitoring (e.g., anomaly detection) system that may be generated / defined (e.g., using some or all of the training image set during training of the defect detector 140dp). The system can distinguish between normal (e.g., acceptable) and abnormal (e.g., unacceptable) input images and alerts the user when the appearance of the input image during defect detection is no longer of a sufficient level that may negatively impact the accuracy of the trained defect detector 140dp, thereby effectively improving the reliability of defect detection by the defect detector 140dp. With respect to image classification, the terms "normal" and "non-abnormal" are generally considered to have the same meaning and are used interchangeably herein.

[0032] In various exemplary embodiments, the defect detector 140dp can perform a defect detection process that is performed in conjunction with the metrology process. In various embodiments, it may be desirable to include the defect detector 140dp in a machine vision inspection system similar to the machine vision inspection system 100 of FIG. 1 that is configured to perform the metrology process, since the system can generate image data that is input to the defect detection process performed by the defect detector 140dp. Thus, a single machine can be configured to perform both the metrology process and the defect detection process, providing advantages over conventional metrology systems. For example, if a defect is detected on a workpiece during the defect detection process, there is no reason to perform the metrology process on the workpiece, thus saving time. More specifically, if a defect is detected on a workpiece during the defect detection process, it may not be necessary to measure the obviously defective portion. Thus, it may be advantageous to perform the defect detection process before the start of the metrology process.

[0033] Additionally, some defects may require further measurement or inspection to determine additional defect parameters. For example, a 2D image allows for quick recognition of a potential defect and quick confirmation of the defect's XY location and approximate XY area. If the 3D characteristics of the potential defect are important, the defect detector 140dp can perform additional processing (e.g., metrology operations) to determine whether the potential defect is an actual defect. For example, if a scratch on the surface of the workpiece must be deeper than a certain threshold to be considered a defect, the defect detector 140dp can obtain a more time-consuming 3D point cloud of the affected area (e.g., using the z-height measurement tool 143z) to know whether the scratch depth is sufficient to reject the part. In various embodiments, various actions can be performed (e.g., programmed to be performed automatically) as a result of the initial defect classification, such as: For example, (1) continue with the standard metrology process, (2) stop or pause the defect detection process and perform a metrology process that includes more informative measures of potential defects (e.g., 3D measurements, measurements with different lighting, touch probe measurements, such as surface roughness), (3) dispose of the workpiece (e.g., scrap or recycle the workpiece), (4) send the workpiece for additional manual inspection, or (5) provide feedback to the production line indicating that there is something wrong with the machine, process, etc.

[0034] In various exemplary embodiments, the defect detection unit 140dp of the memory 140 stores model data and program instructions for various tools and algorithms of the defect detection system that can be used to infer whether various types of defects are present in an image of the workpiece (i.e., to indicate defects in the workpiece surface portion contained within the image). While the machine vision inspection system is operating in a training or learning mode, the defect detection system uses a training image set that includes defective and non-defective images to train the defect detection unit 140dp. The training image set is captured using specific (e.g., current) imaging, lighting, and workpiece conditions. After the defect detection unit 140dp is initially trained, the defect detection unit 140dp is operated in a running mode to infer whether new workpiece images that have not yet been seen contain defects and classify each workpiece image as defective or non-defective accordingly.

[0035] As described in more detail below, FIGS. 3A-3F, 4A-4E, and 5A-5D show examples of some workpiece images that may be acquired by machine vision inspection system 100 and, in some embodiments, may be used to train defect detector 140dp during training mode (and / or some of these images may be examples of workpiece images acquired by machine vision inspection system 100 later during run mode and analyzed by trained defect detector 140dp). Specifically, the example workpiece images of FIGS. 3A-3F and 4A-4E are of various sections (i.e., various XY positions) on a machined aluminum plate. As described in more detail below, FIGS. 3A-3F show examples of non-defective images and FIGS. 4A-4E show examples of defective images (e.g., a training image set typically includes multiple both defective and non-defective workpiece images). The difference between these images helps explain one reason why it is desirable to use multiple training images to train defect detector 140dp. More specifically, because different features may be observed in different workpiece images acquired to inspect a certain type of workpiece (e.g., as shown by the differences between the workpiece images of FIGS. 3A-3F and 4A-4E for inspecting a certain type of machined aluminum plate), the accuracy of defect detection of the defect detector 140dp may be improved by utilizing a variety of workpiece images for training that are similar to and / or allow for superior defect detection in a variety of images that may be acquired later in run mode. For example, particularly with respect to the examples of FIGS. 3A-3F, 4A-4E, and 5A-5D, such training may help the defect detector 140dp to distinguish between defects (e.g., which in the illustrated example may include a variety of different types of scratch defects, etc.) and normal workpiece surface elements (e.g., which in the illustrated example may include a variety of different types of machining marks formed on the plate surface that typically appear as a hashed texture, etc. that varies across the surface).

[0036] While the defect detector 140dp is operating in run mode, the imaging, lighting, and workpiece conditions used by the machine vision inspection system may change (e.g., drift) significantly over time due to, for example, aging of one or more light sources illuminating the workpiece during image data acquisition, dust accumulation (e.g., on lenses), optical misalignment, changes in manufacturing conditions, and other random or systematic factors. As a result, images input to the defect detector 140dp during run mode analysis may differ significantly from the images used to train the defect detector 140dp, which may result in a significant degradation of defect detection performance. For example, imaging conditions may change, causing the defect detector 140dp to interpret non-defective images as defective images, resulting in an increase in false positives. As another example, the changing imaging conditions may make it difficult for the defect detector 140dp to detect or otherwise distinguish actual defects from normal workpiece surface features (e.g., machining marks, etc.).

[0037] Thus, the defect detector 140dp includes an image data monitoring and anomaly detection subsystem that generates a generalized model of the training image set (e.g., defective and non-defective images, which may include all or a portion of the available training images) and stores the generalized model with the defect detector 140dp for use in monitoring image quality during subsequent defect detection. As described in more detail below, in various embodiments, determining the generalized model may include determining training image elements from the training images and determining anomaly detection classification indices based on these training image elements. According to the use of the model, if the appearance / quality of the images used for defect detection deviates too much from the images used to train the defect detector 140dp (e.g., the images may be classified as anomalous), a specified action may be performed (e.g., alerting the user and / or adjusting the system). For example, the user or the system may either adjust the imaging conditions, lighting conditions, and / or workpiece conditions, or retrain the defect detector 140dp to compensate for the deviation and maintain good performance of the defect detection system.

[0038] As discussed above (and as shown in various examples in Figures 3A-3F, 4A-4E, and 5A-5D), training images, which may be of a wide variety, help illustrate the importance of using a training image set to determine a generalized model. More specifically, because training images may have a variety of features, it is beneficial to determine an accurate representation of such differences (e.g., corresponding to ranges or other measures of the various training image elements and their associated anomaly detection classifiers) in accordance with the principles disclosed herein. This helps ensure the accuracy of the anomaly detection process in classifying certain images (e.g., having features that deviate from a particular acceptable range, threshold, etc. of features determined from the training images) as anomalous.

[0039] As noted above, Figures 3A-3F and 4A-4E are example workpiece images of a section of a workpiece that is a relatively "flat" machined aluminum plate. Each image is taken from the same angular viewpoint (e.g., directly above at a 90 degree angle to the plate), but each is at a different XY location on the plate. Each image shows an approximately 2.5 millimeter by 1.9 millimeter (XY) view of the plate surface. Thus, each image shows a magnified view of a portion of the plate surface. Machining marks formed on the plate surface produce a hash texture that typically varies across the surface. In these images, at such a magnified scale, the relatively flat plate may not appear flat. In this example, the height of some of the hash machining mark protrusions may be approximately 5 micrometers or less.

[0040] The sections of the machined aluminum plate shown in Figures 3A-3F do not contain defects. In other words, Figures 3A-3F show examples of images of sections of the machined aluminum plate that are "non-defective". In contrast, Figures 4A-4E show examples of images of sections of the workpiece that contain defects. The images shown in Figures 4A-4E are similar to those shown in Figures 3A-3F, except that the images in Figures 4A-4E include a defect 402 formed on the surface of the machined aluminum plate. The defect 402 in these examples is a scratch formed on the surface of the machined aluminum plate. More specifically, Figure 4A shows scratch defects 402A1 and 402A2, Figure 4B shows scratch defects 402B1 and 402B2, Figure 4C shows scratch defects 402C1 and 402C2, Figure 4D shows scratch defect 402D, and Figure 4E shows scratch defects 402E1 and 402E2. As discussed above, the wide variety of defect and workpiece surface features shown in Figures 3A to 3F and 4A to 4E help explain why it is desirable to use a large number of training images in training the defect detection unit 140dp (e.g., the training image set typically includes a large number of defective and non-defective workpiece images) to help improve the accuracy of the anomaly detection process, and therefore why it is desirable to use a wide variety of training images in determining the training image elements and their associated anomaly detection classification indicators.

[0041] 5A-5D are examples of images of similar portions of a workpiece with and without defects that may be acquired using a machine vision inspection system similar to that of FIG. 1. FIG. 5A shows an example section of a plate that may be classified as "non-defective". FIG. 5B-5D show examples of images of a section of a plate that may be classified as "defective", each image including a scratch defect 502 (e.g., including each of scratch defects 502B, 502C, and 502D). In the examples of FIG. 5A-5D, the images are of similar portions of a workpiece (e.g., the types of machining marks formed on the surfaces of these portions are similar or substantially identical in each image, and the main differences between these images are the characteristics of each of scratch defects 502B, 502C, and 502D). In one exemplary embodiment, the images of FIG. 5A-5C may be included as part of a training image set for training the defect detector 140dp. The image of FIG. 5D may be an example of a run mode image that may be analyzed by the defect detector 140dp to determine whether it should be classified as a defect image or a non-defect image. In various embodiments, the defect detector 140dp may be sufficiently trained to properly classify the image of FIG. 5D as a defect image (e.g., trained with the training images of FIGS. 5A-5C, which include scratch defects 502B and 502C that include similar workpiece portions and have some characteristics similar to scratch defect 502D). In various embodiments, some additional processes may be performed on the example scratch defect 502D. For example, one or more metrology processes may be performed in conjunction with the defect detection process to determine various dimensions or other characteristics of the example scratch defect 502D, as will be described in more detail below in conjunction with FIG. 6. Also, as will be described in more detail below in conjunction with FIGS. 7A-7J and 8A-8F, variations in anomalies may occur in the various run mode images (e.g., similar to and comparable to the images of FIGS. 5A and 5D in certain examples).

[0042] FIG. 6 illustrates the use of one or more video tools to perform metrology operations on an image of a workpiece including a defect (e.g., to determine the defect's dimensions, etc.). As shown, for an image 601 (which may be similar or identical to the image of FIG. 5D, for example) including a scratch defect 602 (which may be similar or identical to scratch defect 502D, for example), a video box tool 606 includes scan lines 608 (which may additionally or alternatively represent a video point tool, etc.) that are used to determine edge locations, dimensions, and / or other aspects of the scratch defect 602. In various exemplary embodiments, the video box tool 606 can be sized, positioned, and rotated until it indicates or defines a region of interest (e.g., an area within the box tool 606). Arrows (which may represent, for example, scan lines, point tools, etc.) shown in FIG. 6 can be used to determine one or more edges of the scratch defect 602. In various demonstrative embodiments, the video box tool 606 can generally use one or more conventional edge gradients along one or more edges of the defect 602 within the region of interest and can determine one or more edges of the defect 602 based on the local magnitude of one or more edge gradients along various scan lines 608, etc.

[0043] In various exemplary embodiments, such metrology operations may also include performing certain morphological or other filtering (e.g., to distinguish the edges of the scratch from the machining patterns of the workpiece). As shown in FIG. 6, a box tool 606 is used in conjunction with scan lines 608 to determine an edge location (e.g., outer edge or perimeter) of the scratch defect 602 in a viewing area contained within the image. Based on such determination, video tools and / or other metrology operations may include determining a dimension D1 of the scratch defect (e.g., corresponding to a length or other dimension of the scratch defect 602). In various exemplary embodiments, the box tool 606, scan lines 608, and / or other video tools and / or metrology operations may also be utilized to determine other dimensions of the scratch defect 602 (e.g., width, depth, etc.). For example, as described above, the video tool portion 143 may include a Z-height measurement tool portion 143z, and a corresponding video tool or operation may be used to determine a Z-height dimension of the scratch defect (e.g., including determining the depth of the scratch relative to other portions or features of the workpiece surface, etc.).

[0044] As part of the general operation of the defect detector, some defects that are detected may require further measurement or inspection to determine additional defect parameters. For example, as described above, various types of analysis and / or processing of defect images, including scratch defect 602 (e.g., using the video tools and / or other operations described above) may allow the XY location and approximate XY area and / or other dimensions of defect 602 to be determined. If the 3D characteristics of the potential defect are important (e.g., if the scratch must be deeper than a certain value to be considered a defect), defect detector 140dp may initiate a process to utilize a Z-height measurement tool (e.g., of Z-height measurement tool portion 143z) or other 3D detection process (e.g., to obtain a 3D point cloud of the affected area to determine scratch depth, etc.).

[0045] As described in more detail below, Figures 7A-7J are example images of similar portions of a workpiece with and without defects at various brightness levels (e.g., similar to Figures 5A and 5D) that may be acquired by a machine vision inspection system similar to that of Figure 1. For various brightness levels, in various exemplary embodiments, the training images may be analyzed to determine training image elements (e.g., with respect to brightness and / or other characteristics) and corresponding anomaly detection classification indices may be determined based on these training image elements, as described above. Following such determination, in one exemplary embodiment, the machine vision inspection system 100 may be determined to provide robust performance accuracy for the training images at approximately ±15% of the baseline brightness of the transmitted illumination light source 220, the epi-illumination light source 230, and / or the oblique illumination light source 240 used to illuminate the workpiece 20. As the transmitted illumination light source 220, the epi-illumination light source 230, and the oblique illumination light source 240 are used, the respective brightnesses may degrade over time. Alternatively or in addition, the brightness of the trans-illumination light source 220, epi-illumination light source 230, and oblique illumination light source 240 may be inadvertently set to values ​​significantly different from those used to acquire the training images, which may result in less accurate image defect detection performance. Alternatively or in addition, the camera aperture or other light transmitting optical elements (e.g., lenses, mirrors, filters, etc.) may change from the conditions or settings used to acquire the training images, which may result in more or less light reaching the camera imager. Alternatively or in addition, the ambient room light may change from the intensity at which the training images were acquired, which may result in more or less light reaching the camera imager. Therefore, it may be desirable to inspect image data generated by the machine vision inspection system 100 to determine whether the brightness of the image being inspected exceeds ±15% of a baseline brightness (e.g., ±15% of the baseline brightness may represent an anomaly detection classification indicator) and to take specified action if the brightness is outside of a range that results in robust defect detection performance accuracy (e.g., alerting a user that there is a problem with the lighting conditions of the workpiece 20 and / or adjusting one or more light settings, etc.).

[0046] Thus, the defect detection portion 140dp may include an image data monitoring and anomaly detection subsystem that processes image data corresponding to an image of the workpiece and is capable of detecting whether the image data is generated at an intensity outside of a range for which robust performance accuracy can be achieved (e.g., ±15% of the baseline intensity). After training the image data monitoring and anomaly detection subsystem with a training image set, the image data monitoring and anomaly detection subsystem is capable of detecting whether operational image data is generated at an intensity outside of a range for which robust performance accuracy can be achieved.

[0047] 7A-7J are example images of similar portions of a workpiece, with and without defects, at various brightness levels, that may be acquired, for example, during run mode, by a machine vision inspection system similar to that of FIG. 1 and / or may be used as training images. In the examples of FIG. 7A-7J, the images are of similar portions of the workpiece (e.g., the types of machining marks formed on the surfaces of these portions are similar or substantially identical in each image, and the primary difference between these images is the brightness level). The images of FIG. 7A-7E are non-defect images (e.g., may be compared to the image of FIG. 5A), and the images of FIG. 7F-7J are defect images (e.g., may be compared to FIG. 5D, and include defects 702F-702J similar to defect 502D).

[0048] In one example embodiment, the image data monitoring and anomaly detection subsystem can be trained to classify image data as "normal" or "non-anomalous" if the image data exhibits a luminance (e.g., average luminance) within ±15% of the baseline luminance, and to classify the image data as "abnormal" or "anomalous" if the image data exhibits a luminance that is not within ±15% of the baseline luminance. For example, if the image data monitoring and anomaly detection subsystem determines that the image data exhibits a luminance that is ±25% of the baseline luminance, the image data monitoring and anomaly detection subsystem can classify the image data as "abnormal" or "anomalous."

[0049] 7A is a non-defective image of a workpiece having a baseline brightness. In accordance with this example, the image data monitoring and anomaly detection subsystem can classify image data corresponding to the image in FIG. 7A as "normal" or "non-anomalous."

[0050] 7B is a non-defective image of a workpiece having a brightness 25% less than the baseline brightness. Pursuant to this example, the image data monitoring and anomaly detection subsystem can classify image data corresponding to the image of FIG. 7B as "abnormal" or "unusual."

[0051] 7C is a non-defective image of a workpiece having a brightness 15% lower than the baseline brightness. In accordance with this example, the image data monitoring and anomaly detection subsystem can classify the image data corresponding to the image of FIG. 7C as "normal" or "non-anomalous."

[0052] Figure 7D is a non-defective image of a workpiece having a brightness 15% greater than the baseline brightness. Pursuant to this example, the image data monitoring and anomaly detection subsystem can classify image data corresponding to the image of Figure 7D as "normal" or "non-anomalous."

[0053] 7E is a non-defective image of a workpiece captured while illuminating the workpiece with a brightness 25% greater than the baseline brightness. Pursuant to this example, the image data monitoring and anomaly detection subsystem can classify image data corresponding to the image of FIG. 7E as "abnormal" or "anomalous."

[0054] 7F is a defect image of a workpiece having a baseline brightness. In accordance with this example, the image data monitoring and anomaly detection subsystem can classify image data corresponding to the image of FIG. 7F as "normal" or "non-anomalous."

[0055] 7G is a defect image of a workpiece having a brightness 25% lower than the baseline brightness. Pursuant to this example, the image data monitoring and anomaly detection subsystem can classify image data corresponding to the image of FIG. 7G as "abnormal" or "unusual."

[0056] 7H is a defect image of a workpiece having a brightness 15% lower than the baseline brightness. Pursuant to this example, the image data monitoring and anomaly detection subsystem can classify the image data corresponding to the image of FIG. 7H as "normal" or "non-anomalous."

[0057] 7I is a defect image of a workpiece having a brightness 15% higher than the baseline brightness. Pursuant to this example, the image data monitoring and anomaly detection subsystem can classify image data corresponding to the image of FIG. 7I as "normal" or "non-anomalous."

[0058] 7J is a defect image of a workpiece having a brightness 25% greater than the baseline brightness. Pursuant to this example, the image data monitoring and anomaly detection subsystem can classify image data corresponding to the image of FIG. 7J as "abnormal" or "unusual."

[0059] It will be appreciated that while some of the image pairs exhibiting 15% and 25% brightness levels (e.g., the images of Figures 7B and 7C, 7D and 7E, 7G and 7H, and 7I and 7J) may not appear significantly different to the human eye, such differences may be significant to the operation of defect detector 140dp. For example, in some embodiments, a brightness level difference of 25% relative to a baseline brightness level may be determined to inhibit the ability of defect detector 140dp to accurately distinguish between defects (e.g., defects 702G and 702J) and normal workpiece surface elements (e.g., with respect to determining whether such defects are present in other surface portions of the images of Figures 7G and 7J and / or images such as Figures 7B and 7E).

[0060] As described in more detail below, Figures 8A-8F are example images of similar portions of a workpiece with and without defects at various focus levels (e.g., similar to Figures 5A and 5D) that may be acquired by a machine vision inspection system similar to that of Figure 1. For various focus levels, in various exemplary embodiments, the training images may be analyzed to determine training image elements (e.g., for focus and / or other characteristics) and corresponding anomaly detection classification indices may be determined based on these training image elements, as described above. Following such determination, in one exemplary embodiment, the machine vision inspection system 100 may be determined for the training images to provide robust performance accuracy when the images are acquired with the workpiece surface of interest within an acceptable depth of field (DOF) range of the camera system 260. The DOF of the camera system may be compared to the distance of the workpiece from the best focus distance to indicate the relative amount of focus or defocus in the images. Images captured with a workpiece within a certain DOF range centered around the best focus distance may be considered to be in good focus, while images captured with a workpiece farther away from the best focus distance but less than some threshold DOF range (e.g., 4 times the DOF) may be considered to have an acceptable amount of defocus that still results in robust defect detection performance accuracy, whereas images captured with a workpiece outside the threshold DOF range (e.g., outside 4 times the DOF, such as 6 times the DOF that is farther or closer to the camera system than the best focus distance) may be considered to have an unacceptable amount of defocus that does not result in robust defect detection performance accuracy.

[0061] 8A-8F are example images of similar portions of a workpiece, with and without defects, at various focus levels, that may be acquired by a machine vision inspection system similar to that of FIG. 1, for example, during run mode and / or used as training images. In the examples of FIG. 8A-8F, the images are of similar portions of the workpiece (e.g., the types of machining marks formed on the surfaces of these portions are similar or substantially identical in each image, and the primary difference between these images is the focus level). The images of FIG. 8A-8C are non-defect images (e.g., can be compared to the image of FIG. 5A), and the images of FIG. 8D-8F are defect images (e.g., can be compared to FIG. 5D, and include defects 802D-802F similar to defect 502D).

[0062] In one example embodiment, the image data monitoring and anomaly detection subsystem can be trained to classify image data as "normal" or "non-anomalous" if the image data exhibits a defocus within four times the system DOF, and to classify image data as "abnormal" or "anomalous" if the image data does not exhibit a defocus within four times the baseline DOF. For example, if the image data monitoring and anomaly detection subsystem determines that the image data exhibits a defocus within eight times the system DOF, the image data monitoring and anomaly detection subsystem can classify the image data as "abnormal" or "anomalous" (e.g., can issue a warning to a user and / or take a specified action, such as automatically adjusting the system, e.g., adjusting the lens or workpiece position to achieve better focus, etc.).

[0063] 8A is a non-defective image of a workpiece having focus within the system DOF of best focus. In accordance with this example, the image data monitoring and anomaly detection subsystem can classify image data corresponding to the image in FIG. 8A as "normal" or "non-anomalous."

[0064] Figure 8B is a non-defective image of a workpiece that is defocused from best focus by four times the system DOF. For example, if an acceptable image defocus threshold is four times the DOF, then no user warning should be provided since the amount of defocus for this image is within acceptable tolerances. Thus, the image data monitoring and anomaly detection subsystem can classify the image data corresponding to the image in Figure 8B as "normal" or "non-anomalous."

[0065] Figure 8C is a non-defective image of a workpiece that is defocused from best focus by 8 times the system DOF. In accordance with this example, the image data monitoring and anomaly detection subsystem may classify the image data corresponding to the image of Figure 8C as "abnormal" or "unusual." As such, the image data monitoring and anomaly detection subsystem may issue a user warning and / or take other action as to the image not being deemed suitable for the defect detection process (e.g., adjust parameters or positions of components of camera system 260 to produce an acceptable DOF).

[0066] 8D is a defect image of a workpiece having focus within the system DOF of best focus. According to this example, the image data monitoring and anomaly detection subsystem can classify image data corresponding to the image of FIG. 8D as "normal" or "non-anomalous."

[0067] Figure 8E is a defect image of a workpiece that is defocused from best focus by four times the system DOF. Pursuant to this example, the image data monitoring and anomaly detection subsystem can classify image data corresponding to the image in Figure 8E as "normal" or "non-anomalous."

[0068] Figure 8F is a defect image of a workpiece that is defocused from best focus by 8 times the system DOF. Pursuant to this example, the image data monitoring and anomaly detection subsystem can classify image data corresponding to the image in Figure 8F as "abnormal" or "unusual."

[0069] As discussed above, image data corresponding to images similar to those shown in FIGS. 7A-7J and 8A-8F may correspond to run mode images classified as normal / non-anomalous or abnormal / anomalous by the image data monitoring and anomaly detection subsystem. In various embodiments, such images may represent training images included in a training image set used to train the image data monitoring and anomaly detection subsystem of the defect detector 140dp. In some embodiments, such images may be based on image data that has been enhanced to have a particular quality useful in training the image data monitoring and anomaly detection subsystem. For example, image data corresponding to the augmented image set may be generated based on a parameter indicative of an augmentation limit. In various embodiments, the parameter indicative of the augmentation limit may be defined automatically.

[0070] Additionally, parameters indicative of the dilation limits (e.g., for generating training images) may be defined under user control. Figure 9 is an example of an image augmentation graphical user interface 900 that may be used to define parameters indicative of the dilation limits. The image augmentation graphical user interface 900 displays a selected baseline image 902.

[0071] The image enhancement graphical user interface 900 includes a checkbox 904 that, when checked, enables the brightness enhancement feature. The brightness enhancement feature allows a user to specify a parameter indicating a lower brightness limit and a parameter indicating an upper brightness limit for generating enhanced training images. More specifically, the graphical user interface 900 includes a slider 906 having a handle 908 that can be selected and moved with a user input device (e.g., a mouse) to set a value 910 of the parameter indicating the displayed lower brightness limit (e.g., 5). A corresponding image 912 is displayed, which is a copy of the baseline image 902 that has been modified to have a lower brightness based on the parameter indicating the lower brightness limit. Similarly, the graphical user interface 900 includes a slider 914 having a handle 916 that can be moved to set a value 918 of the parameter indicating the upper brightness limit (e.g., 7). A corresponding image 920 is displayed, which is a copy of the baseline image 902 that has been modified to have a higher brightness based on the parameter indicating the upper brightness limit.

[0072] The image enhancement graphical user interface 900 also includes a checkbox 922 that, when checked, enables the focus enhancement feature. The focus enhancement feature allows a user to specify a parameter indicating an acceptable focus range for generating enhanced training images. More specifically, the graphical user interface 900 includes a slider 924 having a handle 926 that can be selected and moved to set a value 928 of a parameter indicating an acceptable focus range centered around a best focus distance as a multiple of the system depth of field (e.g., 3 times the DOF). A corresponding image 930 is displayed, which is a copy of the baseline image 902 that has been modified to defocus based on the parameter indicating the depth of field.

[0073] The image enhancement graphical user interface 900 also includes a checkbox 932 that, when checked, enables the noise enhancement feature. The noise enhancement feature allows a user to specify a parameter indicating the amount of noise for generating enhanced training images. More specifically, the graphical user interface 900 includes a slider 934 having a handle 936 that can be selected and moved to set a value 938 of a parameter indicating the maximum amount of digital noise (e.g., 10) for which robust defect detection accuracy is desired. A corresponding image 940 is displayed, which is a copy of the baseline image 902 that has been modified to include additional noise based on the parameter indicating the amount of noise.

[0074] The various parameters set using the image augmentation graphical user interface 900 can be used to generate additional image data for the augmented image set. In various exemplary embodiments, the same values ​​of the various augmentation parameters (e.g., 910, 918, 928, 938) used to generate the image data corresponding to images 912, 920, 930, and 940 based on the baseline image 902 can be applied to all or a selected subset of the image data of the training image set to generate the additional image data for the augmented image set.

[0075] 10 and 11 show a flow diagram of a method 1000 for operating a machine vision inspection system similar to that of FIG. 1. The method 1000 begins at 1010. At 1010, a plurality of training images are acquired. As a specific example, in one embodiment, some or all of the images in FIGS. 3A-3F, 4A-4E, and 5A-5D may be acquired and used as training images. Although a limited number of exemplary training images are shown in these figures, it will be appreciated that in various embodiments, a different number of training images (e.g., 50, 100, 150, etc.) may be utilized. In various embodiments, at least some of the plurality of training images are used to train the defect detector. For example, the training images may include original images of the workpiece acquired by the camera system 260 and input to the defect detector 140dp operating in a training mode (e.g., images in FIGS. 3A-3F, 4A-4E, and 5A-5D, etc.). Additionally, the plurality of training images may include an augmented image set generated using, for example, the image augmentation graphical user interface 900 shown in Figure 9. The method 1000 then proceeds to 1020.

[0076] At 1020, a number of training image elements are determined from the number of training images acquired at 1010. In various embodiments, the training image elements (e.g., determined automatically and / or at least in part by a system designer) may include elements such as average image luminance, standard deviation of luminance, grayscale histogram, dominant frequency (e.g., obtained by performing a Fast Fourier Transform (FFT)), and average contrast measure. Method 1000 then proceeds to 1030.

[0077] At 1030, a plurality of anomaly detection classifiers are determined based at least in part on the plurality of training image elements determined at 1020. In various exemplary embodiments, an automated process and / or a system designer may determine the anomaly detection classifiers based at least in part on the determined training image elements. For example, it may be determined to classify image data having an average image brightness that is not within ±15% of the baseline image brightness as anomalous, and / or to classify image data that exhibits a defocus from best focus of more than four times the system depth of field (DOF) as anomalous, otherwise classifying the image data as normal or non-anomalous. In various embodiments, the training image elements are extracted automatically (e.g., by a pre-trained convolutional neural network, etc.), and from these extracted elements, a generalized model of the "normal" training data set may be generated (e.g., defined, trained, fitted) (e.g., various types of models may include manually defined thresholds on selected elements, thresholds based on vector similarity (e.g., which may be manually adjusted), single-class support vector machines (anomaly detectors), or autoencoders (anomaly detectors), etc.). In various alternative embodiments, a synthetic dataset may also be generated by simulating abnormal and normal images with or without changing / varying / perturbing various factors, and training a two-class classifier to distinguish between normal and abnormal images in the synthetic dataset.In various exemplary embodiments, a two-class classifier is used to classify the elements, and may be based on logistic regression analysis, nearest neighbor analysis, discriminant analysis, a Naive Bayes classifier, a decision tree, a support vector machine (SVM), a neural network, a combination of classifiers, or a traditional network trained on ImageNet, such as those described by Zeiler and Fergus in "Visualizing and Understanding Convolutional Networks," arXiv:1311.2901v3 [cs.CV], November 28, 2013, which is incorporated herein by reference in its entirety. With such a process, elements that make an image anomalous (e.g., reasons for anomaly, etc.) may be identified. In various embodiments, if the training images include an augmented set of images, elements may be extracted from the augmented images, and at least some elements may be defined, at least in part, by the augmentation type used (e.g., mean image intensity, standard deviation of intensity, grayscale histogram, dominant frequency (FFT), average contrast measure, image noise measure, etc.). In some cases, a relatively simple limit-based anomaly detector may be created to consider elements within the range of the extended image set as "normal" and elements outside these ranges as "anomalous" (e.g., may also allow for relatively simple identification of (i.e., reasons for) classifying an image as anomalous). Method 1000 then proceeds to 1040.

[0078] At 1040, run mode images of the workpiece are acquired during the run mode, for example using the same camera system used to acquire the training images at 1010. Method 1000 then proceeds to 1050.

[0079] At 1050, a number of features are determined from the run mode image. For example, the features from the run mode image may include a value corresponding to an average brightness of the run mode image and a value corresponding to a defocus level. Method 1000 then proceeds to 1060.

[0080] At 1060, the run mode image acquired at 1040 is classified as non-anomalous (e.g., acceptable for accurate defect detection) or anomalous (e.g., unacceptable for accurate defect detection) using the anomaly detection classification index determined at 1030 based on the multiple elements of the run mode image determined at 1050. For example, if it is determined at 1050 that the average luminance of the run mode image has a value characteristic of an anomaly and the run mode image has a focus corresponding to the baseline DOF, then the run mode image is classified as anomalous based on the anomaly detection classification index determined at 1030. As an example, if the run mode image is classified as anomalous (e.g., unacceptable for defect detection) or non-anomalous, then the image may be flagged accordingly (e.g., the condition for flagging may be stored or otherwise indicated in association with the image data, etc.). Method 1000 then proceeds to 1070.

[0081] At 1070, a determination is made as to whether the run mode image has been classified as anomalous, e.g., whether a file containing image data corresponding to the run mode image has been flagged as anomalous or non-anomalous to indicate a corresponding anomalous or non-anomalous classification. If it is determined that the run mode image has been classified as anomalous, method 1000 proceeds to 1080. If it is not determined that the run mode image has been classified as anomalous, method 1000 proceeds to 1090.

[0082] At 1080, a process for the anomalous image may be performed, such as by performing a specified action (e.g., alerting a user and / or adjusting the system). In various embodiments, the reason for the anomaly may be identified (e.g., the element that is anomalous or the element that causes the image to be anomalous may be identified), and in some cases, a message or other instruction may be provided to the user as to why the image is anomalous and / or the system may be adjusted accordingly. In various embodiments, adjusting the system may include adjusting imaging conditions, lighting conditions, focusing conditions, and / or workpiece conditions. Examples of such adjustments may include adjusting the brightness of the light source, adjusting the position of the objective lens, and / or adjusting the position of the workpiece, etc.

[0083] Optionally, at 1190 (or may be performed at other times in method 1000 and / or in parallel with some portions of method 1000), a defect detector is used to determine whether the run mode image is classified as a defective image. For example, image data corresponding to the run mode image may be input to the defect detector (e.g., as part of defect detector 140dp). In one exemplary embodiment, the defect detector may be based on multi-label image classification with a neural network, which is implemented by defect detector 140dp and trained to detect defects using a training image set. In response, the defect detector may identify a number of pixels in the workpiece image that have characteristics of a defect, and therefore determine to classify the run mode image as a defective image. Method 1000 then proceeds to 1095.

[0084] At 1095, a determination is made as to whether to acquire additional run mode images. For example, a determination is made to acquire additional run mode images in response to multiple images continuing to be acquired and / or a user pressing a button or selecting an icon using a graphical user interface to indicate inspection of another workpiece image. If additional run mode images are to be acquired, method 1000 proceeds to 1040. If additional run mode images are not to be acquired, method 1000 ends.

[0085] While preferred embodiments of the present disclosure have been shown and described, numerous variations in the arrangement of elements and sequences of operation shown and described will be apparent to those skilled in the art based on the present disclosure. Various alternative configurations can be used to implement the principles disclosed herein.

[0086] Aspects of the above-described embodiments can be modified as necessary to provide further embodiments using concepts from various patents and applications herein. These and other modifications can be made to the embodiments in light of the description set forth above. In general, in the following claims, the terms used should not be construed as limiting the scope of the claims to the specific embodiments disclosed in the specification and claims, but rather as embracing all possible embodiments in addition to the full scope of equivalents to which such claims are entitled.

Claims

1. A light source; a lens that receives image light emanating from a surface of a workpiece illuminated by the light source and transmits the image light along an imaging optical path; a camera that receives imaging light transmitted along the imaging optical path and provides an image of the workpiece; one or more processors; a memory coupled to the one or more processors and configured to store program instructions; the program instructions, when executed by the one or more processors, at least acquiring at least some of a plurality of training images of a given workpiece for use in training a defect detector; determining a plurality of training image elements identified by image analysis of each of the plurality of training images; determining a plurality of anomaly-detection classifiers based at least in part on the determined training image elements; acquiring a run mode image of a workpiece using the camera; image analyzing the run mode image of the workpiece to determine a plurality of identified features; determining whether the run mode image of the workpiece is classified as anomalous using the anomaly detection classifier based at least in part on the features determined from the run mode image of the workpiece; and a workpiece inspection and defect detection system that causes the one or more processors to perform

2. the program instructions are executed by the one or more processors; If the run mode image of the workpiece is determined to be classified as non-anomalous using the anomaly detection classifier, analyzing the run mode image using the defect detection unit to determine whether the run mode image of the workpiece is classified as a defect image that includes a defect in the workpiece; The system of claim 1 , further comprising the one or more processors:

3. The system of claim 1 , wherein the first run-mode image of the workpiece is classified as anomalous by utilization of the anomaly detection classifier.

4. The program instructions, when executed by the one or more processors, generating a message indicating a problem with at least one of imaging conditions, lighting conditions, or workpiece conditions based on the first run mode image of the workpiece being classified as abnormal; The system of claim 3 , further comprising the one or more processors executing:

5. The program instructions, when executed by the one or more processors, determining why the first run mode image of the workpiece is classified as anomalous; The system of claim 3 , further comprising the one or more processors executing:

6. The program instructions, when executed by the one or more processors, generating a message indicating the reason why the first run mode image of the workpiece is classified as abnormal; The system of claim 5 , further comprising the one or more processors executing:

7. The program instructions, when executed by the one or more processors, adjusting at least one of a brightness of the light source, or a position of the light source, or a position of the lens or lens element relative to at least one of the camera or workpiece after determining the reason the first run mode image is classified as anomalous; The system of claim 5 , further comprising the one or more processors executing:

8. 1. A method of operating a workpiece inspection and defect detection system, comprising: acquiring a plurality of training images using a light source, a lens, and a camera of the workpiece inspection and defect detection system to train a defect detector; determining a plurality of training image elements from the training image; determining a plurality of anomaly-detection classifiers based at least in part on the training image elements; acquiring a run mode image of the workpiece; determining a plurality of features from the run mode image of the workpiece; determining whether the run mode image of the workpiece is classified as anomalous using the anomaly detection classifier based at least in part on the features determined from the run mode image of the workpiece; and The method includes:

9. determining whether the run mode image of the workpiece is classified as non-anomalous using the anomaly detection classifier; and analyzing the run mode image using the defect detection unit to determine whether the run mode image of the workpiece is classified as a defect image that includes a defect in the workpiece; The method of claim 8 further comprising:

10. The method of claim 8 , wherein the first run-mode image of the workpiece is classified as anomalous by utilization of the anomaly detection classifier.

11. 11. The method of claim 10, further comprising generating a message indicating a problem with imaging conditions, lighting conditions, or workpiece conditions based on the first run mode image of the workpiece being classified as abnormal.

12. The method of claim 10 , further comprising determining a reason why the first execution mode image is classified as abnormal.

13. The method of claim 12 , further comprising generating a message indicating the reason the first run mode image of the workpiece is classified as abnormal.

14. 13. The method of claim 12, further comprising, after the determination of the reason the first execution mode image is classified as anomalous, adjusting at least one of a brightness of the light source, or a position of the light source, or a position of the lens or lens element relative to at least one of the camera or workpiece.

15. A non-transitory computer-readable medium storing program instructions that, when executed by one or more processors, at least acquiring a plurality of training images using a light source, a lens, and a camera of the workpiece inspection and defect detection system to train a defect detector; determining a plurality of training image elements from the plurality of training images; determining a plurality of anomaly-detection classifiers based at least in part on the determined training image elements; acquiring a run mode image of a workpiece using the camera; determining a plurality of features from the run mode image of the workpiece; determining whether the run mode image of the workpiece is classified as anomalous using the anomaly detection classifier based at least in part on the determined features from the run mode image of the workpiece; and A non-transitory computer-readable medium for causing the one or more processors to execute.

16. The program instructions, when executed by the one or more processors, determining whether the run mode image of the workpiece is classified as non-anomalous using the anomaly detection classifier; and analyzing the run mode image with a defect detection unit to determine whether the run mode image is a defect image that includes a defect in the workpiece; 20. The non-transitory computer-readable medium of claim 15, further causing the one or more processors to execute:

17. The non-transitory computer readable medium of claim 15 , wherein the first run-mode image of the workpiece is classified as anomalous through utilization of the anomaly detection classifier.

18. The program instructions, when executed by the one or more processors, generating a message indicating a problem with at least one of imaging conditions, lighting conditions, or workpiece conditions based on the first run mode image of the workpiece being classified as abnormal; 20. The non-transitory computer-readable medium of claim 17, further causing the one or more processors to execute:

19. The program instructions, when executed by the one or more processors, determining why the first run mode image of the workpiece is classified as anomalous; and generating a message indicating the reason the first run mode image of the workpiece is classified as abnormal; and 20. The non-transitory computer-readable medium of claim 17, further causing the one or more processors to execute:

20. The program instructions, when executed by the one or more processors, adjusting at least one of a brightness of the light source, or a position of the light source, or a position of the lens or lens element relative to at least one of the camera or the workpiece after determining the reason the first run mode image of the workpiece is classified as anomalous; 20. The non-transitory computer-readable medium of claim 19, further causing the one or more processors to execute:

21. Determining the plurality of anomaly detection classifiers based at least in part on the training image elements includes: generating a synthetic data set in which a plurality of said training image elements are modified to simulate anomalous images and are not modified to simulate non-anomalous images; training a two-class classifier capable of distinguishing between anomalous and non-anomalous images in the synthetic dataset; 20. The non-transitory computer readable medium of claim 15, comprising:

Images