Color channel-based workpiece inspection and defect detection system

Abstract

To solve various types of problems in an inspection of a workpiece due to, for example, a change in a type and / or surface of a material of a workpiece of an inspection object, a change in an inspection condition, and solve such problems in a specific type inspection operation.SOLUTION: A workpiece inspection and defect detection system includes a light source configuration, a lens configuration, and a camera configuration for imaging workpieces. The system acquires training and run mode workpiece images for acquiring corresponding sets of training and run mode workpiece image data. Each set of image data includes at least first and second color channel workpiece image data corresponding to first and second color channels. The defect detection portion is trained based at least in part on the image data, and is utilized to perform analysis to determine defect images that include workpieces with defects.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to a work inspection system, and more particularly to a precision measurement system for inspecting workpieces and detecting defects.

Background Art

[0002] An image of a workpiece to be inspected can be obtained using a specific precision measurement system such as a machine vision inspection system (or, abbreviated as a "vision system"). Such systems can be used for various types of applications (e.g., general workpiece inspection, measurement applications for determining accurate dimensional measurements of workpieces, etc.). Some such systems generally include a computer, a camera, and an optical system. In certain configurations, a moving mechanism (e.g., a precision stage, a conveyor, etc.) that moves to enable traversal and inspection of the workpiece may be included. One exemplary prior art machine vision inspection system is the QUICK VISION® series of PC-based vision systems and QVPAK® software available from Mitutoyo America Corporation (MAC) in Aurora, Illinois. The features and operation of the QUICK VISION® series of vision systems and QVPAK® software are generally described, for example, in the "QVPAK 3D CNC Vision Measuring Machine User’s Guide" published in January 2003. This is hereby incorporated by reference in its entirety. This type of system uses a microscope-type optical system and moves the stage to provide an inspection image of the workpiece.

Summary of the Invention

Problems to be Solved by the Invention

[0003] Such measurement systems typically face various challenges in workpiece inspection (e.g., due to changes in the material and / or surface of the workpiece being inspected, changes in inspection conditions, etc.). A system that can improve these problems in specific types of inspection operations (e.g., to improve workpiece defect detection and / or the accuracy of defect detection) is desirable. [Means for solving the problem]

[0004] This summary is provided in a simplified form to introduce a selection of concepts that will be further explained in the following "Modes for Carrying Out the Invention." This summary is not intended to identify the essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

[0005] A workpiece inspection and defect detection system is provided, comprising a light source configuration, a lens configuration, a camera configuration that receives imaging light transmitted along the imaging optical path to provide a workpiece image, one or more processors, and memory. The light source configuration is configured to provide multiple color channels, each corresponding to a different color. The lens configuration receives imaging light generated from the surface of a workpiece illuminated by the light source configuration and transmits the imaging light along the imaging optical path. The camera configuration receives the imaging light transmitted along the imaging optical path to provide a workpiece image.

[0006] The memory is coupled to one or more processors and stores program instructions. When the program instructions are executed by one or more processors, they cause one or more processors to perform the following: acquire training mode work images for acquiring multiple sets of training mode work image data, each set containing a first color channel training mode work image data corresponding to a first color channel and a second color channel training mode work image data corresponding to a second color channel; train a defect detection unit based at least partially on the multiple sets of training mode work image data; acquire run mode work images for acquiring multiple sets of run mode work image data, each set containing a first color channel run mode work image data corresponding to a first color channel and a second color channel run mode work image data corresponding to a second color channel; and use the trained defect detection unit to perform analysis based at least partially on the multiple sets of run mode work image data to determine defective images containing defective work.

[0007] In various implementations, training a defect detection unit based at least partially on multiple sets of training mode work image data includes generating corresponding training mode composite image data used to train the defect detection unit for each set of training mode work image data using corresponding first and second color channel training mode work image data, and using the trained defect detection unit to perform analysis based at least partially on multiple sets of run mode work image data to determine defective images containing defective work, includes generating corresponding run mode composite image data to be analyzed by the trained defect detection unit for each set of run mode work image data using corresponding first and second color channel run mode work image data.

[0008] In various implementations, for each set of training mode work image data, multiple ratios are determined between the corresponding first and second color channel training mode work image data, and for each set of run mode work image data, multiple ratios are determined between the corresponding first and second color channel run mode work image data. In various implementations, the image data of each image described herein includes multiple pixel values. Each pixel value (including, for example, a luminance value) corresponds to a pixel in the image. Pixels at the same position, or imaging of the same surface point on a workpiece in different images, can be designated as corresponding pixels between different images by their corresponding pixel values ​​(for example, different corresponding pixel values ​​may originate from different color light used to illuminate the workpiece in each image). In various implementations, determining multiple ratios between corresponding first and second color channel training mode work image data includes determining the ratio of each pixel value in the corresponding first color channel training mode work image data to the corresponding pixel value in the corresponding second color channel training mode work image data, and determining multiple ratios between corresponding first and second color channel run mode work image data includes determining the ratio of each pixel value in the corresponding first color channel run mode work image data to the corresponding pixel value in the corresponding second color channel run mode work image data.In various implementations, training a defect detection unit based at least partially on multiple sets of training mode work image data includes determining the corresponding pixel values ​​of the corresponding training mode composite image data used to train the defect detection unit for each set of training mode work image data by utilizing the ratio of pixel values ​​of the corresponding first and second color channel training mode work image data, and using the trained defect detection unit to perform analysis based at least partially on multiple sets of run mode work image data to determine a defective image containing a defective work, includes determining the corresponding pixel values ​​of the corresponding run mode composite image data analyzed by the trained defect detection unit for each set of run mode work image data by utilizing the ratio of pixel values ​​of the corresponding first and second color channel run mode work image data.

[0009] In various implementations, the light source configuration includes a first light source that provides a first color light for a first color channel and a second light source that provides a second color light for a second color channel, with the first and second color light corresponding to different wavelengths of light. In various implementations, the first light source is a first light-emitting diode, and the second light source is a second light-emitting diode.

[0010] In various implementations, at least a portion of the workpiece being inspected contains the first and second materials, and the presence of the exposed second material in the image corresponds to a defect that can be detected by the trained defect detection unit.

[0011] In various implementations, at least a portion of the workpiece being inspected contains first and second materials, and the first and second materials have first and second reflectance profiles such that, for a first color light corresponding to a first color channel, the reflectance of the first material is greater than that of the second material, and for a second color light corresponding to a second color channel, the reflectance of the second material is greater than or equal to that of the first material. Components of the light source configuration may be selected to provide the wavelengths of the first and second color light based at least partially on the reflectance profiles of the first and second materials. In various implementations, the light source configuration is configured to provide the first and second color light and also to provide a third color light for a third color channel. Based on the reflectance profiles of the first and second materials, the first and second color light is selected to be used, while the third color light is not selected to be used, for inspecting a workpiece containing the first and second materials. In various implementations, a third color light is selected to be used to inspect a workpiece having the third material, based on the reflectance profile of the third material, which is different from the reflectance profiles of the first and second materials.

[0012] In various implementations, for each set of training mode work image data, the corresponding first and second color channel training mode work image data are generated by the respective first and second camera image planes, and for each set of run mode work image data, the corresponding first and second color channel run mode work image data are generated by the respective first and second camera image planes.

[0013] In various implementations, when a program instruction is executed by one or more processors, it causes one or more processors to further perform one or more measurement operations using one or more defective images.

[0014] Various implementations may provide a method for operating a work inspection and defect detection system (for example, a computer implementation method operated under the control of one or more computing systems consisting of executable instructions). This method includes: acquiring training mode work images to obtain multiple sets of training mode work image data, each set comprising a first color channel training mode work image data corresponding to a first color channel and a second color channel training mode work image data corresponding to a second color channel; training a defect detection unit based at least partially on multiple sets of training mode work image data; acquiring run mode work images to obtain multiple sets of run mode work image data, each set comprising a first color channel run mode work image data corresponding to a first color channel and a second color channel run mode work image data corresponding to a second color channel; and using the trained defect detection unit to perform an analysis based at least partially on multiple sets of run mode work image data to determine defective images containing defective workpieces.

[0015] In various implementations, a work inspection and defect detection system may be provided that performs the following: acquiring training mode work images for acquiring multiple sets of training mode work image data, each set containing a first color channel training mode work image data corresponding to a first color channel and a second color channel training mode work image data corresponding to a second color channel; training a defect detection unit based at least partially on multiple sets of training mode work image data; acquiring run mode work images for acquiring multiple sets of run mode work image data, each set containing a first color channel run mode work image data corresponding to a first color channel and a second color channel run mode work image data corresponding to a second color channel; and using the trained defect detection unit to perform analysis based at least partially on multiple sets of run mode work image data to determine defective images containing defective workpieces. [Brief explanation of the drawing]

[0016] Many of the above aspects and associated advantages of the present invention will be more readily understood by referring to the following detailed description, when interpreted in conjunction with the accompanying drawings.

[0017] [Figure 1] This figure shows various components of one implementation form of a workpiece inspection and defect detection system. [Figure 2] This is a block diagram of the control system and vision component sections of a workpiece inspection and defect detection system, similar to the workpiece inspection and defect detection system in Figure 1, and including certain features disclosed herein. [Figure 3A] This is an example of an image of a defect-free portion of a workpiece, which can be obtained using a workpiece inspection and defect detection system similar to the one shown in Figure 1. [Figure 3B]This is an example of an image of a defect-free portion of a workpiece, which can be obtained using a workpiece inspection and defect detection system similar to the one shown in Figure 1. [Figure 3C] This is an example of an image of a defect-free portion of a workpiece, which can be obtained using a workpiece inspection and defect detection system similar to the one shown in Figure 1. [Figure 3D] This is an example of an image of a defect-free portion of a workpiece, which can be obtained using a workpiece inspection and defect detection system similar to the one shown in Figure 1. [Figure 3E] This is an example of an image of a defect-free portion of a workpiece, which can be obtained using a workpiece inspection and defect detection system similar to the one shown in Figure 1. [Figure 3F] This is an example of an image of a defect-free portion of a workpiece, which can be obtained using a workpiece inspection and defect detection system similar to the one shown in Figure 1. [Figure 4A] This is an example of an image of a defective portion of a workpiece, which can be obtained using a workpiece inspection and defect detection system similar to the one shown in Figure 1. [Figure 4B] This is an example of an image of a defective portion of a workpiece, which can be obtained using a workpiece inspection and defect detection system similar to the one shown in Figure 1. [Figure 4C] This is an example of an image of a defective portion of a workpiece, which can be obtained using a workpiece inspection and defect detection system similar to the one shown in Figure 1. [Figure 4D] This is an example of an image of a defective portion of a workpiece, which can be obtained using a workpiece inspection and defect detection system similar to the one shown in Figure 1. [Figure 4E] This is an example of an image of a defective portion of a workpiece, which can be obtained using a workpiece inspection and defect detection system similar to the one shown in Figure 1. [Figure 5A]An example of an image of a similar part of a work without defects that can be obtained using a work inspection and defect detection system similar to the work inspection and defect detection system of FIG. 1. [Figure 5B] An example of an image of a similar part of a work with defects that can be obtained using a work inspection and defect detection system similar to the work inspection and defect detection system of FIG. 1. [Figure 5C] An example of an image of a similar part of a work with defects that can be obtained using a work inspection and defect detection system similar to the work inspection and defect detection system of FIG. 1. [Figure 5D] An example of an image of a similar part of a work with defects that can be obtained using a work inspection and defect detection system similar to the work inspection and defect detection system of FIG. 1. [Figure 6] A diagram showing the use of one or more video tools for performing measurement operations on an image of a work containing defects. [Figure 7A] An example of an image of a part of a work containing defects that can be obtained using a work inspection and defect detection system similar to the work inspection and defect detection system of FIG. 1. [Figure 7B] An example of an image of a part of a work containing defects that can be obtained using a work inspection and defect detection system similar to the work inspection and defect detection system of FIG. 1. [Figure 7C] An example of an image of a part of a work containing defects that can be obtained using a work inspection and defect detection system similar to the work inspection and defect detection system of FIG. 1. [Figure 7D] An example of a composite image generated based on the images of FIGS. 7A to 7C. [Figure 7E] An example of an image of a defect mask. [Figure 8A] An example of an image of a part of a work containing defects that can be obtained using a work inspection and defect detection system similar to the work inspection and defect detection system of FIG. 1. [Figure 8B]This is an example of an image of a defective portion of a workpiece, which can be obtained using a workpiece inspection and defect detection system similar to the one shown in Figure 1. [Figure 8C] This is an example of an image of a defective portion of a workpiece, which can be obtained using a workpiece inspection and defect detection system similar to the one shown in Figure 1. [Figure 8D] This is an example of a composite image generated based on the images in Figures 8A to 8C. [Figure 8E] This is an example of a defective mask image. [Figure 8F] This is an example of a composite image generated based on the images in Figures 8A and 8C. [Figure 9] This figure shows the use of one or more video tools to perform measurement operations on images of a workpiece containing defects. [Figure 10] Figure 1 shows a flowchart illustrating a method for operating a workpiece inspection and defect detection system similar to the one shown in Figure 1. [Modes for carrying out the invention]

[0018] Figure 1 is a block diagram of an exemplary workpiece inspection and defect detection system 10. In the example of Figure 1, the workpiece inspection and defect detection system 10 may include, be part of, or, in some cases, be referred to as the machine vision inspection system instead. The workpiece inspection and defect detection system 10 includes a vision inspection machine 12 operably connected to a control computer system 14 for exchanging data and control signals. The control computer system 14 is further operably connected to a monitor or display 16, a printer 18, a joystick 22, a keyboard 24, and a mouse 26 for exchanging data and control signals. The monitor or display 16 may display a user interface suitable for controlling and / or programming the operation of the workpiece inspection and defect detection system 10. In various exemplary implementations, it will be understood that touchscreen tablets and / or similar devices, etc., may replace or provide redundancy for any or all of the functions of elements 14, 16, 22, 24, and 26.

[0019] Those skilled in the art will understand that the control computer system 14 and / or other control systems described herein can generally be implemented using any suitable computing system or device, such as a distributed computing environment or a network computing environment. Such a computing system or device may include one or more general-purpose or special-purpose processors (e.g., non-custom or custom devices) that run software to perform the functions described herein. The software may be stored in memory such as random-access memory (RAM), read-only memory (ROM), flash memory, or a combination of such components. The software may also be stored in one or more storage devices, such as optical-based disks, flash memory devices, or any other type of non-volatile storage medium for storing data. The software may include one or more program modules, such as routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. In a distributed computing environment, the functions of the program modules may be combined or distributed across multiple computing systems or devices and accessed via service calls in either a wired or wireless configuration.

[0020] The vision inspection machine 12 includes a movable work stage 32 and an optical imaging system 34 having a lens configuration which may include a zoom lens or an interchangeable objective lens. The zoom lens or interchangeable objective lens generally provides various magnifications to the image provided by the optical imaging system 34. Various exemplary implementations of the vision inspection machine 12 are also described in U.S. Patents 7,454,053, 7,324,682, 8,111,905, and 8,111,938. Each of these patents is incorporated herein by reference in its entirety.

[0021] As some examples of possible system components, the camera configuration of the optical imaging system 34 of the workpiece inspection and defect detection system 10 may include a monochrome camera, in which different colored light-emitting diodes (LEDs) are used for different color channels. In other implementations, a red-green-blue (RGB) camera or a two- or three-sensor camera capable of distinguishing / producing images of different wavelengths may be used. In such cases, different color channels can be implemented using a white light source (e.g., together with a beam splitter and narrowband color filter) rather than illuminating the workpiece at different times using different LEDs. In some implementations, it may be desirable to generate images more quickly. For this, a camera with multiple image sensors may be advantageous (e.g., rather than capturing each color image at different times). In various implementations, the workpiece inspection and defect detection system may include, utilize, and / or receive data from multiple camera configurations and / or other components described herein (e.g., as part of a single or multiple vision inspection machine or its components and / or computer system, etc., as part of the workpiece inspection and defect detection system).

[0022] Figure 2 is a block diagram of the control system unit 120 and vision component unit 200 of a work inspection and defect detection system 100, similar to the work inspection and defect detection system of Figure 1, and including certain features disclosed herein. The control system unit 120 is used to control the vision component unit 200, as will be described in more detail below. The control system unit 120 may be configured to exchange data and control signals with the vision component unit 200. The vision component unit 200 includes an optical assembly unit 205, light sources 220, 230, 240, 300, and a work stage 210, the work stage 210 having a transparent central portion 212. The work stage 210 is controllably movable along the x and y axes in a plane substantially parallel to the surface of the stage on which a workpiece 20 can be placed.

[0023] The optical assembly 205 includes a camera configuration 260 (e.g., including one or more cameras and / or various camera components) and a lens configuration with interchangeable objective lenses 250. In some implementations, the lens configuration of the optical assembly 205 may optionally include a variable focal length (VFL) lens (e.g., a variable acoustic distributed refractive index (TAG) as disclosed in U.S. Patent No. 9,143,674, which is incorporated herein by reference in its entirety.

[0024] In various exemplary implementations, the lens configuration of the optical assembly 205 may further include a turret lens assembly 280 having lenses 286 and 288. Instead of the turret lens assembly, various exemplary implementations may include fixed or manually interchangeable magnification-changing lenses, or zoom lens configurations. In various exemplary implementations, an interchangeable objective lens 250 can be selected from a set of fixed-magnification objective lenses included as part of the variable-magnification lens section (for example, a set of objective lenses corresponding to magnifications such as 0.5x, 1x, 2x or 2.5x, 5x, 10x, 20x or 25x, 50x, 100x, etc.).

[0025] The optical assembly unit 205 is controllably movable along the z-axis, which is substantially perpendicular to the x and y axes, by using a controllable motor 294. The controllable motor 294 drives an actuator to move the optical assembly unit 205 along the z-axis to change the focus of the image on the workpiece 20. The controllable motor 294 is connected to the input / output interface 130 via a signal line 296 to change the image focus within a specific range. The workpiece 20 can be placed on a work stage 210. The work stage 210 can be controlled to move relative to the optical assembly unit 205 so that the field of view of the interchangeable objective lens 250 moves between positions on the workpiece 20 and / or between multiple workpieces 20.

[0026] One or more of the stage light source 220, coaxial light source 230, and surface light source 240 (e.g., a ring light) can emit light sources 222, 232, and / or 242, respectively, to illuminate one or more workpieces 20. For example, during image exposure, the coaxial light source 230 may emit light source 232 along a path including a beam splitter 290 (e.g., a partial mirror). The light source 232 is reflected or transmitted as workpiece light 255, and the workpiece light used for imaging passes through interchangeable objective lenses 250 and turret lens assembly 280 and is collected by a camera configuration 260 (e.g., including a camera). The workpiece image exposure, including an image of the workpiece 20, is captured by the camera configuration 260 and output to the control system unit 120 via a signal line 262.

[0027] In various implementations, one or more of the light sources 220, 230, 240, or 300 may contain multiple light sources (for example, as part of a light source configuration). For example, in one implementation, light source 230 may contain two or three light sources (for example, two or three LEDs such as red, green, and / or blue LEDs), in which case the light source 232 may contain light from any of the currently lit light sources. As will be described in more detail below, as part of such a light source configuration, for example, a first light source may correspond to a first color channel (for example, a blue LED corresponding to the blue color channel), and a second light source may correspond to a second color channel (for example, a red LED corresponding to the red color channel). Different color channels can be used to generate different color channel image data, etc.

[0028] Various light sources (e.g., light sources 220, 230, 240, 300) can be connected to the lighting control interface 133 of the control system unit 120 via associated signal lines (e.g., buses 221, 231, 241, 331, respectively). The control system unit 120 can control the turret lens assembly 280 to rotate along axis 284, and select the turret lens via the signal lines or bus 281 to change the image magnification.

[0029] As shown in Figure 2, in various exemplary implementations, the control system unit 120 includes a controller 125, an input / output interface 130, a memory 140, a work program generator and executor 170, and a power supply unit 190. Each of these components and the additional components described below can be interconnected by one or more data / control buses and / or application programming interfaces, or by direct connections between various elements. The input / output interface 130 includes an imaging control interface 131, a motion control interface 132, an illumination control interface 133, and a lens control interface 134.

[0030] The illumination control interface 133 may include illumination control elements 133a to 133n that control, for example, the selection, power supply, and / or on / off switches of various corresponding light sources of the work inspection and defect detection system 100. In the illustrated embodiment, the illumination control interface 133 also includes an illumination control element 133sip that can function in conjunction with a Structured Illumination Pattern (SIP) generation unit 300 to provide structured illumination during image acquisition. In various implementations, the SIP generation unit 300 can output a predicted pattern to be input to a beam splitter 290, which provides SIP-structured light 232' directed as coaxial light passing through the objective lens 250 to illuminate the field of view, and the SIP-structured light 232' is reflected from the workpiece 20 as workpiece light 255'.

[0031] Memory 140 may include an image file memory section 141, a defect detection section 140dp, a work program memory section 142 (which may include one or more part programs, etc.), and a video tool section 143. The video tool section 143 includes a video tool section 143a and other video tool sections (e.g., 143n) that determine the GUIs, image processing operations, etc., of the corresponding video tools, and a region of interest (ROI) generator 143roi that supports automatic, semi-automatic, and / or manual operations that define various ROIs that can be operated in the various video tools included in the video tool section 143. Examples of video tool operations for such edge feature localization and other work feature inspection operations are described in detail in some of the previously incorporated references and in U.S. Patent No. 7,627,162, which is incorporated herein by reference in its entirety.

[0032] The video tool unit 143 also includes an autofocus video tool 143af that determines a GUI for focus height measurement operations, image processing operations, and so on. In various exemplary implementations, the autofocus video tool 143af may further include a high-speed focus height tool that can be used to measure focus height at high speed using hardware, as described in detail in U.S. Patent No. 9,143,674, which is incorporated herein by reference in its entirety. In various exemplary implementations, the high-speed focus height tool may be a special mode of the autofocus video tool 143af, which otherwise operates according to the conventional methods of the autofocus video tool. Alternatively, the operation of the autofocus video tool 143af may include only the operation of the high-speed focus height tool. High-speed autofocus and / or focus positioning of an image region or region of interest may be based on analyzing the image according to known methods and determining quantitative contrast metrics corresponding to various regions. For example, such methods are disclosed in U.S. Patents No. 8,111,905, No. 7,570,795, and No. 7,030,351. These are each incorporated herein in their entirety by reference.

[0033] In the context of this disclosure, and as will be known to those skilled in the art, the term “video tool” typically refers to a set of relatively complex automated or programmed operations that a machine vision user can implement through a relatively simple user interface. For example, a video tool may include a set of complex pre-programmed image processing operations and calculations that are applied and customized in a particular instance by adjusting several variables or parameters that affect the operations and calculations. In addition to the underlying operations and calculations, a video tool has a user interface that allows the user to adjust these parameters for a particular instance of the video tool. While visible user interface features may also be referred to as video tools, the underlying operations are implicitly included.

[0034] One or more display devices 136 (e.g., display 16 in Figure 1) and one or more input devices 138 (e.g., joystick 22, keyboard 24, and mouse 26 in Figure 1) can be connected to the input / output interface 130. The display devices 136 and input devices 138 may be used to display a user interface. This user interface may include various graphical user interface (GUI) features that can be used to perform inspection operations and / or create and / or modify part programs, view images captured by the camera configuration 260, and / or directly control the vision component unit 200.

[0035] In various exemplary implementations, when a user creates a part program for a workpiece 20 using the workpiece inspection and defect detection system 100, the user generates part program instructions by operating the workpiece inspection and defect detection system 100 in training mode to provide a desired image acquisition training sequence. For example, the training sequence may include placing a specific workpiece feature of a representative workpiece in the field of view (FOV), setting the light level, focusing or autofocusing, acquiring an image, and providing an inspection training sequence to be applied to the image (e.g., using an instance of one of the video tools on the workpiece feature). The training mode operates so that the sequence is captured or recorded and converted into corresponding part program instructions. When the part program is executed, these instructions cause the workpiece inspection and defect detection system to reproduce the trained image acquisition, and the inspection operation automatically inspects a specific workpiece feature on the run-mode workpiece that matches the representative workpiece used when creating the part program (i.e., the corresponding feature at the corresponding location). In various exemplary implementations, specific types of training modes may be used further or instead (for example, a training mode for training a defect detection unit to detect defects, a training mode for training an anomaly detector unit to detect anomaly images unacceptable to the defect detection process, etc.). In various implementations, training modes may be further or instead referred to as learning modes.

[0036] The video tool unit 143 also includes a Z-height measurement tool unit 143z. The Z-height measurement tool unit 143z provides various operations and features related to Z-height measurement operations. In one implementation, the Z-height measurement tool unit 143z may include a Z-height tool 143zt. The Z-height tool 143zt may include, for example, an autofocus tool 143af and a multipoint autofocus tool 143maf. The Z-height tool 143zt, together with a Z-height tool configured in a mode to determine the optimal focus height and / or Z-height measurement, can influence certain aspects of image stack acquisition and associated illumination operations. Generally, the Z-height measurement tool unit 143z can perform at least some operations, similar to known Z-height measurement tools, such as performing operations in training mode and / or run mode or other modes to generate all or part of a focus curve and find its peak as the optimal focus position. For example, certain known operations of the Z-height measurement tool are described in U.S. Patent No. 10,520,301, which is incorporated herein by reference in its entirety.

[0037] The defect detection unit 140dp performs various defect detection operations, as will be described in more detail below. In various implementations, the defect detection unit 140dp utilizes a model that requires training data (e.g., training images). For example, the defect detection unit 140dp can be trained using a set of training images taken using specified imaging, lighting, and work conditions. In various exemplary implementations, the model may be a supervised model (e.g., artificial intelligence (AI)). The defect detection unit 140dp trains a classification model by processing image data corresponding to labeled defect images (e.g., labeled by a user and / or an automated process). In various implementations, the classification model may be an AI classification model.

[0038] In various implementations, the defect detection unit 140dp can implement a defect detection process that runs in conjunction with the measurement process. In various implementations, it may be desirable to include the defect detection unit 140dp in a work inspection and defect detection system similar to the work inspection and defect detection system 100 in Figure 1, which is configured to perform the measurement process. This is because the system can generate image data that is input to the defect detection process implemented by the defect detection unit 140dp. Thus, a single machine is configured to perform both the measurement process and the defect detection process, which can offer advantages over conventional measurement systems. For example, if a defect is detected in a workpiece while the defect detection process is running, there is no reason to perform the measurement process on that workpiece, thus saving time. More specifically, if a defect is detected in a workpiece while the defect detection process is running, there is obviously no need to measure the defective part. Therefore, it may be advantageous to run the defect detection process before starting the measurement process.

[0039] Furthermore, some defects may require further measurement or inspection to determine additional defect parameters. For example, 2D images allow for immediate recognition of suspected defects and quick confirmation of their XY location and approximate XY region. If the 3D characteristics of a potential defect are important, the defect detection unit 140dp can perform additional processing (e.g., measurement operations) to determine whether the potential defect is an actual defect. For example, if a scratch on the surface of a workpiece must be deeper than a certain threshold to be considered a defect, the defect detection unit 140dp may use a more time-consuming 3D point cloud of the affected area (e.g., using the Z height measurement tool unit 143z) to learn whether the scratch depth is sufficient to reject the part. In various implementations, different actions may be taken as a result of the initial defect classification (for example, they may be programmed to occur automatically), such as (1) continuing the standard measurement process, (2) stopping or pausing the defect detection process and performing a measurement process that includes a more useful measure of potential defects (e.g., touch probe measurement results such as 3D, different lighting, surface roughness, etc.), (3) sending the workpiece to scrap (e.g., discarding or recycling the workpiece), (4) sending the workpiece for additional human inspection, or (5) providing feedback to the production line indicating that there is a problem with the machine or process.

[0040] In various exemplary implementations, the defect detection unit 140dp in memory 140 stores model data and program instructions for various tools and algorithms of the defect detection system that can be used to infer whether various types of defects are present in the workpiece image (i.e., whether a portion of the workpiece surface contained in the image is defective). While the workpiece inspection and defect detection system is operating in training mode or learning mode, the defect detection system trains the defect detection unit 140dp using image data from a set of training images, including defective and non-defective images. The set of training images is captured using specified imaging, lighting, and workpiece conditions. After initial training of the defect detection unit 140dp, it operates in run mode (for example, using the same specified imaging, lighting, and workpiece conditions) to infer whether defects are present in new, previously unseen workpiece images and classify each workpiece image accordingly as either a defective or non-defective image.

[0041] As will be explained in more detail below, Figures 3A-3F, 4A-4E, 5A-5D, 7A-7E, and 8A-8F show some examples of workpiece images that may be acquired by the workpiece inspection and defect detection system 100, and in some implementations, the defect detection unit 140dp may be trained using the image data (e.g., as training images and / or test images) during the training mode (and / or some of the images may be examples of workpiece images that are later acquired by the workpiece inspection and defect detection system 100 during the run mode and can be analyzed by the trained defect detection unit 140dp).

[0042] In particular, the workpiece image examples in Figures 3A-3F and 4A-4E are from different sections (i.e., at different XY positions) on a machined aluminum plate. As will be explained in more detail below, Figures 3A-3F show examples of defect-free images, and Figures 4A-4E show examples of defective images (for example, a set of training images typically includes both a large number of defective and defect-free workpiece images). The differences between the images help to illustrate part of why it is desirable to use a large number of training images to train the defect detection unit 140dp. More specifically, partly due to the different possible characteristics of different workpiece images that may be acquired to inspect the type of workpiece (for example, as shown by the differences between the workpiece images in Figures 3A-3F and 4A-4E to inspect the type of machined aluminum plate), the accuracy of defect detection by the defect detection unit 140dp can be improved by using a variety of workpiece images for training. These workpiece images may be similar to a variety of images that may be acquired later during run mode and / or that can make it easier to detect defects in such images. For example, with regard to the specific examples in Figures 3A-3F, 4A-4E, and 5A-5D, such training can help the defect detection unit 140dp distinguish between defects (e.g., various types of scratch defects in the examples shown in the figures) and features of a normal workpiece surface (e.g., various types of machining marks formed on the plate surface, which typically appear as hash textures that vary across the entire surface in the examples shown in the figures).

[0043] As described above, Figures 3A–3F and 4A–4F are examples of workpiece images of sections of a workpiece that is a relatively "flat" machined aluminum plate. Each image is taken from the same viewpoint (e.g., directly overhead at a 90-degree angle to the plate), but each is at a different XY position on the plate. Each image shows a view of the plate surface of approximately 2.5 mm × 1.9 mm (XY). Thus, each image shows a magnified view of a portion of the plate surface. Machined marks formed on the plate surface usually result in a hash texture that varies across the entire surface. In the images, at such a magnified scale, a relatively flat plate may not appear flat. In the current example, the height of some of the hash ridges of machined marks may be approximately 5 micrometers or less.

[0044] The machined aluminum plate sections shown in Figures 3A to 3F do not contain defects. In other words, Figures 3A to 3F show examples of images of "defect-free" machined aluminum plate sections. In contrast, Figures 4A to 4E show examples of images of workpiece sections that contain defects. The images in Figures 4A to 4E are similar to those in Figures 3A to 3F, but Figures 4A to 4E include defects formed on the surface of the machined aluminum plate. The defects in these examples are scratches formed on the surface of the machined aluminum plate. More specifically, Figure 4A shows scratch defects 402A1 and 402A2, Figure 4B shows scratch defects 402B1 and 402B2, Figure 4C shows scratch defects 402C1 and 402C2, Figure 4D shows scratch defect 402D, and Figure 4E shows scratch defects 402E1 and 402E2. As described above, the various characteristics of defects and workpiece surfaces shown in Figures 3A to 3F and 4A to 4E help to explain in part why it is desirable to use a large number of training images for training the defect detection unit 140dp (for example, a set of training images typically includes many images of workpieces with defects and workpieces without defects).

[0045] Figures 5A–5D are examples of images of similar parts of a workpiece that are defective or not, which may be obtained using a workpiece inspection and defect detection system similar to the workpiece inspection and defect detection system in Figure 1. Figure 5A shows an example of a section of a plate that may be classified as "defect-free". Figures 5B–5D show examples of images of similar sections of a similar plate that may be classified as "defective". Each image contains scratch defects (e.g., the respective scratch defects 502B, 502C, and 502D). In the examples of Figures 5A–5D, the images are of similar parts of a workpiece (e.g., the type of machining marks formed on the surface of the part is similar or nominally the same in each image, and the main difference between the images is the characteristics of the respective scratch defects 502B, 502C, and 502D). In one exemplary implementation, the images in Figures 5A–5C may be included as part of a training set of images for training the defect detection unit 140dp.

[0046] The image in Figure 5D may be an example of a run-mode image, which can be analyzed by the defect detection unit 140dp to determine whether it should be classified as a defective or non-defective image. In various implementations, the defect detection unit 140dp may be sufficiently trained to appropriately classify the image in Figure 5D as a defective image (for example, trained with training images in Figures 5A to 5C, which include similar parts of the workpiece and scratch defects 502B and 502C having specific similar characteristics to scratch defect 502D). In various implementations, certain additional processes may be performed with respect to the exemplary scratch defect 502D. For example, one or more measurement processes may be performed in conjunction with the defect detection process. These processes can determine various dimensions or other characteristics of the exemplary scratch defect 502D, as will be described in more detail below with respect to Figure 6.

[0047] Figure 6 illustrates the use of one or more video tools to perform measurement operations on an image of a workpiece containing a defect (for example, to determine the dimensions of the defect). As illustrated, in the case of image 601 (for example, which may be similar to or the same as the image in Figure 5D) containing a scratch defect 602 (for example, which may be similar to or the same as scratch defect 502D), the video box tool 606 includes scanlines 608 (for example, this may further or instead represent video point tools, etc.) which are used to determine the position, dimensions, and / or other aspects of the edges of the scratch defect 602. In various exemplary implementations, the video box tool 606 can be resized, positioned, and rotated until the box tool 606 indicates or defines a region of interest (for example, a region within the box tool 606), and the edges of the scratch defect 602 can be determined using the arrows shown in Figure 6 (for example, representing scanlines, point tools, etc.). In various exemplary embodiments, the videobox tool 606 can typically use one or more conventional edge gradients along the edges of defects 602 within the region of interest, and the edges of defects 602 can be determined based on the local magnitude of the edge gradients along various scanlines 608, etc.

[0048] In various exemplary implementations, such measurement operations may include performing specific morphological filtering or other filtering (e.g., to distinguish the edges of a scratch from the machined pattern of a workpiece; specific types of filtering are described in U.S. Patent No. 7,522,763, which is incorporated herein by reference in its entirety). As shown in Figure 6, in the display area included in the image, a box tool 606 with scanline 608 is used to determine the edge location of the scratch defect 602 (e.g., the outer edge or perimeter). Based on such determination, the video tool and / or other measurement operations may include determining the dimension D1 of the scratch defect (e.g., corresponding to the length and / or other dimensions of the scratch defect 602). In various exemplary implementations, the box tool 606, scanline 608, and / or other video tools and / or measurement operations can be used to determine other dimensions of the scratch defect 602 (e.g., width, depth, etc.). For example, as mentioned above, the video tool unit 143 may include a Z-height measuring tool unit 143z, and the corresponding video tool or operation can be used to determine the Z-height dimension of a scratch defect (for example, determining the depth of the scratch relative to other parts or features of the workpiece surface).

[0049] As part of the general operation of the defect detection unit, some detected defects may require further measurement or inspection to determine additional defect parameters. For example, as described above, various types of analysis and / or processing of the defective image, including the scratch defect 602, can determine the XY position and approximate XY region of the defect 602, and / or other dimensions (e.g., using video tools and / or other operations described above). If the 3D characteristics of a potential defect are important (e.g., if a scratch must be deeper than a certain value to be considered a defect), the defect detection unit 140dp can initiate a process to utilize the Z height measurement tool (e.g., of the Z height measurement tool unit 143z) or other 3D sensing processes (e.g., to acquire a 3D point cloud of the affected area to determine the depth of the scratch).

[0050] In various implementations, the image data of each image described herein includes multiple pixel values ​​(for example, each pixel value corresponds to a pixel in the image). The description herein of using an image (e.g., a training mode image) for training the defect detection unit 140dp indicates that corresponding image data (e.g., training mode image data corresponding to a training mode image) is provided and used for training (for example, conversely, the description herein of using image data for training indicates that the corresponding image is used for training). Similarly, the description herein of the defect detection unit 140dp analyzing an image (e.g., performing an analysis on a run mode image to determine a defective image containing a defective workpiece) indicates that corresponding image data (e.g., run mode image data corresponding to a run mode image) is provided and used for performing the analysis (for example, conversely, the description herein of using image data for analysis during run mode indicates that the corresponding image is used for analysis). As will be described in more detail below, in some implementations for training or analysis, image data (e.g., raw image data acquired from the camera unit) can be used to determine composite image data (e.g., corresponding to a composite image) that can be used for training or analysis.

[0051] In various implementation configurations, the defect detection unit 140dp is configured to detect defects in a workpiece containing multiple layers formed from different materials. Each defect corresponds to imaging of a first material, and it may be expected that only the second material will be imaged. For example, a workpiece containing an aluminum substrate with a copper layer formed on the aluminum substrate may be subjected to various types of processing (e.g., drilling, etching) that could inadvertently remove too much copper, and / or other problems (e.g., scratches as shown in Figures 3A-3F, 4A-4E, and 5B-5D, etc.), resulting in the exposure of the aluminum substrate. This constitutes a defect. In other words, if the defect detection unit 140dp determines that exposed aluminum is present / imaged in an area of ​​the workpiece where it is expected that only exposed copper will be present, the defect detection unit 140dp can determine that there is a defect in that area. As another example, a workpiece may contain one or more parts that are painted or coated. In this case, chipping or scratching of the paint or coating, or problems with the paint or coating, may expose the underlying material, and such exposure may be considered a defect. Generally, if the defect detection unit 140dp determines that a second material is present / imaged in an area of ​​the workpiece that is expected to contain only a first material (e.g., paint, coating, top layer, etc.), the defect detection unit 140dp can determine that there is a defect in that area.

[0052] In various implementations, the defect detection unit 140dp can be configured to determine the presence of defects in a workpiece based on a process that at least partially depends on multiple different reflectance profiles of different materials contained in the workpiece. Different materials (e.g., copper, aluminum, gold, silver, etc.) are known to have different reflectance profiles (e.g., reflectance versus wavelength curves), and the reflectance of each material changes according to the corresponding wavelength of light directed at the material. For example, by including the spectral reflectance profiles of different materials (e.g., silver, copper, aluminum, and carbon steel) in a single graph, it is possible to facilitate the comparison of spectral reflectance profiles and see the reflectance of each material at a specific wavelength (for example, see the graph at https: / / www.researchgate.net / figure / Spectral-reflectance-of-aluminum-and-various-other-metals-O-Elsevier-Reprinted-with_fig1_283437763 as one specific example of a graph containing several spectral reflectance profiles for comparison).

[0053] For example, in one particular implementation, when illuminated with blue light having a wavelength of approximately 450 nanometers (e.g., emitted from a blue LED), copper may have a reflectivity of approximately 50%, while aluminum may have a reflectivity of approximately 95%, according to specific test evaluation criteria. When illuminated with green light having a wavelength of approximately 570 nanometers (e.g., emitted from a green LED), copper may have a reflectivity of approximately 65%, while aluminum may have a reflectivity of approximately 92%. When illuminated with red light having a wavelength of approximately 700 nanometers (e.g., emitted from a red LED), copper may have a reflectivity of approximately 96%, while aluminum may have a reflectivity of approximately 90%. Other materials (e.g., gold, silver, etc.) have different reflectivity properties, which may be indicated by their corresponding reflectivity-wavelength curves.

[0054] Defect detection systems according to various embodiments of this disclosure can advantageously utilize different reflectance profiles for two or more different materials contained in a workpiece under analysis. For example, two wavelengths (i.e., colors) of light may be selected to illuminate a workpiece containing two different materials while different color channel image data are acquired. These two wavelengths are selected such that the difference in the reflectance curve values ​​of the two different materials for at least one of the wavelengths is relatively large (e.g., approaching the maximum difference within the practical limits of the system), and / or the ratio of the difference between the two wavelengths is relatively large. For example, according to the test evaluation criteria described above, blue light having a wavelength of about 450 nanometers (e.g., output from a blue LED) may be selected and used. In contrast, copper may have a reflectance of about 50%, and aluminum may have a reflectance of about 95%. In this case, the difference between 50% and 95% is relatively large and may approach the maximum difference in reflectance curves (e.g., wavelengths and other components usable within the practical limits of the system). In such an example, the second wavelength may be red light having a wavelength of about 700 nanometers (e.g., output from a red LED). In contrast, copper may have a reflectivity of approximately 96%, and aluminum may have a reflectivity of approximately 90%. In this case, the blue / red ratio can be approximately 50% / 96% = 0.52 for copper reflectivity and approximately 95% / 90% = 1.05 for aluminum reflectivity (for example, the ratio for copper is far below 1, and the ratio for aluminum is close to or above 1), which is a relatively large ratio difference. As will be explained in more detail below, the ability to detect defects can be improved by inputting such different color channel image data into the defect detection unit 140dp. In one implementation, determining the ratio between different color channel image data can provide particularly advantageous characteristics for enabling defect detection, for example, as shown below with respect to Figure 8F.

[0055] In various implementations, a monochrome camera can provide the highest XY resolution (compared to a color camera where each color image contains only a portion of the camera's pixels designated for that color, for example, when a Bayer filter is used). In some implementations, red, green, and / or blue LEDs are used to capture a monochrome image of the workpiece (for example, the LEDs may be selected according to a specific desired wavelength as described above). Specifically, different blue LEDs may be available that provide different blue wavelengths within the blue light range, such as a wavelength corresponding to 430 nanometers, another wavelength corresponding to 440 nanometers, and another wavelength corresponding to 450 nanometers. Following the example above, a blue LED corresponding to 450 nanometers may be selected for use in the system, and the desired ratios and / or other characteristics of the system can be achieved according to the reflectance profile values ​​of copper and aluminum at 450 nanometers (for example, approaching and / or nearly corresponding to the maximum difference in the reflectance curves, as opposed to 430 nanometers or 440 nanometers, which may correspond to a smaller relative difference in the reflectance of copper and aluminum). Similarly, different red LEDs may be available that offer different red wavelengths within the range of red light, such as a wavelength corresponding to 650 nanometers, another wavelength corresponding to 675 nanometers, and yet another wavelength corresponding to 700 nanometers. Following the example above, a red LED corresponding to 700 nanometers could be selected for use in the system to achieve the desired ratio, etc.

[0056] In such examples, it will be understood that in some implementations, an LED and / or other light source may be characterized by a single wavelength, but may output a spectrum with a corresponding spectral bandwidth (e.g., possibly less than 60 nanometers). For example, a particular LED may be a relatively narrowband emitter that outputs a nearly Gaussian-shaped spectrum. In various implementations, the single wavelength used to characterize such a light source may be based on a characterizing wavelength, such as the peak wavelength, center wavelength, centroid wavelength, or dominant wavelength of the corresponding spectrum. In such cases, it will be understood that the light source outputs light with the characterizing wavelength, as well as light with other wavelengths of the corresponding ambient spectrum.

[0057] During operation, each monochrome image can be captured while one of the red, green, or blue LEDs is emitting light. These monochrome images can be used to generate a composite RGB image of the workpiece. The image data of the composite RGB image includes multiple red pixel values ​​corresponding to the monochrome image captured while the workpiece was illuminated only by the red LED, multiple green pixel values ​​corresponding to the monochrome image captured while the workpiece was illuminated only by the green LED, and multiple blue pixel values ​​corresponding to the monochrome image captured while the workpiece was illuminated only by the blue LED. In some implementations, a composite monochrome image of the workpiece is generated by capturing monochrome images using only the blue and red LEDs. In this case, the image data of the composite monochrome image may correspond to multiple pixels with values ​​based on the ratio of the pixel values ​​of the monochrome image captured while the workpiece was illuminated only by the blue LED to the pixel values ​​of the monochrome image captured while the workpiece was illuminated only by the red LED. In various implementations, a defect mask image (e.g., on the composite image) may be drawn or otherwise determined to identify defects for training the defect detection unit 140dp.

[0058] In one or more implementation configurations, the light source 230 of the vision component section 200 of the workpiece inspection and defect detection system 100 includes multiple LEDs, each outputting light having multiple different colors (i.e., wavelengths) corresponding to multiple color channels. Each color channel corresponds to a different color. In various implementation configurations, each LED may have a wavelength range greater than 25 nanometers but less than 60 nanometers (e.g., a wavelength range of 50 nanometers). For example, the light source 230 may include a light source configuration comprising a first LED providing a first color light, a second LED providing a second color light, and a third LED providing a third color light. The light source 230 is configured to provide a first color light source, a second color light, and a third color light, but a subset of these can be selected based on the reflectance profile (e.g., spectral reflectance curve) of the material contained in the workpiece. For example, based on the reflectance profiles of the first and second materials, the first and second colored lights may be selected to be used to inspect a workpiece containing the first and second materials, while the third colored light may not be selected. The first and second colored lights may be selected so as to obtain a relatively large difference between the values ​​and / or ratios of different materials relative to the first and second colored lights. In this regard, as described above, with respect to the reflectance-to-wavelength data for a particular material contained in the workpiece being inspected, it may be desirable to select two colors (i.e., wavelengths) that provide a desired large difference between the values ​​and / or ratios of different materials relative to the first and second colored lights by selecting the wavelengths of light used to illuminate workpieces containing different materials. Furthermore, the third colored light may be selected to be used to inspect a workpiece containing the third material based on the reflectance profile of the third material, which is different from the reflectance profiles of the first and second materials. For example, a first color light and a second color light can be selected for a workpiece containing a first material and a second material, and a first color light and a third color light can be selected for a workpiece containing a first material and a third material.

[0059] As mentioned above, a multilayer workpiece may contain multiple materials. Each material may have a different spectral reflectance profile. Therefore, each material may reflect light differently depending on the wavelength (i.e., color) of the light illuminating the material. Thus, if a multilayer workpiece including an aluminum substrate covered with a layer of copper is imaged, for example, while aluminum has a relatively high reflectance and copper has a relatively low reflectance, the resulting image may have pixels with relatively high values ​​in each area where aluminum is exposed (if present). For example, if aluminum and copper are illuminated with blue light having a wavelength of 450-495 nanometers, according to specific test evaluation criteria, aluminum may have a relative reflectance of approximately 80% and copper may have a relative reflectance of less than 40% (for example, aluminum may have about twice the reflectance of copper). As will be explained in more detail below, in various mounting configurations, when color image data including blue and red image data is captured, the ratio of blue to red image data may be more sensitive to the presence of defects than the color image data itself.

[0060] In some implementations, a camera configuration including a camera can be used with different color planes for generating multicolor images. As defined herein, each of these different color planes of a camera may correspond to a different color image having corresponding different color channel image data. More specifically, a camera that generates a blue color plane, a green color plane, and a red color plane may be defined herein as generating a blue color image, a green color image, and a red color image, each having corresponding color channel image data. In some implementations, each of the different color planes may be described or defined as an image plane. Hereinafter, each image plane is considered to correspond to a different color image having corresponding color channel image data that has been generated. Thus, for each of several sets of training mode work image data, a corresponding first color channel training mode work image data (e.g., corresponding to an image similar to the one shown in Figure 8A) and a corresponding second color channel training mode work image data (e.g., corresponding to an image similar to the one shown in Figure 8C) may be generated by the respective first and second camera image planes. Furthermore, for each of the multiple sets of run mode work image data, a corresponding first color channel run mode work image data (for example, corresponding to an image similar to the one shown in Figure 8A) and a corresponding second color channel run mode work image data (for example, corresponding to an image similar to the one shown in Figure 8C) can be generated by the respective first and second camera image planes.

[0061] In various implementations, a workpiece can be imaged using an RGB color camera that generates red, green, and blue image data (i.e., RGB image data). For example, an RGB camera can capture a first image of a workpiece while it is illuminated with blue light emitted from a blue LED, and then capture a second image of the workpiece while it is illuminated with red light emitted from a red LED. As described above, the specific blue and red LEDs used can be selected so that the light emitted from the blue LED and the light emitted from the red LED achieve the desired ratio and / or other characteristics of the system. In such implementations, the reflectance spectra of copper and aluminum in the visible wavelength of light are taken into consideration. In some implementations, the light emitted from the blue LED and the light emitted from the red LED are short and long, respectively, within the limits of what the sensitivity of the RGB camera can tolerate.

[0062] In various implementation configurations, the sensitivity of the defect detection unit 140dp can be optimized by selecting a red LED that outputs red light with a characteristic wavelength centered around the point where the reflectivity of copper and aluminum is approximately the same (i.e., the isosbestic point). Alternatively, a blue LED can be selected that outputs blue light with a characteristic wavelength that maximizes the difference in reflectivity between copper and aluminum (while remaining within the wavelength range passed through the blue filter of an RGB Bayer filter machine vision camera and / or within the practical limits / restrictions of the system).

[0063] In various implementations, it is desirable to select color light corresponding to each of several different color channels according to wavelengths that produce the maximum or desired ratio or other characteristics between two color channels (e.g., a blue color channel and a red color channel), thereby enabling the defect detection unit 140dp to be optimally trained and to have the best ability to identify defects as part of a run-mode process (e.g., similar blue and red images are generated to obtain a color ratio). Once the wavelengths of two different colors that provide the desired / optimal ratio are determined, different configurations can be utilized to realize a system having different color wavelengths for different color channels. With respect to the color channels as defined herein, these can be generated by utilizing any other known techniques that can be used to generate or filter different color light, such as different color light sources (e.g., separate LEDs for each color), different filters, or those that can be used for different color channels according to the principles described herein.

[0064] For example, as part of the manufacturing process, an LED or other light source that produces a desired wavelength for a desired aspect ratio of the image can be selected and included. Another example is selecting different color filters that pass through the desired wavelengths of the corresponding color channels. In some cases, such filters may be tunable or adjustable to achieve the desired wavelengths. In some implementations, different filters available can be rotated or inserted into the system to select the desired wavelengths. For example, in some implementations, each color filter may pass through a spectral bandwidth such as less than 40 nanometers or less than 20 nanometers to determine the wavelength of its respective color channel.

[0065] In various implementations, a monochrome camera that generates monochrome image data can be used to image the workpiece. For example, using a step-and-repeat procedure, the camera is advanced (stepped) to a certain position, fixed in that position, and two images are captured: a first image while the workpiece is illuminated with red light (e.g., from a red LED) and a second image while the workpiece is illuminated with blue light (e.g., from a blue LED). This procedure is then repeated. The defect detection unit 140dp then uses the images for training and / or run-mode analysis (e.g., the ratio of the image data of the two images can be determined). Such an implementation using monochrome images may have the advantage of better spatial resolution compared to an implementation using a Bayer filter. It can also collect more light compared to an implementation using a Bayer filter, although this effect may be relatively small compared to an implementation using a Bayer filter with two narrowband LEDs that are well within the Bayer filter's pass wavelength. Implementations without a Bayer filter may be limited by the camera's sensitivity, but not by the Bayer filter's pass wavelength. This advantageously maximizes the freedom to select the desired wavelength (for example, according to the reflectance profile of the workpiece material). However, this implementation can slow down image acquisition time (for example, it could be about twice as slow in some cases) because additional time is required at each location to acquire two different images of the two color channels.

[0066] To reduce image acquisition time, various implementations allow for imaging of the workpiece using a monochrome camera containing two image sensors. For example, the monochrome camera could be a multisensor prism-based multispectral area scan camera (such as those commercially available from JAI Corporation). This may include predefined configurations for the visible and near-infrared (NIR) wavelength bands, or configurations tailored to application requirements using two or three custom wavelength bands. In various implementations, the use of such a monochrome camera can enable full resolution and the selection of custom color path wavelengths. Furthermore, using such a monochrome camera can enable high-speed image acquisition, where two images (e.g., of two color channels) can be captured in close proximity to each other, including a first image while the workpiece is illuminated with a first color light (e.g., red light from a red LED) and a second image while the workpiece is illuminated with a second color light (e.g., blue light from a blue LED). In such implementations (for example, or other implementations utilizing different camera configurations), if the workpiece moves relative to the system between the acquisition times of the two images (for example, if the workpiece moves on a conveyor belt relative to the system), the defect detection unit 140dp can be configured to offset the images so that the pixel data overlaps based on the XYZ position of each image (for example, areas of the image without overlapping data may be ignored, and the overlapping data may be considered to correspond to a common field of view between the images).

[0067] In various implementations, an RGB image is captured (for example, the image data of red, green, and blue pixels can be considered to correspond to the respective red, green, and blue color channel images), and the defect detection unit 140dp can be configured to use the image data of the RGB image (for example, as described in more detail below) to create a composite image with the same resolution or half the height and width dimensions. The defect detection unit 140dp can also be configured to convert each RGB pixel value to the ratio of the monochrome pixel value. Furthermore, the defect detection unit 140dp can be configured to convert each RGBG quad pixel to a mono pixel with a single ratio (for example, as described in more detail below).

[0068] The defect detection unit 140dp can be configured to apply various kernels to generate a composite image. Table 1 below shows an example of raw image data containing four RGBG quad pixels. More specifically, the raw image data includes a first RGBG quad pixel having pixels R1, G1, B1, and G1; a second RGBG quad pixel having pixels R2, G2, B2, and G2; a third RGBG quad pixel having pixels R3, G3, B3, and G3; and a fourth RGBG quad pixel having pixels R4, G4, B4, and G4. In various implementations, the red pixels R1, R2, R3, and R4 may be considered to correspond to the red color channel image, the green pixels G1, G2, G3, and G4 may be considered to correspond to the green color channel image, and the blue pixels B1, B2, B3, and B4 may be considered to correspond to the blue color channel image. [Table 1]

[0069] Table 2 below shows an example of coarse-resolution composite ratio data with four pixel values ​​generated based on the raw image data shown in Table 1. More specifically, the coarse-resolution composite ratio data includes a first pixel with a value based on B1 / R1, a second pixel with a value based on B2 / R2, a third pixel with a value based on B3 / R3, and a fourth pixel with a value based on B4 / R4. In various implementations, the actual value of each pixel in the composite ratio data may be based on further calculations that utilize the respective ratios such as B1 / R1, as will be described in more detail (e.g., applying scaling factors to obtain values ​​in the range of 0 to 255 and / or other processes). [Table 2]

[0070] Table 3 below shows an example of fine-resolution composite ratio data with four pixel values ​​generated based on the raw data shown in Table 1. More specifically, the fine-resolution composite ratio data includes a first pixel with a value based on (B1+B2+B3+B4) / (4*R1), a second pixel with a value based on (B2+B4) / (R2+R1), a third pixel with a value based on (B3+B4) / (R1+R3), and a fourth pixel with a value based on 4*B4 / (R1+R2+R3+R4). In various implementations, the actual value of each pixel in the composite ratio data may be based on further calculations utilizing their respective ratios, such as (B1+B2+B3+B4) / (4*R1), as will be explained in more detail (e.g., applying scaling factors to obtain values ​​in the range of 0 to 255 and / or other processes). [Table 3]

[0071] In various implementations, synthetic white illumination (SWI) can be created and used when illuminating a workpiece. SWI is a combination of illumination values ​​for red, green, and blue LEDs aimed at creating white light with a desired color balance. Furthermore, customized SWI can be created and used when illuminating a workpiece. A customized SWI is a combination of illumination values ​​for red and blue LEDs only, for illuminating a workpiece while being imaged using a color camera (e.g., a color machine vision camera), where the wavelengths and / or ratios of the red and blue illumination are selected according to the principles disclosed herein. Similarly, synthetic red-blue illumination (SRBI) can be created and used when illuminating a workpiece. In some implementations, a graphical user interface can be used to select one of the above color options and / or other color options for illuminating a workpiece with a certain light source configuration (e.g., using light source 230).

[0072] Figures 7A to 7C are examples of images of a portion of a workpiece containing defects, which may be obtained using a workpiece inspection and defect detection system similar to the workpiece inspection and defect detection system in Figure 1. The workpieces shown in Figures 7A to 7C have an aluminum substrate on which a layer of copper is formed. The workpieces shown in Figures 7A to 7C include defects 702, 704, 706, and 708, which are circled for illustrative purposes. In one example, each of defects 702, 704, 706, and 708 may correspond to exposed aluminum in the corresponding image of the workpiece surface. More specifically, in certain mounting configurations, various types of processing (e.g., drilling, etching) may inadvertently remove too much of the covering copper and / or other problems (e.g., scratches) may occur, which may expose the aluminum substrate, and this may be considered a defect.

[0073] The images shown in Figures 7A to 7C are sets of monochrome images of the same workpiece, obtained under different lighting conditions. More specifically, Figure 7A is a monochrome image of a portion of the workpiece taken by a monochrome camera while the workpiece was illuminated with blue light emitted from, for example, a blue LED. Figure 7B is a monochrome image of a portion of the workpiece taken by a monochrome camera while the workpiece was illuminated with green light emitted from, for example, a green LED. Figure 7C is a monochrome image of a portion of the workpiece taken by a monochrome camera while the workpiece was illuminated with red light emitted from, for example, a red LED.

[0074] As can be seen by comparing the images in Figures 7A, 7B, and 7C, certain features in each image may appear brighter or darker (for example, the reflectivity of copper, aluminum, etc., in each image will differ depending on the color of the light used to illuminate the workpiece in each image). In various implementations, the image data of each image described herein contains multiple pixel values. Each pixel value (including, for example, a luminance value) corresponds to a pixel in the image. Images of pixels at the same position, or the same surface point on a workpiece in different images, can be designated as corresponding pixels between different images (for example, different corresponding pixel values ​​may originate from different colored light used to illuminate the workpiece in each image).

[0075] Figure 7D is an example of a composite image (generated, for example, based on image data from Figures 7A to 7C). More specifically, Figure 7D is a composite color image based on image data containing red, green, and blue color information (for example, the red color information may correspond to the monochrome image shown in Figure 7C, the green color information to the monochrome image shown in Figure 7B, and the blue color information to the monochrome image shown in Figure 7A). Composite color images such as the composite color image shown in Figure 7D can be useful to the user, for example, for observing and / or inspecting parts of a workpiece.

[0076] Figure 7E is an example of a defect mask image that can be obtained based on the composite image of Figure 7D (for example, based on one or more of the images from Figures 7A to 7C). In various implementations, the defect mask may be handwritten (for example, as described in more detail below), and / or provided by the user by other means, or provided by an automated process, in order to identify defects. The image shown in Figure 7E is an example of one of several images containing image data that may be used to train a model utilized by the defect detection unit 140dp to determine whether a workpiece contains defects. In various implementations, in order to train the model, the user may classify each training image in the set of training images as either defect-free or defect-containing. Thus, any of the images shown in Figures 7A to 7D may be classified as defect-containing (for example, containing one or more defects on the workpiece surface in the image).

[0077] As part of the handwriting process, the user can mark pixels corresponding to each defect in the image (for example, using a selector in the user interface) (for example, by marking pixels on a transparent layer overlaid on the original image using a tool like Paint to create an image where the color pixels correspond to matching defect pixels in the original image, as shown in the example in Figure 7E). For example, as part of the image data, the user marks pixels corresponding to each defect by setting the values ​​of the pixels contained in each of the defects 702, 704, 706, and 708 in the image shown in Figure 7E to a predetermined value (e.g., 255). The user then makes the defect detection unit 140dp available to train the defect detection unit 140dp in training mode by marking one or more selections, for example using a graphic user interface. After the defect detection unit 140dp has been trained using image data from a set of training images (including, for example, the image shown in Figure 7E), the user can operate the defect detection unit 140dp in run mode to detect defects contained in a set of run mode images generated under similar conditions (e.g., the same lighting conditions and / or composite image generation process) as the set of training mode images used to train the defect detection unit 140dp.

[0078] The images shown in Figures 7A to 7E can be considered examples of both training mode and run mode images. In one exemplary implementation, some or all of the image data shown in Figures 7A to 7E may be included as part of a set of training mode image data for training the defect detection unit 140dp (for example, the set of image data may include image data from images corresponding to the same field of view of the workpiece). Alternatively, some or all of the images in Figures 7A to 7D may be considered examples of run mode images, which can be analyzed by the defect detection unit 140dp to determine whether each image should be classified as a defective or non-defective image.

[0079] Figures 8A to 8C are examples of images of defective parts of a workpiece that can be obtained using a workpiece inspection and defect detection system similar to the one in Figure 1. The workpieces shown in Figures 8A to 8C have an aluminum substrate on which a layer of copper is formed. The workpieces shown in Figures 8A to 8C include defects 802, 804, and 806, which are circled for illustrative purposes (each defect may correspond to exposed aluminum, for example, as in the examples in Figures 7A to 7C).

[0080] The images shown in Figures 8A to 8C are sets of monochrome images of the same workpiece, obtained under different lighting conditions. More specifically, Figure 8A is a monochrome image of a portion of the workpiece taken by a monochrome camera while the workpiece was illuminated with blue light emitted from, for example, a blue LED. Figure 8B is a monochrome image of a portion of the workpiece taken by a monochrome camera while the workpiece was illuminated with green light emitted from, for example, a green LED. Figure 8C is a monochrome image of a portion of the workpiece taken by a monochrome camera while the workpiece was illuminated with red light emitted from, for example, a red LED.

[0081] In one example, the images shown in Figures 7A to 7C show a defect in the first part of the workpiece, and the images shown in Figures 8A to 8C show a defect in the second part of the workpiece. In another example, the images shown in Figures 7A to 7C show a defect in a part of the first workpiece, and the images shown in Figures 8A to 8C show a defect in a part of the second workpiece. As described above, in various implementation configurations, the image data of each image described herein includes multiple pixel values ​​(for example, each pixel value corresponds to a pixel in the image, and the imaging of pixels at the same position, or the same surface point on the workpiece in different images, can be specified as corresponding pixels between different images, with the corresponding pixel values ​​being the corresponding pixels).

[0082] Figure 8D is an example of a composite image (for example, generated based on image data from Figures 8A to 8C). More specifically, Figure 8D is a composite color image based on image data containing red color information, green color information, and blue color information (for example, the red color information may correspond to the monochrome image shown in Figure 8C, the green color information to the monochrome image shown in Figure 8B, and the blue color information to the monochrome image shown in Figure 8A). Color images such as the color image shown in Figure 8D can be useful to the user, for example, for observing and / or inspecting parts of a workpiece.

[0083] Figure 8E is an example of a defect mask image based on the composite image shown in Figure 8D or Figure 8F (for example, and / or based on one or more of the images in Figures 8A to 8C, similar to the defect mask in Figure 7E). The image shown in Figure 8E is an example of one of many images containing image data that can be used to train a model used by the defect detection unit 140dp to determine whether a workpiece contains defects. To train the model, the user can classify each of the multiple training images as either defect-free or defect-containing. Thus, any of the images shown in Figures 8A to 8D can be classified as defect-containing (for example, each image contains one or more defects on the workpiece surface). The user can then mark the pixels corresponding to each defect in each defect-containing image (for example, by using a tool such as Paint to mark pixels on a transparent layer overlaid on the original image, creating an image where the color pixels correspond to the matching defect pixels in the original image). For example, as part of the image data, the user can mark the pixels corresponding to each defect by setting the value of the pixels contained in each of the defects 802, 804, and 806 in the image shown in Figure 8E to a predetermined value (e.g., 255).

[0084] After the defect detection unit 140dp is trained using image data from a set of training images (including, for example, the images shown in Figure 8E), the user can operate the defect detection unit 140dp in run mode to detect defects in a set of run mode images generated under similar conditions (e.g., the same lighting conditions and / or composite image generation process) as the set of training mode images used to train the defect detection unit 140dp. Note that in various implementations, defect masks (which may be handwritten by the user or generated in other ways to help the defect detection unit 140dp learn the characteristics of defects), such as those shown in Figures 7E and 8E, may not be included as part of the run mode images. This is because, during run mode, the defect detection unit 140dp detects defects without the user or another process creating a handwritten defect mask. More specifically, in various implementations, such defect masks may be primarily used in training and training images, rather than in run mode images.

[0085] Figure 8F is an example of a composite image (e.g., a monochrome image) generated based on the images in Figures 8A and 8C, as will be described in more detail below. The image shown in Figure 8F is one of several examples of images that can be used as a training image (its image data may be used to train a model that is used by the defect detection unit 140dp to determine whether a workpiece contains defects) or as a run-mode image analyzed by the defect detection unit 140dp to determine whether a workpiece contains defects. To train the model, the user may classify the image shown in Figure 8F as a defective image. After the defect detection unit 140dp has been trained using image data from a set of training images (e.g., including the image shown in Figure 8F), the user can operate the defect detection unit 140dp in run mode to detect defects in a set of run-mode images generated under similar conditions (e.g., the same lighting conditions and / or composite image generation process) as the set of training-mode images used to train the defect detection unit 140dp.

[0086] The composite image shown in Figure 8F is based on image data containing multiple pixel values. As described above, each pixel value is generated using a scaling factor and the ratio of the corresponding pixel value contained in the image data of the images shown in Figures 8A and 8C, respectively, acquired while the workpiece was illuminated with blue and red light. Below is an example of how to calculate or determine the pixel values ​​contained in the image data corresponding to the composite image (e.g., monochrome image) shown in Figure 8F. For the sake of simplicity in explaining Figure 8F, the exposed aluminum defects 802, 804, and 806 are shown to be clearly distinguishable from the surrounding copper surface portion (e.g., somewhat similar to the defect mask in Figure 8E). The process for forming Figure 8F, described below, aims to make the exposed aluminum defects more easily distinguishable from the copper surface portion (e.g., compared to Figures 8A-8D), and the resulting contrast between features in the correspondingly formed composite image can typically depend on various factors (e.g., the characteristics of the workpiece surface features in the image).

[0087] In one specific example, the first pixel in the image shown in Figure 8A (e.g., illuminated with blue light and corresponding to a point on the surface of the copper and / or the workpiece) may have a value of 100 (e.g., a luminance value), and the corresponding first pixel in the image shown in Figure 8C (e.g., illuminated with red light and corresponding to a point on the surface of the same copper and / or the workpiece) may have a value of 200 (e.g., a luminance value). Under various mounting configurations and specific conditions, the ratio of 100 / 200 may indicate that the reflectivity of copper under blue light is approximately 0.5 times that of copper under red light (for example, assuming that reflectivity and luminance, as indicated by the pixel value, are roughly linearly related, as will be explained in more detail below). The defect detection unit 140dp can be configured to calculate an intermediate ratio value between blue and red (i.e., 100 / 200 = 0.5) by dividing 100 by 200. The defect detection unit 140dp can be configured to scale the intermediate ratio between blue and red using a scaling factor to obtain a value in the range of 0 to 255. This range corresponds to the range of values ​​that can be contained in a 1-byte luminance value (for example, a monochrome image, an RGB image, or another image). If the maximum ratio value is defined as 1.5 and the minimum ratio value is defined as 0.1, the defect detection unit 140dp can be configured to calculate the ratio scale value (i.e., 1.5 - 0.1 = 1.4) by subtracting 0.1 from 1.5. Furthermore, the defect detection unit 140dp can be configured to scale the intermediate ratio value between blue and red using a scaling factor by first subtracting the minimum ratio value from the intermediate ratio value between blue and red (i.e., 0.5 - 0.1 = 0.4), then multiplying that value by a ratio obtained by dividing the maximum byte value minus 1 by the ratio scale value (i.e., 0.4 * (255 - 1) / 1.4 = 72), and then adding 1 to the result (72 + 1 = 73). Thus, the defect detection unit 140dp can be configured to determine that the corresponding first pixel in the composite image has a value of 73 (i.e., as part of the image data of the composite image).

[0088] In another example, the second pixel in the image shown in Figure 8A (e.g., corresponding to a point on the surface of an exposed aluminum and / or workpiece, illuminated with blue light and potentially corresponding to a defect according to the above example) may have a value of 190, and the corresponding second pixel in the image shown in Figure 8C (e.g., illuminated with red light and corresponding to a point on the surface of the same exposed aluminum and / or workpiece) may have a value of 180. Under various mounting configurations and specific conditions, the ratio of 190 / 180 may indicate that the reflectivity of aluminum under blue light is approximately 1.055555 times that of aluminum under red light (for example, assuming that reflectivity and luminance, as indicated by the pixel value, are nearly linearly related, as will be explained in more detail below). The defect detection unit 140dp can be configured to calculate an intermediate ratio value between blue and red (i.e., 190 / 180 = 1.055555) by dividing 190 by 180. The defect detection unit 140dp can be configured to scale the intermediate ratio between blue and red using a scaling factor to obtain a value in the range of 0 to 255. This range corresponds to the range of values ​​that can be contained in a 1-byte luminance value (for example, a monochrome image, an RGB image, or another image). Assuming that the maximum ratio value is defined as 1.5 and the minimum ratio value is defined as 0.1, similar to the process described above for the first pixel, the defect detection unit 140dp can be configured to calculate the ratio scale value (i.e., 1.5 - 0.1 = 1.4) by subtracting 0.1 from 1.5. Furthermore, the defect detection unit 140dp can be configured to scale the intermediate ratio value between blue and red using a scaling factor by first subtracting the minimum ratio value from the intermediate ratio value between blue and red (i.e., 1.055555 - 0.1 = 0.9555555), then multiplying that value by a ratio obtained by dividing the maximum byte value minus 1 by the ratio scale value (i.e., 0.9555555 * (255 - 1) / 1.4 = 173), and then adding 1 to the result (173 + 1 = 174). Thus, the defect detection unit 140dp can be configured to determine that the corresponding second pixel in the composite image has a value of 174 (i.e., as part of the image data of the composite image).

[0089] In various implementations, similar calculations can be performed to determine the remaining pixel values ​​as part of the image data of a composite image (e.g., the image in Figure 8F). A similar process can also be used to determine pixel values ​​as part of the image data of other types of composite images, such as the composite images in the examples in Tables 2 and 3 above. More specifically, for each ratio in Tables 2 and 3 (e.g., B1 / R1, (B1+B2+B3+B4) / (4*R1), (B2+B4) / (R2+R1), etc.), scaling calculations (using scaling factors and / or other calculations) can be applied to each ratio to determine the value of each pixel in the composite image.

[0090] As described above, in some cases, reflectivity and luminance (e.g., represented by pixel values) may be nearly linearly related. More specifically, in various implementations, reflectivity and luminance may be nearly linearly related (e.g., have a high correlation) primarily. Secondarily (e.g., when the work surface is optically rough), there may be light scattering, in which case there may be some variability in the relationship between reflectivity and luminance. In applications like those described in the example above (e.g., a defect is defined as the presence of exposed aluminum on a work surface where copper is assumed), it is preferable to use the ratio of blue to red rather than the ratio of blue (e.g., for generating the composite image in Figure 8F). This is preferable in that the divisor of the ratio is preferably a relatively large number near the defect location (e.g., a luminance value of 200 for copper with respect to the divisor), which minimizes the influence of camera noise on the ratio value (e.g., also the composite image of the corresponding ratio, as in Figure 8F). For example, if a ratio of blue to red is used, and the brightness of the red channel copper pad is 200 (with + / -1 noise), the ratio of the blue channel value to the red channel value could be (blue channel value / 199), (blue channel value / 200), or (blue channel value / 201), and the effect of noise is relatively small. Conversely, if a ratio of red to blue is used, and the average brightness of the blue channel on the copper is 100 (with + / -1 noise), the ratio could be (red channel value / 99), (red channel value / 100), or (red channel value / 101), and the effect of noise is almost twice as large, which is undesirable.

[0091] It will be understood that the generated composite image data (e.g., ratio composite image data) of an image like the one in Figure 8F (e.g., ratio composite image) may enable the defect detection unit 140dp to detect defects more accurately. More specifically, following the example above, the difference in the ratio examples of blue and red for copper and aluminum (e.g., the ratio of blue and red for copper is 0.5 and the ratio of blue and red for aluminum is 1.055555) and the example ratio pixel values ​​of 73 for copper and 174 for aluminum (e.g., showing large value differences and large percentage differences) may help the defect detection unit 140dp to distinguish between aluminum and copper (e.g., in the corresponding training image and / or corresponding run-mode image). In implementations where exposed aluminum corresponds to defects, such processes and composite images may correspondingly enable the defect detection unit 140dp to detect defects more easily.

[0092] In various implementations, the above-described composite image generation process can be used to generate both training mode images used to train the defect detection unit 140dp and run mode images analyzed by the defect detection unit 140dp. More specifically, for each of several sets of training mode work image data, including a first color channel training mode work image data (similar to the image data shown in Figure 8A) and a second color channel training mode work image data (similar to the image data shown in Figure 8C), the defect detection unit 140dp determines the ratio of each pixel value in the first color channel training mode work image data to the corresponding pixel value in the corresponding second color channel training mode work image data. Furthermore, for each of the multiple sets of run-mode work image data, including run-mode work image data for the first color channel (for example, similar to the image data of the image shown in Figure 8A) and run-mode work image data for the second color channel (for example, similar to the image data of the image shown in Figure 8C), the defect detection unit 140dp determines the ratio of each pixel value in the run-mode work image data for the corresponding first color channel to the corresponding pixel value in the run-mode work image data for the corresponding second color channel. Thus, for each set of training-mode work image data, the defect detection unit 140dp uses the ratio of pixels in the corresponding first and second color channel training-mode work image data to determine the corresponding pixel value in the corresponding training-mode composite image data used to train the defect detection unit. Also, for each set of run-mode work image data, the defect detection unit 140dp uses the ratio of pixels in the corresponding first and second color channel run-mode work image data to determine the corresponding pixel value in the corresponding run-mode composite image data (for example, the image that the trained defect detection unit analyzes to determine is a defective image containing a defective work).

[0093] In various implementations, it may be preferable to use ratio-composite monochrome image data, as shown in Figure 8F, rather than image data of images as shown in Figures 8A to 8D, to detect defects in an image. In some cases, creating composite monochrome image data using the ratio of blue pixel values ​​to red pixel values ​​(for example, as mentioned above in relation to Figure 8F) may be beneficial because the resulting composite monochrome image data can improve the ability of the defect detection unit 140dp to distinguish between different materials, thereby improving defect detection. For example, defects 802, 804, and 806 may be easier to detect in the composite monochrome image data of the image shown in Figure 8F than in the image data of the images shown in Figures 8A to 8D.

[0094] The workpiece in the example in Figure 8F is described as containing at least two different materials (e.g., copper and aluminum), and generating image data of a composite monochrome image based on the ratio of the pixel values ​​described above can also help to reliably distinguish between three or more materials. For example, generating image data of a composite monochrome image based on the ratio of pixel values ​​can also help to detect other aspects, such as areas that are bright or dark due to black plastic moldings or other surface features (e.g., divots) that may be present in the workpiece. With respect to such additional materials and / or aspects, it may also be desirable to distinguish the first and second materials (e.g., copper and aluminum) from each other and to distinguish these materials from additional materials and / or features (e.g., black plastic moldings, divots) that may be present in the image. Since the defect detection unit 140dp can help to make the additional material easier to distinguish from aluminum and / or copper (partly due to, for example, the reflectance profile of the additional material and the corresponding reflectance at blue and red wavelengths), ratios with large value differences and large percentage differences can be determined for the additional material as well as for copper and aluminum.

[0095] As mentioned above, in various implementations, users can create handwritten defect masks based on observation and marking of a composite monochrome image, such as the composite monochrome image shown in Figure 8F, which is generated based on the ratio of blue and red pixel values ​​of the image data shown in Figures 8A and 8C, respectively. In addition, in various implementations, users can create handwritten defect masks based on observation and marking of a composite RGB image, such as the composite RGB image shown in Figure 8D, which consists of red, green, and blue pixel values. Furthermore, users can create handwritten defect masks based on observation and marking of other images, such as the images shown in Figures 8A, 8B, or 8C.

[0096] As described above, in various implementations, the images shown in Figures 8A to 8F can be considered examples of both training mode and run mode images. In one exemplary implementation, some or all of the image data in Figures 8A to 8F may be examples of image data included as part of a set of training mode image data for training the defect detection unit 140dp (for example, the set of image data may include image data from images corresponding to the same field of view). Also, the images in Figures 8A to 8D and / or 8F may be examples of run mode images, and their image data can be analyzed by the defect detection unit 140dp to determine whether the corresponding image should be classified as a defective or non-defective image. Although the images in Figures 7A to 7E and 8A to 8F are described herein as possible examples of both training mode and run mode images, it will be understood that generally different images may be used in training mode and run mode. More specifically, during training mode, training mode images (i.e., images with corresponding training mode data) are acquired to train the defect detection unit 140dp, and then during run mode (e.g., during a manufacturing operation to produce a workpiece), different run mode images (i.e., images with corresponding run mode data) are acquired and analyzed by the defect detection unit 140dp and / or used as part of the analysis to determine defective images containing defective workpieces. Generally, training mode images are captured using specified imaging, illumination, and workpiece conditions, and run mode images may be acquired using similar (e.g., nearly equivalent) specified imaging, illumination, and workpiece conditions (e.g., they may be acquired using the same or similar equipment and / or components, such as the same or similar light source configuration, lens configuration, and / or camera configuration).

[0097] In various implementations, a user interface may be provided. The user interface may offer the user selection or options for utilizing various color images and / or aspect ratios, as described herein, for observing various images and / or for training or utilizing the defect detection unit 140dp. For example, the user may be provided with a user interface that offers selection or options for generating and / or observing composite images (for example, having blue / red image data such as the composite image shown in Figure 8F).

[0098] One advantage of using only or primarily using image data from a composite image (e.g., a composite monochrome image) for training the defect detection unit 140dp in training mode and / or for analysis in run mode is that the dataset is smaller compared to using complete blue and red image datasets (which may enable faster processing, for example). In some implementations, some or all of the acquired / generated image data (e.g., any of the image data in Figures 8A to 8F) may be provided to the defect detection unit 140dp for training. Each image may contain some unique data that can improve training.

[0099] Figure 9 illustrates the use of one or more video tools to perform measurement operations on an image of a workpiece containing a defect (for example, to determine the dimensions of the defect). As illustrated, in the case of image 902 (for example, which may be the same as or identical to the image in Figure 8E or Figure 8F) containing defect 904 (for example, which may be the same as or identical to defect 806 shown in Figure 8E or Figure 8F), the video box tool 906 includes scanlines 908 (for example, this may further or instead represent video point tools, etc.) which are used to determine the position, dimensions, and / or other aspects of the edges of defect 904. In various exemplary implementations, the video box tool 906 can be resized, positioned, and rotated until the box tool 906 indicates or defines a region of interest (for example, a region within the box tool 906), and the edges of defect 904 can be determined using the arrows (for example, representing scanlines, point tools, etc.) shown in Figure 9. In various exemplary embodiments, the videobox tool 906 can typically use one or more conventional edge gradients along the edges of defects 904 within the region of interest, and the edges of defects 904 can be determined based on the local magnitude of edge gradients along various scanlines 908, etc.

[0100] In various exemplary implementations, such measurement operations may include performing specific morphological filtering or other filtering (e.g., to distinguish the edges of a defect from other features of the workpiece; specific types of filtering are described in U.S. Patent No. 7,522,763, which is incorporated herein by reference in its entirety). Alternatively, various other processes may be performed, such as thresholding and / or creating a binary image (e.g., to form a defect mask workpiece image, as shown in Figure 8E, on which the measurement operations can be performed). As shown in Figure 9, in the display area included in the image, the box tool 906 with scanline 908 is used to determine the edge location of the defect 904 (e.g., the outer edge or perimeter). Based on such determination, the video tool and / or other measurement operations may include determining the dimensions D2 of the defect (e.g., corresponding to the length and / or other dimensions of the defect 904). In various exemplary implementations, the box tool 906, scanline 908, and / or other video tools and / or measurement operations can be used to determine other dimensions of the defect 904 (e.g., width, depth, etc.). For example, as mentioned above, the video tool unit 143 may include a Z-height measuring tool unit 143z, and the Z-height dimension of the defect 904 can be determined using the corresponding video tool or operation (for example, determining the depth of a scratch on other parts or features of the workpiece surface).

[0101] Figure 10 shows a flowchart of Method 1000 for operating a workpiece inspection and defect detection system similar to the workpiece inspection and defect detection system of Figure 1. During operation, the workpiece inspection and defect detection system inspects multiple workpieces. At least some of the workpieces to be inspected may include first and second materials. The first and second materials have first and second reflectance profiles (for example, with respect to a first color light corresponding to a first color channel, the reflectance of the first material is greater than that of the second material, and with respect to a second color light corresponding to a second color channel, the reflectance of the second material is greater than or equal to that of the first material). For example, at least some of the workpieces to be inspected may include aluminum and copper. These have reflectance profiles such that with respect to blue light corresponding to a blue channel, the reflectance of aluminum is greater than that of copper, and with respect to red light corresponding to a red color channel, the reflectance of copper may be greater than or equal to that of aluminum.

[0102] Method 1000 begins at 1010. In 1010, the work inspection and defect detection system acquires training mode work images and obtains multiple sets of training mode work image data. Each set of training mode work image data includes a first color channel training mode work image data corresponding to a first color channel and a second color channel training mode work image data corresponding to a second color channel. For example, using the light source 230, objective lens 250, and camera configuration 260 of the work inspection and defect detection system 100, a first set of training mode work images (for example, each image corresponds to the same field of view, such that the camera configuration 260 and the work are in the same position relative to each other) can be acquired to obtain a corresponding first set of training mode work image data. Similarly, using the light source 230, objective lens 250, and camera configuration 260 of the work inspection and defect detection system 100, a second set of training mode work images (for example, each image corresponds to the same field of view) similar to the images shown in Figures 8A and 8C can be acquired to obtain a corresponding second set of training mode work image data. Next, method 1000 proceeds to 1020.

[0103] In step 1020, the defect detection unit is trained based at least partially on multiple sets of training mode work image data acquired in step 1010. For example, the defect detection unit 140dp can be input image data corresponding to images similar to those shown in Figures 7A, 7C, 8A, and 8C, and / or image data corresponding to composite images generated based on these images (e.g., the image in Figure 8F), and this image data can be used to train the model implemented by the defect detection unit 140dp. Next, method 1000 proceeds to step 1030.

[0104] In various implementations, training of the defect detection unit in 1020, which is at least partially based on multiple sets of training mode work image data, includes generating corresponding training mode composite image data used for training the defect detection unit, using the corresponding first and second color channel training mode work image data for each set of training mode work image data. For example, with respect to the sets of training mode work image data corresponding to the images shown in Figures 8A and 8C, the defect detection unit 140dp can generate training mode composite image data corresponding to the composite image shown in Figure 8F, which is used for training the defect detection unit 140dp.

[0105] In various implementations, in 1020, for each set of training mode work image data, a plurality of ratios between the corresponding first and second color channel training mode work image data are determined. In various implementations, the determination of the plurality of ratios between the corresponding first and second color channel training mode work image data includes determining the ratio between each pixel value of the corresponding first color channel training mode work image data and the corresponding pixel value of the corresponding second color channel training mode work image data. In various implementations, training of the defect detection unit based at least partially on a plurality of sets of training mode work image data includes, for each set of training mode work image data, determining the corresponding pixel value of the corresponding training mode composite image data used for training the defect detection unit, using the ratio of the pixel values ​​of the corresponding first and second color channel training mode work image data. For example, with respect to a set of training mode work image data corresponding to the images shown in Figures 8A and 8C, the defect detection unit 140dp can determine the ratio of pixel values ​​between the blue color channel image data corresponding to the image shown in Figure 8A and the red color channel image data corresponding to the image shown in Figure 8C, thereby determining the corresponding pixel values ​​of the training mode composite image data (for example, corresponding to the composite image shown in Figure 8F) used for training the defect detection unit 140dp.

[0106] In 1030, the work inspection and defect detection system acquires run-mode work images and obtains multiple sets of run-mode work image data. Each set of run-mode work image data includes a first color channel run-mode work image data corresponding to a first color channel and a second color channel run-mode work image data corresponding to a second color channel. For example, using the light source 230, objective lens 250, and camera configuration 260 of the work inspection and defect detection system 100 (for example, a similar component having the same operating procedures / conditions as a part of the system, such as a part of the vision inspection machine 12 and / or vision component unit 200, or a vision inspection machine and / or vision component unit similar to a part of the system), a first set of run-mode work images (for example, each image corresponds to the same field of view) including images similar to those shown in Figures 7A and 7C can be acquired, and a corresponding first set of run-mode work image data can be acquired. Similarly, using the light source 230, objective lens 250, and camera configuration 260 of the work inspection and defect detection system 100 (for example, or similar components having similar operating procedures / conditions as part of the system), a second set of run-mode work images (for example, each image corresponding to the same field of view) similar to the images shown in Figures 8A and 8C can be acquired to obtain a corresponding second set of run-mode work image data. Next, method 1000 proceeds to 1040.

[0107] In step 1040, the defect detection unit trained in step 1020 is used to perform an analysis on multiple sets of run-mode work image data acquired in step 1030, at least partially, to determine defective images containing defective workpieces. For example, the defect detection unit 140dp can determine that an image similar to the image shown in Figure 8F is a defective image containing a defective workpiece. Next, method 1000 is completed.

[0108] In various implementations, the use of a trained defect detection unit in 1040 to determine a defective image containing a defective workpiece by performing an analysis at least partially based on multiple sets of run-mode work image data includes generating a corresponding run-mode composite image data for each set of run-mode work image data, using the corresponding first and second color channel run-mode work image data, which is then analyzed by the trained defect detection unit. For example, for sets of run-mode work image data corresponding to the images shown in Figures 8A and 8C, the defect detection unit 140dp can generate run-mode composite image data corresponding to the composite image shown in Figure 8F, which is then trained and analyzed by the defect detection unit 140dp.

[0109] In various implementations, in 1040, for each set of run-mode work image data, multiple ratios between the corresponding first and second color channel run-mode work image data are determined. In various implementations, the determination of multiple ratios between the corresponding first and second color channel run-mode work image data includes determining the ratio between each pixel value of the corresponding first color channel run-mode work image data and the corresponding pixel value of the corresponding second color channel run-mode work image data. In various implementations, the use of a trained defect detection unit to determine defective images containing defective work by performing an analysis at least partially based on multiple sets of run-mode work image data includes determining the corresponding pixel value of the corresponding run-mode composite image data analyzed by the trained defect detection unit for each set of run-mode work image data using the ratio of the pixel values ​​of the corresponding first and second color channel run-mode work image data. For example, with respect to a set of run-mode work image data corresponding to the images shown in Figures 8A and 8C, the defect detection unit 140dp can determine the ratio of pixel values ​​between the blue color channel image data corresponding to the image shown in Figure 8A and the red color channel image data corresponding to the image shown in Figure 8C, and then determine the corresponding pixel values ​​of the run-mode composite image data (for example, corresponding to the composite image shown in Figure 8F) that is the target of analysis by the defect detection unit 140dp.

[0110] In various implementations, Method 1000 also includes selecting components of the light source configuration to provide wavelengths of first and second color light, at least in part, based on the reflectance profiles of the first and second materials. For example, by comparing the reflectance profiles of aluminum and copper, a first wavelength (e.g., corresponding to the wavelength of blue light) can be determined where the difference between the reflectance of aluminum and copper is approximately maximized and / or greater than a first threshold (e.g., within the practical limits of the system). A second wavelength (e.g., corresponding to the wavelength of red light) can be determined where the difference between the reflectance of aluminum and copper is approximately minimized and / or less than a second threshold (e.g., within the practical limits of the system). A first LED (e.g., a blue LED) may be selected to be installed in or otherwise used within a system that outputs light having a first wavelength (e.g., as a characteristic wavelength of the spectrum output by the first LED), and a second LED (e.g., a red LED) may be selected to be installed in or otherwise used within a system that outputs light having a second wavelength (e.g., as a characteristic wavelength of the spectrum output by the second LED).

[0111] In various implementations, method 1000 may also include performing one or more measurement operations using one or more defective images (e.g., including defective workpieces). For example, the defect detection unit 140dp can use run-mode image data corresponding to the images in Figures 8A and 8C to generate run-mode composite image data corresponding to the composite image 902 shown in Figure 9, and, as previously mentioned in relation to Figure 9, can use one or more video tools to perform a measurement operation in which the dimension D2 of the defect 904 is determined.

[0112] While preferred implementations of the disclosure have been illustrated and described, a number of variations in the arrangement and operation order of the illustrated and described features will be apparent to those skilled in the art based on this disclosure. Various alternative forms can be used to implement the principles disclosed herein.

[0113] For example, the use of different color channels to enhance defect detection has been described above in relation to the images shown in Figures 7A-7E and 8A-8F. In accordance with this disclosure, different color channels can also be used to enhance defect detection in relation to the images shown in Figures 3A-3F, 4A-4E, and 5A-5D. The defects shown in Figures 3A-3F, 4A-4E, and 5B-5D are, in one specific example, scratches on a textured plate, which may be an aluminum substrate covered with copper. Each scratch may expose the aluminum substrate beneath the copper, which constitutes a defect and appears in different color images, particularly composite images generated using the ratio of blue pixel values ​​to red pixel values, as described above. More specifically, for each sample image shown in Figures 3A-3F, 4A-4E, and 5B-5D, two images are captured, including a first monochrome image captured while the workpiece is illuminated with blue light and a second monochrome image captured while the workpiece is illuminated with red light. Using the corresponding image data, a composite monochrome image data of the corresponding composite monochrome image can be generated, which has pixels having values ​​calculated based on the ratio of the values ​​of the corresponding pixels contained in the first monochrome image and the second monochrome image. Such composite monochrome image data is used in correspondence with the training mode and run mode of the defect detection unit 140dp.

[0114] All U.S. patents and U.S. patent applications referenced herein are incorporated herein by reference in their entirety. The embodiments of implementation may be modified as necessary to provide further embodiments by adopting concepts from various patents and applications. These and other modifications may be made to the embodiments in light of the detailed descriptions above. Generally, in the following claims, the terms used should not be interpreted as limiting the claims to any specific embodiment disclosed in the specification and claims, but rather as encompassing all possible embodiments, along with the full scope of equivalents given to the claims.

Claims

1. A workpiece inspection and defect detection system, A light source configuration configured to provide multiple color channels, where each color channel corresponds to a different color, and A lens configuration that receives imaging light generated from the surface of a workpiece illuminated by the aforementioned light source configuration and transmits the imaging light along the imaging optical path, A camera configuration that receives imaging light transmitted along the imaging optical path in order to provide a work image, One or more processors, A memory connected to one or more processors and storing program instructions, Equipped with, When the program instruction is executed by one or more processors, it will cause at least one or more processors to: To acquire training mode work images for obtaining multiple sets of training mode work image data, each set containing a first color channel training mode work image data corresponding to the first color channel and a second color channel training mode work image data corresponding to the second color channel, Training the defect detection unit based at least partially on the plurality of sets of training mode work image data, To acquire a run mode work image for obtaining multiple sets of run mode work image data, each set containing a first color channel run mode work image data corresponding to the first color channel and a second color channel run mode work image data corresponding to the second color channel, Using the trained defect detection unit, an analysis is performed on at least partially the plurality of sets of run-mode work image data to determine defective images that include defective workpieces. Have them do it, Training the defect detection unit based at least partially on the plurality of sets of training mode work image data is For each set of training mode work image data, For each corresponding pixel between different images, a value is determined based on the ratio between the pixel value of the first color channel training mode work image data and the pixel value of the second color channel training mode work image data. Generates training mode composite image data corresponding to the set of training mode work image data used to train the defect detection unit, each containing a pixel having the determined value. This includes, Using the trained defect detection unit, an analysis is performed on at least partially the multiple sets of run-mode work image data to determine defective images that include defective workpieces. For each set of run mode work image data, For each corresponding pixel between different images, a value is determined based on the ratio between the pixel value of the first color channel run mode work image data and the pixel value of the second color channel run mode work image data. The trained defect detection unit generates run mode composite image data corresponding to the set of run mode work image data, which includes pixels having the determined values. Including, system.

2. The system according to claim 1, wherein the light source configuration comprises a first light source that provides a first color light for the first color channel and a second light source that provides a second color light for the second color channel, and the first and second color lights correspond to different wavelengths of light.

3. The system according to claim 2, wherein the first light source is a first light-emitting diode, and the second light source is a second light-emitting diode.

4. The system according to claim 1, wherein at least a portion of the workpiece to be inspected contains first and second materials, and the presence of exposed second material in the image corresponds to a defect detectable by the trained defect detection unit.

5. The system according to claim 1, wherein at least a portion of the workpiece to be inspected includes a first and a second material, the first and second materials having first and second reflectance profiles such that, with respect to a first color light corresponding to the first color channel, the reflectance of the first material is greater than that of the second material, and with respect to a second color light corresponding to the second color channel, the reflectance of the second material is greater than or equal to that of the first material.

6. The system according to claim 5, wherein the components of the light source configuration are selected to provide the wavelengths of the first and second color light, at least partially based on the reflectance profiles of the first and second materials.

7. The light source configuration is configured to provide the first and second color light, and also to provide the third color light of the third color channel. The system according to claim 5, wherein, based on the reflectance profiles of the first and second materials, the first and second colored lights are selected to be used, and the third colored light is not selected to be used, for inspecting a workpiece having the first and second materials.

8. The system according to claim 7, wherein the third color light is selected to be used for inspecting a workpiece having the third material based on a reflectance profile of the third material that is different from the first and second reflectance profiles.

9. For each set of training mode work image data, the corresponding first and second color channel training mode work image data are generated by the respective first and second camera image planes. The system according to claim 1, wherein for each set of run mode work image data, the corresponding first and second color channel run mode work image data are generated by the respective first and second camera image planes.

10. The system according to claim 1, wherein when the program instruction is executed by the one or more processors, it causes the one or more processors to perform one or more measurement operations relating to the dimensions and / or location of a defect using one or more defective images.

11. A method for operating a workpiece inspection and defect detection system, To acquire training mode work images for obtaining multiple sets of training mode work image data, each set containing a first color channel training mode work image data corresponding to the first color channel and a second color channel training mode work image data corresponding to the second color channel, Training the defect detection unit based at least partially on the plurality of sets of training mode work image data, To acquire a run mode work image for obtaining multiple sets of run mode work image data, each set containing a first color channel run mode work image data corresponding to the first color channel and a second color channel run mode work image data corresponding to the second color channel, Using the trained defect detection unit, an analysis is performed on at least partially the plurality of sets of run-mode work image data to determine defective images that include defective workpieces. Includes Training the defect detection unit based at least partially on the plurality of sets of training mode work image data is For each set of training mode work image data, For each corresponding pixel, a value is determined based on the ratio between the pixel value of the first color channel training mode work image data and the pixel value of the second color channel training mode work image data. The system generates training mode composite image data corresponding to the set of training mode work image data used to train the defect detection unit, each of which has the determined values ​​as a pixel value. This includes, Using the trained defect detection unit, an analysis is performed on at least partially the multiple sets of run-mode work image data to determine defective images that include defective workpieces. For each set of run mode work image data, For each corresponding pixel, a value is determined based on the ratio between the pixel value of the first color channel run mode work image data and the pixel value of the second color channel run mode work image data. The trained defect detection unit generates run mode composite image data corresponding to the set of run mode work image data, each of which has the determined values ​​as a pixel value. Including, method.

12. At least a portion of the workpiece to be inspected includes first and second materials, the first and second materials having first and second reflectance profiles such that, with respect to a first color light corresponding to the first color channel, the reflectance of the first material is greater than that of the second material, and with respect to a second color light corresponding to the second color channel, the reflectance of the second material is greater than or equal to that of the first material. The method according to claim 11, further comprising selecting components of a light source configuration to provide wavelengths of the first and second color light, at least partially based on the reflectance profiles of the first and second materials.

13. The method according to claim 11, further comprising using one or more images with defects to perform one or more measurement operations relating to the dimensions and / or location of defects.

14. A workpiece inspection and defect detection system, A light source configuration configured to provide multiple color channels, where each color channel corresponds to a different color, and A lens configuration that receives imaging light generated from the surface of a workpiece illuminated by the aforementioned light source configuration and transmits the imaging light along the imaging optical path, A camera configuration that receives imaging light transmitted along the imaging optical path in order to provide a work image, Equipped with, To acquire training mode work images for obtaining multiple sets of training mode work image data, each set containing a first color channel training mode work image data corresponding to the first color channel and a second color channel training mode work image data corresponding to the second color channel, Training the defect detection unit based at least partially on the plurality of sets of training mode work image data, To acquire a run mode work image for obtaining multiple sets of run mode work image data, each set containing a first color channel run mode work image data corresponding to the first color channel and a second color channel run mode work image data corresponding to the second color channel, Using the trained defect detection unit, an analysis is performed on at least partially the plurality of sets of run-mode work image data to determine defective images that include defective workpieces. It is configured to do the following: Training the defect detection unit based at least partially on the plurality of sets of training mode work image data is For each set of training mode work image data, For each corresponding pixel, a value is determined based on the ratio between the pixel value of the first color channel training mode work image data and the pixel value of the second color channel training mode work image data. The system generates training mode composite image data corresponding to the set of training mode work image data used to train the defect detection unit, each of which has the determined values ​​as a pixel value. This includes, Using the trained defect detection unit, an analysis is performed on at least partially the multiple sets of run-mode work image data to determine defective images that include defective workpieces. For each set of run mode work image data, For each corresponding pixel, a value is determined based on the ratio between the pixel value of the first color channel run mode work image data and the pixel value of the second color channel run mode work image data. The trained defect detection unit generates run mode composite image data corresponding to the set of run mode work image data, each of which has the determined values ​​as a pixel value. Including, system.

Images