Machine vision system utilizing autofocus and inspection processes

Abstract

To provide precision metrology.SOLUTION: A machine vision system includes a vision component portion and an inspection portion. The inspection portion includes a variable focal length (VFL) lens, a VFL lens controller, an inspection portion light source, an inspection portion objective lens, and an inspection portion camera. The vision component portion performs an autofocus process which indicates z-heights of a plurality of sampling points on a surface of a workpiece for determining coarse surface profile data. The inspection portion thereafter performs an inspection process, which comprises acquiring an extended depth of field (EDOF) image for each inspection point of a plurality of inspection points on the surface of the workpiece, and for which an inspection scan path is followed which includes adjustments in relation to the distance between the inspection portion and the surface of the workpiece and is determined on the basis of at least in part, the coarse surface profile data from the autofocus process.SELECTED DRAWING: None

Description

Technical Field

[0001] The present disclosure relates to precision measurement, and more particularly, to a machine vision system that utilizes an autofocus process in combination with an inspection process (e.g., to obtain an extended depth of field image).

Background Art

[0002] Precision non-contact measurement systems such as precision machine vision systems (or, abbreviated as "vision systems") may be used to obtain precise dimensional measurements of objects and to inspect various other object characteristics, and may include a computer, a camera, an optical system with an objective lens, and a precision stage that moves to enable scanning and inspection of a workpiece. One exemplary prior art system is the QUICK VISION® series of PC-based vision systems and QVPAK® software available from Mitutoyo America Corporation (MAC, Aurora, Illinois). The features and operation of the QUICK VISION® series of vision systems and QVPAK® software are generally described, for example, in the "QVPAK 3D CNC Vision Measuring Machine User’s Guide" issued in January 2003, which is hereby incorporated by reference in its entirety. This type of system uses a microscope-type optical system including an objective lens and moves the stage and / or the optical system to provide an inspection image of the workpiece.

[0003] Quality control of workpieces including specific surface profiles such as those generated by etching, molding, and / or machining is becoming increasingly demanding with respect to throughput, measurement resolution, and accuracy. For example, in the mass production of electronic components, it is necessary to perform non-contact inspection of components or elements deposited on substrates that are not perfectly flat. The inspection process requires the detection of sub-micrometer-sized defects in these components and demands high throughput. In connection with such requirements, systems and configurations that can improve or otherwise enhance machine vision systems (e.g., in relation to improved capabilities and / or characteristics for rapidly and accurately inspecting the surface profiles of various workpieces) would be desirable.

SUMMARY OF THE INVENTION

[0004] This summary is provided to introduce a series of concepts in a simplified form, and these concepts are further described in the following detailed description. This summary is not intended to identify the key features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

[0005] According to one aspect, a machine vision system is provided, which includes a vision component unit having an optical assembly unit, a moving mechanism, and an inspection unit coupled to the optical assembly unit. The inspection unit includes a variable focal length (VFL) lens, a VFL lens controller configured to control the VFL lens to periodically modulate the refractive power of the VFL lens over a range of refractive powers at an operating frequency to periodically modulate the focusing position of the inspection unit, an inspection unit light source configured to provide light source light for illuminating the workpiece, an inspection unit objective lens having an optical axis OA' and configured to input workpiece light generated from the workpiece and transmit the workpiece light along an imaging optical path OPATH' passing through the VFL lens, where the optical axis OA' corresponds to the z-axis direction and a surface point on the surface of the workpiece has a corresponding z-height along the z-axis direction, and an inspection unit camera configured to receive the workpiece light transmitted by the VFL lens along the imaging optical path OPATH' and provide a corresponding workpiece image exposure.

[0006] The vision component unit is configured to be utilized to perform an autofocus process. The autofocus process indicates the z-heights of a plurality of sampling points SP on the surface of the workpiece to determine rough surface profile data of the surface of the workpiece, and a sampling scanning path SSP is followed for the autofocus process. The inspection unit is configured to be utilized to perform an inspection process. The inspection process includes obtaining an extended depth of field (EDOF) image for each inspection point IP of a plurality of inspection points IP on the surface of the workpiece, and an inspection scanning path ISP is followed for the inspection process. The inspection scanning path ISP includes an adjustment regarding the distance between the inspection unit and the surface of the workpiece and is determined at least partially based on the rough surface profile data from the autofocus process.

[0007] According to another aspect, a method for operating a machine vision system is provided. The machine vision system includes a vision component unit including an optical assembly unit, a moving mechanism, and an inspection unit coupled to the optical assembly unit. The inspection unit includes a variable focal length (VFL) lens, a VFL lens controller configured to periodically modulate the refractive power of the VFL lens over a range of refractive powers at an operating frequency to periodically modulate the focusing position of the inspection unit, an inspection unit light source configured to provide light source light for illuminating a workpiece, an inspection unit objective lens having an optical axis OA' and configured to input workpiece light generated from the workpiece and transmit the workpiece light along an imaging optical path OPATH' passing through the VFL lens, where the optical axis OA' corresponds to the z-axis direction and a surface point on the surface of the workpiece has a corresponding z-height along the z-axis direction, and an inspection unit camera configured to receive the workpiece light transmitted by the VFL lens along the imaging optical path OPATH' and provide a corresponding workpiece image exposure.

[0008] Generally, the method includes two steps. The first step includes utilizing the vision component unit to perform an autofocus process. The autofocus process includes determining the rough surface profile data of the surface of the workpiece by indicating the z-heights of a plurality of sampling points SP on the surface of the workpiece, and for the autofocus process, a sampling scan path SSP is followed. The second step includes utilizing the inspection unit to perform an inspection process. The inspection process includes obtaining an extended depth of field (EDOF) image for each inspection point IP of a plurality of inspection points IP on the surface of the workpiece, and for the inspection process, an inspection scan path ISP is followed. The inspection scan path ISP includes an adjustment regarding the distance between the inspection unit and the surface of the workpiece and is determined based at least in part on the rough surface profile data from the autofocus process.

[0009] According to yet another aspect, a measurement system including a visual component part is provided. The visual component part includes an optical assembly part, a movement mechanism, an inspection part coupled to the optical assembly part, a memory for storing programmed instructions, and one or more processors for executing the programmed instructions for performing operations. The inspection part includes a variable focal length (VFL) lens, and a VFL lens controller configured to control the VFL lens to periodically modulate the refractive power of the VFL lens over a range of refractive powers at an operating frequency to periodically modulate the focusing position of the inspection part. The inspection part also includes an inspection part light source configured to provide light source light for illuminating the workpiece, an inspection part objective lens having an optical axis OA' and configured to input workpiece light generated from the workpiece and transmit the workpiece light along an imaging optical path OPATH' passing through the VFL lens, where the optical axis OA' corresponds to the z-axis direction and a surface point on the surface of the workpiece has a corresponding z-height along the z-axis direction, and an inspection part camera configured to receive the workpiece light transmitted by the VFL lens along the imaging optical path OPATH' and provide a corresponding workpiece image exposure.

[0010] During operation, the one or more processors execute the programmed instructions to perform an autofocus process using the visual component part, where the autofocus process determines rough surface profile data of the surface of the workpiece by indicating the z-heights of a plurality of sampling points SP on the surface of the workpiece, and for the autofocus process, a sampling scan path SSP is followed. Executing an inspection process by using an inspection unit, the inspection process including obtaining an extended depth of field (EDOF) image for each inspection point IP of a plurality of inspection points IP on the surface of a workpiece, and for the inspection process, an inspection scanning path ISP being followed, the inspection scanning path ISP including an adjustment regarding the distance between the inspection unit and the surface of the workpiece, and being determined at least partially based on rough surface profile data from an autofocus process, and performing the operations including the above.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4A

Figure 4B

Figure 5

Figure 6

Figure 7

Figure 8A

Figure 8B

Figure 9A

Figure 9B

Figure 10

Figure 11

Figure 12A

Figure 12B

Figure 13

Figure 14

Figure 15

Embodiments for Carrying Out the Invention

[0012] FIG. 1 is a block diagram of one exemplary machine vision system 10. The machine vision system 10 includes a control computer system 14 and a vision measuring machine 12 operably connected to exchange data and control signals. The control computer system 14 is further operably connected to exchange data and control signals with a monitor or display 16, a printer 18, a joystick 22, a keyboard 24, and a mouse 26. The monitor or display 16 may display a user interface suitable for controlling and / or programming the operation of the machine vision system 10. In various implementations, it will be understood that a touch screen tablet or other computing element, etc., may be used in place of any or all of the functions of elements 14, 16, 22, 24, and 26 and / or may redundantly provide those functions.

[0013] One of ordinary skill in the art will understand that the control computer system 14 and / or the control system unit 120 (FIG. 2) and / or other control units or components described herein can generally be implemented using any suitable computing system / devices, including, for example, a distributed or network computing environment. Such a computing system or device can include one or more general-purpose or special-purpose processors (e.g., non-custom or custom devices) that execute software to perform the functions described herein. The software can be stored in a memory such as random access memory (RAM), read only memory (ROM), flash memory, etc., or a combination of such components. The software can also be stored in one or more storage devices such as an optical-based disk, a flash memory device, or any other type of non-volatile storage medium for storing data. The software can include one or more program modules, including routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types. In a distributed computing environment, the functions of the program modules can be combined or distributed across multiple computing systems or devices in either a wired or wireless configuration and can be accessed via service calls.

[0014] The vision measuring machine 12 includes a movable workpiece stage 32 and an optical imaging system 34 (e.g., as part of the vision component section 200 and including the optical assembly section 205 (FIG. 2)) that may include a zoom lens or an interchangeable objective lens. In various implementations, the workpiece stage 32 may be movable (e.g., in the x-axis and / or y-axis directions) to move a workpiece or the like disposed on the workpiece stage to the optical imaging system 34. The zoom lens or the interchangeable objective lens generally provides various magnifications for the image provided by the optical imaging system 34. Certain aspects of the machine vision system 10 are also described in U.S. Patent Nos. 7,454,053, 7,324,682, 8,111,905, and 8,111,938, each of which is hereby incorporated by reference in its entirety. As will be described in more detail below, in various implementations, the inspection section 300 (e.g., including a variable focal length (VFL) lens and configured to acquire an extended depth of field (EDOF) image or the like) may be coupled (e.g., attached and / or mounted, etc.) to a part of the optical imaging system 34 (e.g., the optical assembly section 205 (FIG. 2)), and / or may otherwise be part of the optical imaging system 34. In such implementations, the workpiece stage 32 may also be movable (e.g., in the x-axis and / or y-axis directions) to move a workpiece or the like disposed on the workpiece stage to the inspection section 300.

[0015] As will be described in more detail below, one of the main goals of the machine vision systems described herein is speed, i.e., the ability to rapidly scan a workpiece, including by utilizing a variable focal length (VFL) lens (e.g., to perform inspection operations that capture extended depth of field (EDOF) images), e.g., by rapidly moving in the x-axis / y-axis directions along one or more workpieces. One technical problem encountered is that when the workpiece has a large amount of variation in height in the z-axis direction, such variation can be greater than the focus range that can accommodate the operation of the VFL lens at the required optical resolution, and it will be understood that the higher resolution associated with a higher aperture number can significantly reduce the focus range of the VFL lens. Therefore, when scanning is performed across the workpiece (e.g., by moving in the x-axis / y-axis directions for each of a plurality of inspection points), certain sections of the workpiece may fall outside the focus range (i.e., that which can accommodate the operation of the VFL lens), and as a result, the desired EDOF images of those sections may not be acquired (e.g., due to falling outside the focus range). Certain conventional systems have been able to address such problems by determining the z-height at each inspection point and making adjustments to the z-height in order to keep each inspection point within the focus range (i.e., that which can accommodate the operation of the VFL lens), but this significantly reduces throughput.

[0016] To address such problems, as described in more detail below, a machine vision system as described herein may first utilize a vision component unit 200 to perform an autofocus process that indicates the z-height of a plurality of sampling points SP on the surface of a workpiece to determine rough surface profile data. Next, the system may utilize an inspection unit 300 (including, for example, a VFL lens such as a TAG lens) to perform an inspection process that includes obtaining an extended depth of field (EDOF) image for each inspection point on the surface of the workpiece, and an inspection scan path is followed for the inspection process. The inspection scan path includes adjustments regarding the distance between the inspection unit and the surface of the workpiece (e.g., to keep the distance within the focusing range), and is determined based at least in part on the rough surface profile data from the autofocus process.

[0017] FIG. 2 is a block diagram of a control system unit 120, a vision component unit 200, and an inspection unit 300 of a machine vision system 100 similar to the machine vision system of FIG. 1, including certain features disclosed herein. As described in more detail below, the control system unit 120 may be utilized to control the vision component unit 200 and may also be coupled to be utilized to control at least a portion of the inspection unit 300. The control system unit 120 may be configured to exchange data and control signals with the vision component unit 200 and / or the inspection unit 300.

[0018] The inspection unit 300 will be described in more detail below with respect to FIG. 5. Briefly, the inspection unit light source can emit / provide light source light 332 to illuminate one or more workpieces 20 disposed on the stage 210 (e.g., when the workpiece 20 is disposed below the inspection unit 300). The light source light 332 is reflected or transmitted from the workpiece 20 as image light 355, and the image light used for imaging passes through the inspection unit objective lens 350 for utilization as part of the operation of the inspection unit 300, as will be described in more detail below. In various implementations, the inspection unit 300 may be configured to acquire an extended depth of field (EDOF) image and may include a variable focal length (VFL) lens 370 (FIG. 5), e.g., a tunable acoustic gradient (TAG) lens that creates a lens effect using sound waves in a fluid medium, as disclosed in U.S. Patent Nos. 9,143,674 and 9,830,694. Each of the above patent documents is hereby incorporated by reference in its entirety. In various implementations, the inspection unit 300 may be coupled to the control system unit 120 via associated signal lines (e.g., as part of the bus 395).

[0019] The vision component unit 200 includes an optical imaging system 34' and a workpiece stage 210 (which may be similar or identical to, for example, the optical imaging system 34 and the stage 32 of FIG. 1). In some implementations, at least a portion of the workpiece stage 210 may be implemented as a conveyor, and one or more workpieces 20 are disposed thereon and thereby moved (e.g., to pass under the optical imaging system 34' for imaging). In various implementations, the optical imaging system 34' includes at least an optical assembly unit 205 and one or more light sources (e.g., light sources 220, 230, and / or 240).

[0020] In the example of FIG. 2, the workpiece stage 210 includes a central transparent portion 212 and is controllably movable along the x-axis and / or y-axis in a plane generally parallel to the surface of the stage on which the workpiece 20 can be placed. The workpiece stage 210 is controllably movable along the x-axis and / or y-axis by a movement mechanism 294XY configured to change the stage position of the workpiece stage 210 relative to at least a part of the imaging system 34'. In various implementations, the workpiece stage 210 may further or alternatively be controllably movable along the z-axis by using the movement mechanism 294XY. In various implementations, the movement mechanism 294XY (e.g., a controllable motor) may drive an actuator to move the workpiece stage 210 relative to the optical assembly 205 including the objective lens 250 of the imaging system 34' (e.g., to move the workpiece 20), so that different images can be obtained at different image positions, such as moving the field of view (FOV) of the optical assembly 205 of the imaging system to different parts of the workpiece 20 to obtain images of different features of the workpiece 20. It will be understood that similar operations and movements can be performed in relation to the operation of the inspection unit 300.

[0021] The encoder-based measurement unit XYMP (e.g., included in or attached to the movement mechanism 294XY, or otherwise coupled to or in proximity to the workpiece stage 210) includes one or more position encoders and provides a position measurement value indicating the position of the workpiece stage 210 (e.g., relative to the optical assembly 205 and / or the inspection unit 300). The movement mechanism 294XY (e.g., a controllable motor) and the encoder-based measurement unit XYMP are connected to the input / output interface 130 via a signal line 296XY (which may include separate signal line portions for providing control signals to and / or receiving signals from the movement mechanism 294XY and the encoder-based measurement unit XYMP).

[0022] The optical assembly unit 205 includes a camera system 260 and an interchangeable optical assembly objective lens 250. In various implementations, the optical assembly unit 205 may further include a turret lens assembly 280 having lenses 286 and 288. As an alternative to the turret lens assembly, in various implementations, a fixed or manually interchangeable zoom lens, or a zoom lens configuration, etc. may be included. In various implementations, the interchangeable optical assembly objective lens 250 can be selected from a set of fixed magnification objective lenses (e.g., a set of objective lenses corresponding to magnifications such as 0.5x, 1x, 2x or 2.5x, 5x, 10x, 20x or 25x, 50x, 100x, etc.) included as part of a variable magnification lens unit.

[0023] The optical assembly unit 205 and / or the inspection unit 300 is controllably movable along the z-axis, which is generally orthogonal to the x-axis and the y-axis, by using a moving mechanism 294Z (e.g., a controllable motor) that drives an actuator to move the optical assembly unit 205 and / or the inspection unit 300 along the z-axis (e.g., to change the focus of the image of the workpiece 20). In various implementations, the optical assembly unit 205 and / or the inspection unit 300 may further or alternatively be controllably movable along the x-axis and / or the y-axis by using the moving mechanism 294Z. An encoder-based measurement unit ZMP (e.g., one included in or attached to the moving mechanism 294Z, or coupled to or proximate to the optical assembly unit 205 and / or the inspection unit 300) includes one or more position encoders and provides a position measurement value indicating the position of the optical assembly unit 205 and / or the inspection unit 300 (e.g., relative to the stage 210 on which the workpiece 20 is disposed). The moving mechanism 294Z and the encoder-based measurement unit ZMP are connected to the input / output interface 130 via a signal line 296Z (e.g., which may include separate signal line portions for providing control signals to and / or receiving signals from the moving mechanism 294Z and the encoder-based measurement unit ZMP). For example, the control signal may be provided via the input / output interface 130 to control the moving mechanism 294Z to move the optical assembly unit 205 and / or the inspection unit 300 (e.g., to change the focus of the image, such as by changing the focusing position of the objective lens 250 or 350 with respect to the workpiece 20). In various implementations, separate moving mechanisms (e.g., similar to the moving mechanism 294Z) are included and are utilized to move the optical assembly unit 205 and the inspection unit 300 separately / independently along the z-axis (e.g., in implementations where it may be desirable to have independent control / movement along the z-axis direction for each).

[0024] For the operation of the optical assembly unit 205, one or more of the stage light source 220, the coaxial light source 230, and the surface light source 240 (e.g., ring light) may respectively emit light source light 222, 232, and / or 242 to illuminate one or more workpieces 20. For example, during image exposure, the coaxial light source 230 can emit the light source light 232 along a path including a beam splitter 290 (e.g., a partial mirror). The light source light 232 is reflected or transmitted from the workpiece 20 as image light 255, and the image light used for imaging passes through the optical assembly objective lens 250 and the turret lens assembly 280 and is collected by the camera system 260. The workpiece image exposure including the image of the workpiece 20 is captured by the camera system 260 (e.g., including a pixel array) and output to the control system unit 120 on the signal line 262. In addition to carrying the image data, the signal line 262 can carry a signal from a controller 125 (e.g., for starting image acquisition, etc.) for controlling the camera system 260.

[0025] Various light sources (e.g., light sources 220, 230, 240) may be connected to the illumination control interface 133 of the control system unit 120 via associated signal lines (e.g., buses 221, 231, 241 respectively). The control system unit 120 can control the turret lens assembly 280 to rotate along the axis 284 and select a turret lens (e.g., having a magnification such as 1x, 2x, 4x, or 6x, etc.) via the signal line or bus 281 to change the magnification of the image.

[0026] As shown in FIG. 2, in various exemplary implementations, the control system unit 120 includes a controller 125 (e.g., one or more processors or operating as part of one or more processors), an input / output interface 130, a memory 140, a workpiece program generation and execution unit 170, and a power supply unit 190. Each of these components, and additional components described below, can be interconnected by one or more data buses / control buses and / or application programming interfaces, or by direct connections between various elements. In various implementations, the memory 140 is coupled to one or more processors (e.g., of the controller 125) and stores program instructions that, when executed by the one or more processors, cause the one or more processors to perform the operations and / or functions described herein.

[0027] The input / output interface 130 includes an imaging control interface 131, a motion control interface 132, an illumination control interface 133, and a lens control interface 134. The motion control interface 132 may include a position control element 132a and a speed / acceleration control element 132b, although such elements may be integrated and / or indistinguishable. The illumination control interface 133 may include illumination control elements 133a to 133n that control, for example, the selection, power supply, and on / off switching of various corresponding light sources of the machine vision system 100.

[0028] Memory 140 may include an image file memory section 141, a workpiece program memory section 142 that may contain one or more partial programs, etc., and a video tool section 143. Certain machine vision systems generally utilize automated video inspection. U.S. Patent No. 6,542,180 (‘180 patent) teaches various aspects of such automated video inspection, which is hereby incorporated by reference in its entirety. As taught by the ‘180 patent, automated video inspection measurement equipment generally has programming capabilities that enable a user to define an automated inspection event sequence for each specific workpiece configuration. This can be implemented, for example, through a recording mode that progressively “learns” the inspection event sequence by storing a sequence of machine control instructions corresponding to the sequence of inspection operations performed by the user, either through text-based programming or with the aid of a graphical user interface, or through a combination of both methods. Such a recording mode is often referred to as a “learning mode” or a “training mode”. Once an inspection event sequence is defined in the “learning mode”, such a sequence can be used to automatically acquire (and further analyze or inspect) images of the workpiece during the “execution mode”.

[0029] Machine control instructions that include a specific inspection event sequence (i.e., how to acquire each image and how to analyze / inspect each acquired image) are generally stored as a “partial program” or “workpiece program” specific to a particular workpiece configuration. For example, the partial program defines how to position the camera relative to the workpiece, what lighting level, what magnification level, etc., for acquiring each image. Further, the partial program defines how to analyze / inspect the acquired image, for example, by using one or more video tools such as an edge / boundary detection video tool.

[0030] Video tools (or, abbreviated as "tools") and other graphical user interface functional units can be used manually to achieve manual inspection and / or machine control operations (in the "manual mode"). To create an automatic inspection program or "sub-program", their setup parameters and operations can also be recorded during the learning mode. The video tools can include, for example, edge / boundary detection tools, autofocus tools, shape or pattern matching tools, dimensional measurement tools, etc.

[0031] The video tool section 143 includes a video tool section 143a, other video tool sections (e.g., 143n) that determine the GUI, image processing operations, etc. for each corresponding video tool, and a region of interest (ROI) generator 143roi. This region of interest (ROI) generator 143roi supports automatic, semi-automatic, and / or manual operations that define various ROIs operable in various video tools included in the video tool section 143. Examples of the operations of such video tools for identifying the location of edge features and performing other workpiece feature inspection operations are described in more detail in certain of the previously incorporated references and in U.S. Patent No. 7,627,162, which is incorporated herein by reference in its entirety.

[0032] The video tool section 143 also includes an autofocus video tool 143af that determines a GUI for the focus height measurement operation, image processing operations, etc. In various implementations, the autofocus video tool 143af may further include a high-speed focus height tool that can be used to measure the focus height at high speed, as described in more detail in U.S. Patent No. 9,143,674, which is hereby incorporated by reference in its entirety. In various implementations, the high-speed focus height tool may be a special mode of the autofocus video tool 143af that can operate differently according to conventional methods for the autofocus video tool, or the operation of the autofocus video tool 143af may include only the operation of the high-speed focus height tool. High-speed autofocus and / or focus position determination for an image region or region of interest may be based on analyzing an image to determine corresponding quantitative contrast metrics for various regions according to known methods. For example, such methods are disclosed in U.S. Patent Nos. 8,111,905, 7,570,795, and 7,030,351, each of which is hereby incorporated by reference in its entirety.

[0033] In the context of the present disclosure, as is known to those skilled in the art, the term "video tool" generally refers to a relatively complex set of automated or programmed operations that can be implemented by a user of machine vision via a relatively simple user interface. For example, a video tool may include a complex pre-programmed set of image processing operations and calculations that are applied and customized in a particular instance by adjusting some variables or parameters that govern the operations and calculations. In addition to the basic operations and calculations, a video tool includes a user interface that allows the user to adjust those parameters for a particular instance of the video tool. It should be noted that visible user interface functions may sometimes be referred to as video tools, and the basic operations are implicitly included.

[0034] Generally, the memory unit 140 stores data that can be used to capture or acquire an image of the workpiece 20 and to operate the vision system component unit 200 so that the acquired image of the workpiece 20 has desired image characteristics. The memory unit 140 may also include data that defines a graphical user interface operable through the input / output interface 130. The memory unit 140 may also store inspection result data and further store data that can be used to operate the machine vision system 100 to perform various inspection and measurement operations (e.g., those partially implemented as video tools) on the acquired image, either manually or automatically, and to output the results through the input / output interface 130.

[0035] One or more display devices 136 (e.g., the display 16 of FIG. 1) and one or more input devices 138 (e.g., the joystick 22, keyboard 24, and mouse 26 of FIG. 1) can be connected to the input / output interface 130. The display device 136 and the input device 138 can be used to display a user interface that may include various graphical user interface (GUI) functions that can be used to execute inspection operations and / or to create and / or modify sub-programs, to view images captured by the camera system 260, and / or to directly control the vision component unit 200.

[0036] In various exemplary implementations, when a user creates a sub - program for a workpiece 20 using the machine vision system 100, the user operates the machine vision system 100 in a learning mode to generate sub - program instructions and provide a desired image acquisition training sequence. For example, the training sequence may include positioning specific workpiece features of a representative workpiece within the field of view (FOV), setting the light level, focusing or autofocusing, acquiring an image, and providing an inspection training sequence applied to the image (e.g., using an instance of one of the video tools for that workpiece feature). The learning mode operates such that the sequence is captured or recorded and converted into corresponding sub - program instructions. These instructions, when the sub - program is executed, cause the machine vision system to reproduce the trained image acquisition and automatically inspect that specific workpiece feature on the execution - mode workpiece (i.e., the corresponding feature at the corresponding location) that matches the representative workpiece used when creating the sub - program, for the inspection operation.

[0037] These analysis and inspection methods used to inspect features within the workpiece image are typically embodied in various video tools (e.g., video tools 143a, 143n, etc.) included in the video tool section 143 of the memory 140, as outlined above. Many known video tools, abbreviated as "tools", are included in commercially available machine vision systems such as the QUICK VISION (registered trademark) series of vision systems and related QVPAK (registered trademark) software described above.

[0038] The video tool part 143 also includes a Z-height measurement tool part 143z, which provides various operations and functions related to the Z-height measurement operation. In one implementation, the Z-height measurement tool part 143z can include a Z-height tool 143zt (including, for example, an autofocus tool, a multi-point autofocus tool, etc.). The Z-height tool 143zt, together with the Z-height tool configured in a mode for determining the best focus height and / or Z-height measurement value, can manage specific aspects of image stack acquisition and related pattern projection operations and processing. Briefly described, the Z-height measurement tool part 143z can perform at least some operations similar to known Z-height measurement tools. For example, it can operate in a learning mode and an execution mode to generate all or part of a focus curve and find peaks as the best focus position, etc.

[0039] In various implementations, the machine vision system 100 of FIG. 2 can incorporate one or more types of focus measurements (such as for controlling continuous autofocus and / or performing surface height measurement, etc., for use in, for example, automatic video inspection, etc.). As will be described in more detail below, in various implementations, such techniques can be utilized as part of an autofocus process, such as being executed by the vision component part 200. For example, in various implementations, the vision component part 200 may be configured to be utilized to execute an autofocus process that indicates the z-height of a plurality of sampling points on the workpiece surface (such as for determining rough surface profile data of the workpiece surface). In various implementations, such a process can include the use of image-based focus measurement techniques or signal-based focus measurement techniques. In various implementations, the autofocus process described herein (such as being configured to be utilized by the vision component part 200 to execute it) may further or alternatively be referred to as a focus measurement process.

[0040] Generally, image-based focus measurement can be based on the analysis of contrast in the acquired image, and for a given field of view, the image with the highest contrast generally corresponds to the most in-focus image. Since the distance from the camera corresponding to any image to the workpiece and / or the distance from the lens to the workpiece are generally known in machine vision systems, the surface height measurement can be inferred from the best in-focus image position. Such techniques for image-based focus measurement (such as may be included in machine vision system 100) are described in more detail below with respect to FIGS. 4A and 4B.

[0041] Generally, (in contrast to, e.g., image-based focus measurement), signal-based focus measurement can be based on the use of an auxiliary focus sensor that does not rely on the image of the machine vision system to determine the best focus position or surface height. Various types of auxiliary focus sensors are known, including triangulation sensors, knife-edge focus sensors, chromatic confocal sensors, Shack-Hartmann wavefront sensors, etc., which are described in U.S. Patent Nos. 4,336,997, 4,950,878, 6,184,974, 7,301,133, 7,723,657, and 7,728,961, each of which is hereby incorporated by reference in its entirety. Generally, the auxiliary focus sensor receives optical signals from the workpiece, converts them into electrical signals, and performs focus measurement by comparing them with a reference signal corresponding to the best focus (0) position of the workpiece surface height. Thus, by determining how much above or below the acquired electrical signal is from the reference signal, it can be determined how much out of focus the image is in the positive (+) or negative (-) direction along the optical axis (e.g., vertical axis) with respect to the best focus (0) position of the objective lens. In certain implementations, signal-based focus measurement techniques can generally be faster than image-based focus measurement techniques. Thus, signal-based focus measurement techniques may be well-suited for use in a tracking autofocus (TAF) sensor configured to continuously and automatically maintain focus in a machine vision system in real time. Correspondingly, as shown in FIG. 2, in some implementations, the tracking autofocus (TAF) unit 1000 can be fully or partially included in the vision component unit 200 or fully or partially included in the control system unit 120 (see 1000’). The TAF unit 1000 will be described in more detail below with respect to FIG. 3.

[0042] FIG. 3 shows an exemplary TAF assembly that can be incorporated into the machine vision system of FIG. 2, including a TAF unit 1000, a light source 230, and an optical assembly unit 205'. As will be apparent to those skilled in the art, the optical assembly unit 205' in FIG. 3 is an example of the optical assembly unit 205 described above with reference to FIG. 2 and shares some of the same elements such as the camera system 260 and the objective lens 250, but is specifically configured to properly incorporate the TAF unit 1000 into the machine vision system 100. The TAF unit 1000 includes a focus sensor 1012 and a motion control unit 1014 that adjusts the focus of the optical assembly unit 205' to a height corresponding to the current surface height 1016 of the imaged workpiece in response to a focus signal derived from the focus sensor 1012. The optical assembly unit 205' includes a collimation lens 1004, a first beam splitting surface 1006, a second beam splitting surface 1008, an objective lens 250, the nominal focal plane FP of the objective lens 250, and a camera system 260.

[0043] The motion control unit 1014 (which can control, for example, the moving mechanism 294Z and / or other moving or adjusting mechanisms) is configured to perform automatic adjustment necessary to align the nominal focal plane FP of the objective lens 250 with the workpiece surface 1016 in order to maintain the focus of the optical assembly unit 205' based on the focus signal derived from the focus sensor 1012. In FIG. 3, the objective lens 250 provides a workpiece inspection image to the camera system 260 via the image light 1020 transmitted through the second beam splitting surface 1008. The objective lens 250 also provides a reflected beam 1022' that is reflected from the second beam splitting surface 1008 along the optical axis OA of the TAF unit 1000 toward the focus sensor 1012. During operation, the radiation from the light source 230 is input to the collimation lens 1004, and the collimation lens 1004 outputs the illumination beam 1022 to the first beam splitting surface 1006. The first beam splitting surface 1006 outputs (and deflects) it so as to be input to the second beam splitting surface 1008, and the second beam splitting surface 1008 outputs (and deflects) it to the objective lens 250. The objective lens 250 inputs the illumination beam 1022 and focuses it on the nominal focal plane FP along the optical axis OA. The nominal focal plane FP is located at a distance ZNOM from the reference plane REF fixed to the objective lens 250.

[0044] As shown in FIG. 3, the workpiece surface 1016 may be located at a sensed distance ZNOM + ΔZ along the optical axis OA. The workpiece surface 1016 reflects the focused illumination beam 1022 to provide a reflected beam 1022'. The objective lens 250 receives and transmits the reflected beam 1022'. The transmitted reflected beam 1022' is then input to the second beam splitting surface 1008, and the second beam splitting surface 1008 outputs (and deflects) it to the first beam splitting surface 1006. The first beam splitting surface 1006 outputs it along the optical axis OA and inputs it to the focus sensor 1012.

[0045] The focus sensor 1012 can be based on any of the various auxiliary focus sensor techniques described above, such as the knife-edge focusing technique, the chromatic confocal technique, and the Shack-Hartmann type wavefront sensing technique. The focus sensor 1012 receives optical signals (e.g., the transmitted reflected beam 1022') from an object (i.e., the workpiece surface 1016), converts them into electrical signals (e.g., voltage V), and can compare them with a reference signal corresponding to the surface height corresponding to the best focus (0) position or the nominal focal plane FP, as shown, for example, in graph 1024. Specifically, when it is determined that the voltage V is positive or negative, the workpiece surface 1016 that generates the reflected beam 1022' is determined to be at a distance +ΔZ away from the objective lens 250 or at a distance -ΔZ closer to the objective lens 250 with respect to the nominal focal plane FP. In the example of FIG. 3, the workpiece surface 1016 is illustrated at a position +ΔZ away from the focal plane FP in the positive Z direction.

[0046] In various implementations, based on the focus signal derived from the focus sensor 1012, the motion control unit 1014 can perform the adjustments necessary to focus the optical assembly unit 205' to the Z height corresponding to the current surface height 1016. For this purpose, in the illustrated example, the motion control unit 1014 may move the objective lens 250 along the optical axis in the positive Z direction by +ΔZ, or move the workpiece stage (not shown) that supports the workpiece surface 1016 along the optical axis in the negative Z direction by -ΔZ, or move both the objective lens 250 and the workpiece stage that supports the workpiece surface 1016 to align the nominal focal plane FP with the workpiece surface 1016. In this regard, the motion control unit 1014 can control the controllable motor 294Z to operate the objective lens 250 and / or control the appropriate linear actuator and linear encoder to focus the optical assembly unit 205' to the current surface height 1016.

[0047] In yet another example, the optical assembly unit 205’ can further include a collimation adjustment element 1026, and the motion control unit 1014 can control the collimation adjustment element 1026 based on a focus signal derived from the focus sensor 1012 to provide a certain amount of collimation adjustment to the illumination beam 1022 so as to focus the optical assembly unit 205’ to a Z height corresponding to the current surface height 1016. The collimation adjustment element 1026 may be an electronically adjustable variable focus lens and may be provided between the first beam splitting surface 1006 and the second beam splitting surface 1008.

[0048] In various embodiments, the TAF unit 1000 is configured to apply a laser beam point (LP) to the current XY position of the current Z height, so that as a result, the operator can visually confirm the current XY position being tracked by the TAF unit 1000. For this purpose, a suitable laser beam source (not shown) is arranged.

[0049] In all of the above examples, the z-height of the workpiece surface (e.g., as determined at a sampling point on the workpiece surface) may be determined and recorded (e.g., as recorded in the memory unit 1013, etc.). For example, after the motion control unit 1014 is utilized to perform adjustments necessary to focus the optical assembly unit 205' to the Z-height corresponding to the current surface height 1016 (e.g., corresponding to the z-height of the sampling point on the workpiece surface 1016), the z-height of the workpiece surface / sampling point may be recorded (e.g., in the memory unit 1013, etc., as will be described in more detail below and as utilized to determine the inspection scanning path of the inspection unit 300). In various implementations, such determination and recording of the z-height of the workpiece surface / sampling point may be performed based on a focus signal derived from the focus sensor 1012 (e.g., without the need for further operations to adjust the focus position, etc.). For example, in the configuration of FIG. 3 where the workpiece surface 1016 is shown at a position +ΔZ away from the focal plane FP in the positive Z direction, it is indicated by a focus signal derived from the focus sensor 1012, and that focus signal may be utilized to determine and record the corresponding z-height of the workpiece surface / sampling point (e.g., in some implementations, this may result in an overall process that is faster than one that requires physical adjustments at each sampling point). In any case, in various implementations, such z-height data may be determined and recorded for a plurality of sampling points along the workpiece. For example, to perform an autofocus process at each of the sampling points, the optical assembly unit 205 (e.g., including the TAF unit 1000) may move along the sampling scanning path with respect to the workpiece surface. In various implementations, the sampling scanning path may include relative movement for movement to the z-height of each of the sampling points and sensing of the z-height of each of the sampling points (i.e., between the optical assembly unit 205 and the workpiece 20).

[0050] As described above, the TAF unit 1000 may be completely or partially included in the visual component unit 200, or may be completely or partially included in the control system unit 120 (see 1000’). For example, when only the focus sensor 1012 is included in the visual component unit 200 while the motion control unit 1014 is included in the control system unit 120, the TAF unit 1000 may be partially included in the visual component unit 200. As will be apparent to those skilled in the art, various operations related to the TAF unit 1000 may be implemented by a hardware configuration, a software configuration, or a combination of a hardware and a software configuration. Therefore, various hardware and / or software components and features of the TAF unit 1000 may be appropriately arranged between the visual component unit 200 and the control system unit 120 of the machine vision system 100 according to each implementation form.

[0051] Referring further to FIG. 3, the signal lines 1015A, 1015B, and / or 1015C (for example, those for controlling a moving mechanism such as the moving mechanism 294Z respectively) may be for controlling the Z height of the objective lens 250, the Z height of the workpiece stage 210, and / or the variable focal point of the collimation adjustment element 1026 respectively. Therefore, the signal lines 1015A, 1015B, and / or 1015C may be used to focus the optical assembly unit 205’ on the current Z surface height 1016, and may be connected to the input / output interface 130 when the TAF motion control unit 1014 is included in the control system unit 120. Further, when the focus sensor 1012 is included in the control system unit 120, the signal input line to the TAF focus sensor 1012 may be connected to the input / output interface 130.

[0052] In various implementations, the TAF unit 1000 is configured to generate an electrical signal (e.g., a voltage V) indicative of a current focus state relative to a reference signal corresponding to a best focus (e.g., V=0) position, as shown diagrammatically in graph 1024 of FIG. 3. For example, when V is determined to be positive or negative, V can be converted to a corresponding current Z height such that the current Z height at the current XY position is determined to differ by +ΔZ or −ΔZ from the Z height of the desired TAF focus at the current XY position. The signal / deviation signal unit of the TAF unit 1000 can be configured to generate a surface tracking error signal, such as a voltage V, that includes the difference between the current Z height at the current XY position and an indication of the Z height of the desired TAF focus at the current XY position. Alternatively or additionally, the signal / deviation signal unit may generate a TAF deviation signal, such as +ΔZ or −ΔZ, indicative of the difference between the current Z height at the current XY position and the Z height of the desired TAF focus at the current XY position. In contrast to such signal-based focus measurements (such as those of FIG. 3), image-based focus measurements (such as those based on analysis of contrast within acquired images in an image stack) may alternatively be performed in certain configurations, as described in more detail below with respect to FIGS. 4A and 4B.

[0053] FIGS. 4A and 4B show how an image stack including images at various focal lengths / distances acquired by the optical assembly portion 205 can be utilized to determine an autofocus height (i.e., autofocus position or Z L position or Z L height) of a point (e.g., a sampling point) on the workpiece surface along the Z L axis. As used herein, the "Z L axis" may correspond to the Z axis of the optical assembly portion 205 (e.g., corresponding to the optical axis OA of the objective lens 250 of the optical assembly portion 205). In various implementations, the image stack IS may be captured at an autofocus height (i.e., Z L height or Z L height) of a location (e.g., of a sampling point) on the workpiece surface.In order to determine the position), it can be obtained by the optical assembly unit 205 operating in autofocus mode (e.g., controlled by the autofocus unit 1000' and / or in relation to the operation of an autofocus tool 143af, etc.).

[0054] Specifically, FIGS. 4A and 4B show, for each of one or more points (e.g., sampling points) on the surface of the workpiece, the relative Z L position along the optical axis direction (i.e., the direction coinciding with the Z L axis of the optical assembly unit 205) (i.e., the autofocus height) related to the operation of determining the position. As shown in FIGS. 4A and 4B, the in-focus position of the optical assembly unit 205 may be moved over a range of position Z L (i), and the image stack acquisition axis may correspond to the focus axis at each image acquisition position. The optical assembly unit 205 may capture image (i) at each position Z L (i). For each captured image (i), a focus metric fm(k,i) may be calculated based on a region of interest or a sub-region of interest ROI(k) (e.g., a set of pixels) within the image (e.g., with the corresponding surface point at the center of the region of interest or sub-region of interest ROI(k)). The focus metric fm(k,i) is related to the in-focus position of the optical assembly unit 205 along the direction of the optical axis (e.g., according to the distance to the workpiece surface) when image (i) is captured. This results in focus curve data (e.g., a set of focus metrics fm(k,i) at position Z L (i), which is one type of focus peak determination dataset) that is sometimes simply referred to as a "focus curve" or an "autofocus curve". In one embodiment, the focus metric value may involve calculating the contrast or sharpness of the region of interest within the image.

[0055] The Z L position corresponding to the peak of the focus curve corresponding to the best in-focus position along the z-axis (e.g., Z L k601 in FIG. 4A) is the Z of the region of interest used to determine the focus curveL It is at a position. The image stack is shown as including 11 images (image (1) to image (11)) for illustration purposes, but it will be understood that in actual embodiments, fewer or more images may be utilized. In various implementations, the number of images may at least partially depend on the selected scanning range of the autofocus process. As one specific numerical example, for performing an autofocus process over a scanning range of 75 microns, in one implementation, the corresponding image stack may include about 8 to 10 images. Additionally, in various implementations, the images of the image stack may be smaller or larger depending on the implementation (e.g., may be related to the size of the camera's field of view and / or other factors), (e.g., including fewer or more regions of interest of a given size).

[0056] As shown by the autofocus curves generated for images (1) to (11), in the illustrated example, image (6) (i.e., the image having the corresponding focus metric fm(1,6)) appears to be close to or at the best focus. For example, features at the center of ROI(1) (e.g., features at the sampling points as shown in FIG. 12A) appear to be most in focus in image (6) compared to other images, and in the other images mentioned above, that portion of the workpiece surface appears to be increasingly out of focus and may appear increasingly blurred in images further away from image (6). When the focus metric value is based on contrast as described above, one method includes comparing the central pixel of the ROI (e.g., corresponding to the sampling point) with its adjacent pixels within the ROI from the perspective of color / brightness, etc. By finding the image with the highest overall contrast corresponding to the in-focus position when the image was acquired, an indication / measurement of the relative Z L position (i.e., the autofocus height) can be obtained along the optical axis OA and the image stack acquisition axis.

[0057] In FIG. 4B, as described above, the central region of interest ROI(1) is at a position Z along the optical axis of the optical assembly 205 L (6) in the corresponding image (6) is considered to be approximately in focus. Therefore, the surface point (e.g., sampling point) on the workpiece surface corresponding to the center of ROI(1) is assumed to approximately correspond to the in-focus position of ROI(1) in the image (6) within the image stack, and the relative position Z L (6) can be determined to be there. In some cases, the relative Z of the surface points corresponding to the centers of ROI(2) and ROI(3) L position can be determined by performing a similar process for each of the other regions of interest (regions of interest ROI(2) and ROI(3)).

[0058] In some cases, the determined peak in-focus position (i.e., the one corresponding to the autofocus height) may be between two images in the image stack. However, it will be understood that the focus peak position can be determined by interpolation or other techniques according to the fitting of the focus curve to the focus metric values determined for the images.

[0059] In various implementations, for a plurality of sampling points, the corresponding image stacks are processed to obtain a set of three-dimensional surface coordinates (e.g., the one corresponding to the general surface shape or profile of the workpiece), and the Z LA height coordinate map (e.g., a point cloud) can be determined or output. In various implementations, in connection with determining rough surface profile data of the surface of a workpiece (e.g., the z-height of the surface position between sampling points can be determined by interpolation, etc.), a relatively sparse pattern of sampling points (e.g., captured with fewer image stacks and corresponding processing) is obtained and can be processed relatively quickly. Thereafter, to assist in determining a three-dimensional inspection scanning path (e.g., including adjustments to maintain the distance between the inspection unit and the workpiece surface within the operating range of the inspection unit) for specified inspection points on the workpiece surface, an extended depth of field (EDOF) image can be obtained for the specified inspection points on the workpiece surface (e.g., the EDOF image can provide very accurate image data for inspecting the workpiece surface at the specified inspection points and can be used to determine dimensions such as corresponding workpiece features, etc.).

[0060] FIG. 5 is a schematic diagram of an inspection unit 300 including a VFL lens 370 (which may also be referred to as a VFL lens system 300 or a VFL inspection unit 300 in some implementations, additionally or alternatively). The inspection unit 300 can be operated in accordance with the principles described herein. As will be described in more detail below, the imaging optical path OPATH' (also referred to herein as the workpiece imaging optical path) includes various optical components arranged along a path that conveys image light 355 from the workpiece 20 to the camera 360. The image light is generally conveyed along the direction of the optical axis OA'. In the implementation shown in FIG. 5, all of the optical axes OA' are aligned. However, it will be understood that this implementation is merely illustrative and not limiting. More generally, the imaging optical path OPATH' can include mirrors and / or other optical elements, but can take any form operable to image the workpiece 20 using a camera (e.g., camera 360) in accordance with known principles. In the illustrated implementation, the imaging optical path OPATH' includes a VFL lens 370 (which can be included in a 4f imaging configuration) and is at least partially utilized to image the surface of the workpiece 20 during workpiece image exposure.

[0061] As shown in FIG. 5, the inspection unit 300 includes a light source 330, an exposure (strobe) time controller 333es, an objective lens 350, a tube lens 351, a relay lens 352, a VFL lens 370, a relay lens 356, a lens controller 380, a camera 360, a valid focus position (Z height vs. phase) calibration unit 373ph, and a workpiece focus signal processing unit 375 (optional). In various implementation forms, various components can be interconnected directly or by means of one or more data / control buses (for example, a system signal and control bus 395) and / or an application programming interface, etc. In various implementation forms, the components 331, 333es, 373ph, 375, and 380 can be included in the control unit 305.

[0062] As will be described in more detail below, in various implementation forms, the VFL lens controller 380 can control the drive signal of the VFL lens 370 so as to periodically modulate the refractive power of the VFL lens 370 over the range of refractive powers occurring at each phase timing within the range of periodic modulation. The camera 360 (for example, including an image detector) receives the light transmitted through the VFL lens 370 along the imaging optical path OPATH' during image exposure and provides a corresponding camera image. The objective lens 350 inputs the image light generated from the workpiece 20 during image exposure and transmits the image light through the VFL lens 370 along the imaging optical path OPATH' to the camera 360 during image exposure, providing a workpiece image in the corresponding camera image. The effective focus position EFP in front of the objective lens 350 during image exposure corresponds to the refractive power of the VFL lens 370 during that image exposure. The exposure time controller 333es is configured to control the image exposure timing used for the camera image.

[0063] Regarding the general configuration shown in FIG. 5, the light source 330 may be a "coaxial" light source or other light source configured to emit light source light 332 (e.g., stroboscopic illumination or continuous illumination) through the objective lens 350 onto the surface of the workpiece 20 along a path including a beam splitter 390 (e.g., a partial reflection mirror as part of the beam splitter). The objective lens 350 receives image light 355 (e.g., workpiece light) focused at an effective focal position EFP close to the workpiece 20 and outputs the image light 355 to the tube lens 351. The tube lens 351 receives the image light 355 and outputs it to the relay lens 352. In other implementations, a similar light source may illuminate the field of view non-coaxially. For example, a ring light source may illuminate the field of view.

[0064] In various implementations, the objective lens 350 may be an interchangeable objective lens, and the tube lens 351 may be included as part of a turret lens assembly (e.g., similar to the interchangeable objective lens 250 and turret lens assembly 223 of FIG. 2). In the implementation shown in FIG. 5, the image light 355 arising from the nominal focal plane of the objective lens 350 is focused by the tube lens 351 to form an intermediate image at the nominal intermediate image plane IIPnom. When the VFL lens 370 is in a state where it has no lens effect (no refractive power), the nominal focal plane of the objective lens 350, the nominal intermediate image plane IIPnom, and the image plane of the camera 360 form a set of conjugate planes according to known microscope imaging principles. In various implementations, any of the other lenses referred to herein may be formed from separate lenses, compound lenses, etc., or may operate in conjunction with them.

[0065] The relay lens 352 receives the image light 355 from the tube lens 351 (or more generally from the intermediate image plane in various alternative microscope configurations) and outputs it to the VFL lens 370. The VFL lens 370 receives the image light 355 and outputs it to the relay lens 356. The relay lens 356 receives the image light 355 and outputs it to the camera 360. In various implementations, the camera 360 may capture a camera image during an image exposure period (also referred to as an image exposure), for example, during the integration period of the camera 360, and provide the corresponding image data to the control system unit. Some of the camera images may include workpiece images provided during a workpiece image exposure (e.g., of a certain region of the workpiece 20). In some implementations, the image exposure (e.g., the workpiece image exposure) may be restricted or controlled by the strobe timing of the light source 330 within the image integration period of the camera 360. In various implementations, the camera 360 may have a pixel array of more than 1 megapixel (e.g., having a 1.3 megapixel, 1280×1024 pixel array, and 5.3 microns per pixel).

[0066] In the example of FIG. 5, relay lenses 352 and 356 and VFL lens 370 are designated as being included in a 4f optical configuration, relay lenses 352 and tube lens 351 are designated as being included in a Keplerian telescope configuration, and tube lens 351 and objective lens 350 are designated as being included in a microscope configuration. It will be understood that all of the illustrated configurations are merely exemplary and not limiting with respect to the present disclosure. In various implementations, the illustrated 4f optical configuration enables placement of VFL lens 370 (which may be, for example, a low numerical aperture (NA) device) at the Fourier plane of objective lens 350. This configuration can maintain telecentricity in workpiece 20 and minimize scale change and image distortion (including, for example, providing a constant magnification for each effective focus position (Z-height) of workpiece 20). A Keplerian telescope configuration (including, for example, tube lens 351 and relay lens 352) may be included between the microscope configuration and the 4f optical configuration and may be configured to provide a desired size of the projection of the effective diameter of the objective lens at the location of VFL lens 370 so as to minimize image aberrations and the like.

[0067] In various implementations, lens controller 380 may include a drive signal generator section 381, a timing clock 381', and an imaging circuit / routine 382. Drive signal generator section 381 may operate (e.g., in conjunction with timing clock 381') to provide a periodic drive signal to high-speed VFL lens 370 via signal line 380' (to provide and / or adjust an amplitude drive signal, as will be described in more detail below). In various implementations, inspection unit 300 may include a control system configurable to operate with lens controller 380 for adjusted operation (e.g., including components of control unit 305, coupled to and / or including certain components similar to those of control system section 120 of FIG. 2).

[0068] In various implementation forms, the lens controller 380 can generally perform various functions related to imaging the workpiece 20 so as to synchronize with the desired phase timing of the VFL lens 370, and controlling, monitoring, and adjusting the driving and response of the VFL lens 370. In various implementation forms, the image circuit / routine 382 performs the imaging operation of the inspection unit 300 in synchronization with the phase timing of the VFL lens 370.

[0069] In various cases, drift of the operating characteristics of the VFL lens can occur due to undesirable temperature variations. As shown in FIG. 5, in various implementation forms, the inspection unit 300 may optionally include a lens heater / cooler 337 associated with the VFL lens 370. The lens heater / cooler 337 may be configured to input a certain amount of thermal energy into the VFL lens 370 and / or execute a cooling function to facilitate heating and / or cooling of the VFL lens 370 according to some implementation forms and / or operating conditions. Additionally, in various implementation forms, a VFL lens monitoring signal can be provided by a temperature sensor 336 associated with the VFL lens 370 to monitor the operating temperature of the VFL lens 370.

[0070] Regarding the general operation of the VFL lens 370, in the various implementation forms described above, the lens controller 380 can achieve a high-speed VFL lens that periodically and rapidly adjusts or modulates its refractive power at a VFL lens resonance frequency such as 400 kHz, 250 kHz, 70 kHz, or 30 kHz, that is, at a high speed, and periodically modulates its refractive power. As shown in FIG. 5, by driving the VFL lens 370 using periodic modulation of a signal, the effective focusing position EFP of the inspection unit 300 (that is, the focusing position in front of the objective lens 350) can be quickly moved within a range Refp (for example, a focusing range, etc.) surrounded by the effective focusing position EFP1 (or EFPmax or peak focal distance Z1max+) corresponding to the maximum refractive power of the VFL lens 370 combined with the objective lens 350 and the effective focusing position EFP2 (or EFPmin or peak focal distance Z1max-) corresponding to the maximum negative refractive power of the VFL lens 370 combined with the objective lens 350. In various implementation forms, the effective focusing positions EFP1 and EFP2 can approximately correspond to specified phase timings separated by 180 degrees (for example, phase timings of 90 degrees and 270 degrees). For the sake of discussion, the middle of the range Refp can be designated as EFPnom, but can approximately correspond to the zero refractive power of the VFL lens 370 combined with the nominal refractive power of the objective lens 350. According to this description, EFPnom can approximately correspond to the nominal focal distance of the objective lens 350 in some implementation forms (for example, can correspond to the working distance WD of the objective lens 350).

[0071] In some implementations, the optional focus signal processing unit 375 may receive data from the camera 360. Also, the focus signal processing unit 375 may provide data or signals used to determine when the imaged surface area (e.g., of the workpiece 20) is at the effective focus position. For example, a group of images acquired by the camera 360 at different effective focus positions (Z heights), such as part of an image stack, may be analyzed using known "maximum contrast" or "best focus image" analysis to determine when the imaged surface area of the workpiece 20 is at the corresponding effective focus position (Z height). However, more generally, any other suitable known image focus detection configuration may be used. In any case, a workpiece focus signal processing unit 375, etc., may receive one or more images acquired during the periodic modulation of the effective focus position of the VFL lens 370 (sweeping of multiple effective focus positions) to determine the image and / or image timing that is most in focus on the target feature (e.g., of the workpiece).

[0072] In some implementations, the focus signal processing unit 375 can determine the phase timing corresponding to the best focus (or an amplitude drive signal having a related peak focus distance timing) (e.g., of workpiece features) and output that "best focus" phase timing value (or "best focus" amplitude drive signal having "best focus" peak focus distance timing) to the effective focus position calibration unit 373ph.

[0073] The effective focus position (Z height vs. phase) calibration unit 373ph can store "phase" calibration data determined by a calibration process. The effective focus position calibration unit 373ph can provide effective focus position (Z height vs. phase) calibration data that associates each effective focus position (Z height) with its respective "best focus" phase timing within the period of the resonance frequency of the VFL lens 370.

[0074] Generally speaking, the effective focus position calibration unit 373ph includes the recorded effective focus position (Z height) calibration data. In various implementation forms, the relevant recorded effective focus position (Z height) calibration data 373ph may be merged with, and / or may not be distinguishable from, a lens controller 380, a workpiece focus signal processing unit 375, or a host computer system connected to a system signal and control bus 395, etc. In various implementation forms, the exposure (strobe) time controller 333es controls the image exposure time of the inspection unit 300 (e.g., with respect to the phase timing of the periodically modulated effective focus position). More specifically, during image exposure, the exposure (strobe) time controller 333es can use the effective focus position (Z height) calibration data available in the effective focus position (Z height vs. phase) calibration unit 373ph to control the light source 330 to strobe at each respective time.

[0075] For example, the exposure (strobe) time controller 333es can control the strobe light source to strobe at each respective phase timing within one period of the standard imaging resonance frequency of the VFL lens 370, so as to obtain an image having image data corresponding to one or more specific effective focus positions within the sweep (periodically modulated) range of the VFL lens 370. In other implementation forms, the exposure time controller 333es can control the high-speed electronic camera shutter of the camera 360 to obtain an image having image data corresponding to one or more respective controlled times and / or related effective focus positions. In some implementation forms, the exposure (strobe) time controller 333es may be merged with, or may not be distinguishable from, the camera 360. It will be understood that the operations of the exposure time controller 333es and other feature parts and elements outlined above can be implemented to manage the acquisition of the workpiece image.

[0076] The inspection unit 300 includes an extended depth of field (EDOF) mode 331, which is configured to collect at least one image of a workpiece having a depth of field greater than that which can be provided by the visual component unit 200 when focused at a single focus position. Operations and components related to the extended depth of field mode are further described below with reference to FIGS. 6-9B.

[0077] The embodiment of the EDOF mode 331 shown in FIG. 5 can be used to perform computational deconvolution of a preliminary image from the inspection unit 300 to provide a relatively sharp EDOF image. For example, the inspection unit 300 is configured to collect a first preliminary image during at least one sweep of the modulated focus position over the entire EDOF focus range during image exposure and process the first preliminary image, which may be blurred, to determine a relatively sharp image. In one embodiment, the preliminary image can be processed or deconvolved using a known or predetermined point spread function (PSF) corresponding to the focus range of the preliminary image. The point spread function P(FP) characterizes the circular image of a point light source at a given distance from the imaging system as a function of the blur circle, i.e., the radius r of the blur circle and the focus position FP. The point spread function may be determined experimentally for the imaging system (e.g., the inspection unit 300), or a modeled point spread function on a function such as a pillbox or Gaussian curve may be used according to known methods, or estimated using basic diffraction principles, e.g., Fourier optics. Such point spread functions at various focal distances within the focus range can be weighted according to their expected exposure contributions or applicability. For example, when the focal distance moves during exposure, each focal distance contributes to the image exposure for the corresponding time period within that exposure, and the point spread function corresponding to that distance can be weighted accordingly. The contributions of such weighted point spread functions can be summed or integrated over the expected focus range R. Alternatively, if the focal distance change is a known function of time, the contributions of such point spread functions can be integrated over the time period corresponding to the sweep of the expected focus range R in a manner similar to the method shown with reference to Equation 3 below.

[0078] In the case of an imaging system having a modulated focus position, the integrated point spread function h follows the following relationship.

[0079]

Equation

[0080] The deconvolution of the first preliminary image may be understood as an inverse operation of deconvolving a high depth of field image exposed over a range of focus positions each having a respective duration in the exposure from the integrated point spread function h. The integrated point spread function h may be called the "blur function" in some applications. The first preliminary image can be represented as a two-dimensional function g(x, y), which is the convolution of the extended depth of field image f(x, y) (corresponding to an image array having dimensions m×n) and the integrated point spread function h as follows. g(x,y)=f*h=Σ m Σ n f(m,n)h(x - m,y - n) Equation 2

[0081] In the frequency domain, this convolution can be represented by the product of the Fourier transforms of f and h, denoted as F and H. G = F·H Equation 3

[0082] The Fourier transforms of f and h can be efficiently determined using the fast Fourier transform (FFT) algorithm. The EDOF image (in the frequency domain) can be determined by processing (i.e., multiplying) the image G by the reciprocal of H, denoted as H here. r The reciprocal H rIt can be calculated by several known methods. For example, the simple pseudo-inverse matrix of H can be determined by the following formula.

[0083]

Number

[0084]

Number

[0085] A more robust alternative to the pseudo-inverse matrix may be calculated according to the Wiener deconvolution or the Lucy-Richardson iterative algorithm, which are described in Digital Image Processing by Kenneth R. Castleman (Prentice-Hall, Inc., 1996), which is hereby incorporated by reference in its entirety. In addition, processing the image may include block-based noise removal.

[0086] In different embodiments, in order to provide a relatively sharp EDOF image in real time, deconvolution may be optically performed using a passive optical filter disposed on the Fourier plane of the EDOF imaging system according to the basic method of Fourier optics.

[0087] In an exemplary embodiment, the inspection unit 300 can provide a first preliminary image that is a blurred image including information acquired over the entire desired focusing range during exposure. Next, the first preliminary image can be computationally processed as outlined above to provide an extended depth of field image including a depth of field greater than the depth of field that the inspection unit 300 can provide at a single focal position (e.g., in some implementations, up to 10 times or up to 100 times greater). As a specific numerical example, if the depth of field at a single focal position can be 90 μm, the extended depth of field image provided using the same embodiment of the inspection unit 300 can be as large as 9 mm.

[0088] FIG. 6 shows an exemplary timing diagram 600 of the focus height during image exposure as can be used in one embodiment of an EDOF imaging system (e.g., inspection unit 300). Timing diagram 600 further shows the exposure time of the camera of the imaging system. Generally speaking, the EDOF image exposure, also referred to as frame exposure in the following description, can be performed by the imaging system over at least one sweep of the modulation of the focus height of the imaging system over the desired focusing range during exposure. In the specific example shown in timing diagram 600, the frame exposure is performed corresponding to at least one cycle of the periodic modulation of the focus height of the imaging system over the desired focusing range. The high-speed periodic modulation is preferably performed using the VFL lens 370 (e.g., a variable acoustic refractive index distribution type (TAG) lens). More specifically, in one embodiment, the following steps reflected in FIG. 6 are repeated at least once to provide an EDOF image that is substantially in focus over an extended depth of field greater than that provided by the imaging system at a single focal position. · A step of periodically modulating the focus position (focal plane) of the imaging system over a plurality of focus positions along the focal axis direction without macroscopically adjusting the spacing between elements within the imaging system, wherein the focus position is periodically modulated within a focusing range including the surface height of the workpiece (e.g., at a frequency of at least 300 Hz), the modulating step · A step of exposing a first preliminary image during the image integration time while modulating the focus position within the focusing range Processing a first preliminary image to remove a blurred image contribution that occurs during an image integration time to provide an EDOF image that is substantially in focus over an overall depth of field greater than that provided by the imaging system at a single focal position.

[0089] In the above description, it will be understood that when the blurred image contribution is computationally removed, the first preliminary image can be a blurred image that initially includes the blurred image contribution. In this case, the first preliminary image includes the detected and / or recorded image data. Processing the first preliminary image to remove the blurred image contribution involves computational processing on the first preliminary image data to provide an EDOF image (a second or modified image) that is substantially in focus over an overall depth of field greater than that provided by the imaging system at a single focal position. Thus, in this embodiment, the first preliminary image and the provided EDOF image include different images and / or image data.

[0090] In contrast, when the blurred image contribution is removed using an optical filter and a passive Fourier image processing method, the first preliminary image and the EDOF image occur simultaneously and the first preliminary image need not be the detected or recorded image. Processing the first preliminary image to remove the blurred image contribution involves passive optical processing on the first preliminary image light input to the EDOF imaging system to provide an EDOF image that is substantially in focus over an overall depth of field greater than that provided by the imaging system at a single focal position at the output or detector of the EDOF imaging system. Thus, in such an embodiment, it can be contemplated that the first preliminary image is optically processed while passing through the EDOF imaging system and before being detected by the camera or detector of the EDOF imaging system such that the provided EDOF image is the only detected or recorded image in such an embodiment.

[0091] In some embodiments, in response to user input (e.g., using the user interface functionality of the EDOF mode 331), the control signal component associated with the nominal center of the range R of the periodic modulation is adjusted so that the periodic modulation occurs around the desired nominal center of the range. In some embodiments, such adjustment may even be controlled to vary automatically during image exposure, for example, to further extend the depth of focus beyond what is achieved by a single periodic modulation.

[0092] The timing diagram shows seven periods of modulation of the focus height for each frame exposure for illustrative purposes, but it should be understood that in various embodiments, the inspection unit 300 configured according to the principles disclosed herein may comprise an imaging system that modulates the focus height over a much larger number of periods per frame exposure. For example, an exemplary imaging system can collect video images at 60 frames per second and modulate the focus height at a rate of 70 kHz, thus providing 1,000 periods of focus height modulation per frame exposure. One advantage of such a configuration is that the timing relationship between frame exposures in the periodic modulation is not critical. For example, Equation 1 shows that the integrated point spread function used to remove the blurred image contribution depends on the focus position as a function of time over the entire image exposure. If the assumed integrated point spread function does not match the actual focus position as a function of time throughout the image exposure, the blurred image contribution is not processed in an ideal manner. If the assumed integrated point spread function is based on a complete periodic modulation of the focus over the entire depth of focus and only a single period (or a few periods) of the periodic focus modulation is used during the image exposure, the actual integrated actual focus position can be significantly "uneven" compared to the assumed integrated point spread function when the exposure ends after a non-integer number of periods. In contrast, when the accumulated number of periods is important, e.g., at least 5 periods or preferably more during the image exposure, the uneven contribution of the incomplete periods may be relatively unimportant when the exposure ends after a non-integer number of periods, and the assumed integrated point spread function operates in a nearly ideal manner.

[0093] In some embodiments, collecting a first image during at least one period of a periodically modulated focus position may include exposing the image during an integer number of periods. Based on the foregoing description, it will be understood that this can be particularly beneficial when the EDOF image exposure includes a relatively small number of periods of periodic focus modulation (e.g., 5 periods or less). For example, this can occur when the exposure time has to be made relatively short to avoid excessive exposure and / or stop of motion, etc.

[0094] In the example shown in timing diagram 600, the focus position is modulated sinusoidally. In some embodiments, the image integration time includes focus changes over the entire desired focus range (e.g., at least one period of the periodically modulated focus position as shown in FIG. 6). In some embodiments, it may be desirable to expose the image only during the more linear portions of the sinusoidal modulation. This allows for a more balanced exposure time for each height within the focus position modulation (e.g., it is possible to eliminate the relatively long focus position dwell times at the extrema of the sinusoidal focus modulation). Thus, in some embodiments, exposing the image during the image integration time includes providing illumination having intensity variations (e.g., on / off cycles or more gradual intensity variations) synchronized with the periodically modulated focus position such that each exposure contribution to the respective focus positions within the range of the periodically modulated focus position is affected differently. It will be understood that when the strobe illumination is off, the frame exposure may not receive any substantial image contribution. Timing diagram 600 shows two exemplary integration periods IPA and IPB that can be used to expose the image. The exemplary integration periods IPA and IPB exclude regions near the extrema of the sinusoidal modulation, i.e., both are at least 15 percent of the period length away from the extremum portions of the sinusoidal modulation. The integration periods IPA and IPB can be controlled by providing the corresponding strobe illumination during the frame exposure according to known methods.

[0095] FIG. 7 is a flowchart 700 showing an embodiment of a routine 700 that operates the inspection unit to perform computational deconvolution of a preliminary image from an EDOF imaging system and provide a relatively sharp EDOF image in substantially real time.

[0096] In block 710, the focus position of the inspection unit is periodically modulated over a plurality of focus positions along the focal axis direction without macroscopically adjusting the spacing between elements within the imaging system. The focus position is periodically modulated within a focus range that includes the surface height of the workpiece (e.g., in some embodiments, at a frequency of at least 300 Hz or a much higher frequency). In block 720, while modulating the focus position within the focus range, a first preliminary image is exposed during the image integration time. In various implementations, the first preliminary image may be considered of a type that includes in-focus and out-of-focus images and / or an EDOF image of image data. In block 730, data from the first preliminary image is processed to remove the blurry image contribution that occurs within the focus range during the image integration time (e.g., to provide an image that is substantially in focus over the entire depth of field greater than that provided by the imaging system at a single focus position). In various implementations, this may also be characterized by extracting a focused image from the blurry image.

[0097] When using a very high-speed periodically modulated VFL lens such as a TAG lens, since the focus position changes very rapidly, the only way that can be used to obtain an EDOF image is, for example, to continuously expose the EDOF image within the focus range of the high-speed variable focus lens as in some of the examples outlined above. However, this method of EDOF image exposure has several drawbacks in various implementation forms. For example, one drawback of the method when using a periodically modulated variable focus lens is that the focus position changes in a sinusoidal manner and does not change at a constant rate. This means that continuous (including partial continuity) EDOF image exposure is not uniform across the entire focus range, which is detrimental in many implementation forms. An alternative method of obtaining an EDOF image using such a VFL lens, which may be more desirable in certain implementation forms, is described below. The alternative method includes using a plurality of separate image exposure increments to obtain a preliminary EDOF image within the focus range according to the principles described below. Such a method can be a more adaptable, accurate, and / or robust method in various implementation forms. It should be understood that when using a very high-speed and periodically modulated VFL lens (such as a TAG lens), since the focus position can change very rapidly, significant problems regarding timing, control, and "exposure amount" may occur in an actual system. To provide a practical solution to such problems, the separate image exposure increments used as components of the EDOF image exposure are obtained over a plurality of periodic focus modulations according to the principles disclosed below.

[0098] Figures 8A, 8B, 9A, and 9B each show exemplary timing diagrams 800A, 800B, 900A, and 900B that illustrate various aspects of different image exposure implementations suitable for an EDOF imaging system (e.g., inspection unit 300). Timing diagrams 800A, 800B, 900A, and 900B are somewhat similar to timing diagram 600 in that the EDOF image exposure can be acquired during a periodic modulation of the focus height or focus position of the inspection unit over its depth of focus. However, in contrast to the implementation shown in timing diagram 600 where continuous image exposure can be used, in the implementations shown in timing diagrams 800A, 800B, 900A, and 900B, the EDOF imaging system is configured to expose a preliminary image using an image exposure that includes a plurality of separate image exposure increments according to the principles described below.

[0099] In particular, timing diagram 800A of FIG. 8A shows a periodically modulated focus position MFP of a variable focus imaging system that is periodically modulated (as shown along the time axis) over a plurality of focus positions along the focus axis direction (as shown along the focal plane Z position axis) over a depth of focus FR that is assumed to include the distance to the surface of the workpiece imaged by the variable focus imaging system. The variable focus imaging system can operate at a very high focus modulation frequency (e.g., at least 3 kHz, or 30 kHz, or a frequency above that in various implementations). As shown in FIG. 800A, the preliminary image is exposed using an image exposure that includes a plurality of separate image exposure increments EI acquired at respective focus positions FP (e.g., each of focus positions Z1 to Z8) during the camera image integration time that includes a plurality of periods of the periodically modulated focus position MFP. The reference abbreviations EI and / or FP (and / or CT for the controlled timing shown in FIG. 8B used below) may include an index "i" that designates a particular "i-th" exposure increment EI, or focus position FP, or controlled timing CT. In the case of exposure increment EI, the index "i" generally ranges from 1 to the number of separate image exposure increments included in the preliminary image exposure (e.g., in the example shown in FIG. 8A, EI 1 ~EI 8 )).

[0100] Each of the plurality of separate image exposure increments EI is determined by a respective instance of an illumination source strobe operation or a camera shutter strobe operation having respective controlled timings that define a separate focus position FP for the corresponding separate image exposure increment EI. It will be understood that a variable focus imaging system having a periodically modulated focus position will have a particular focus position at a particular timing or phase within each period of the modulation. The instantaneous phase of the periodic modulation can be known based on a drive signal for the variable focus lens, or by directly monitoring the focus position, etc. Thus, by knowing the calibration relationship between the focus position and the phase of the periodic modulation (such as that stored in the calibration unit 373ph), a strobe element (e.g., a strobe illumination source 330 or a high-speed electronic camera shutter such as that of the camera 360) can be controlled to briefly activate the exposure at a particular phase timing in order to acquire an exposure increment at the desired corresponding focus position. This principle can be understood in more detail, for example, by referring to U.S. Patent Nos. 8,194,307 and 9,143,674. These patent documents are hereby incorporated by reference in their entirety.

[0101] As shown in FIG. 800A, each controlled timing (such as represented by respective increment times T1 - T8) is distributed over a plurality of cycles of a periodically modulated focus position MFP and is configured to provide a set of discrete focus positions FP that are approximately equally spaced along the focus axis direction (such as represented by focus position values Z1 - Z8). It has been determined that equally spaced and / or "weighted" exposure contributions to the "raw" or preliminary EDOF image can be advantageous with respect to signal processing and / or computational operations that may be performed subsequently to improve the EDOF image. For example, such exposure contributions can be advantageous when performing deconvolution operations on the preliminary EDOF image by using a blur kernel that characterizes the variable focus imaging system over its entire depth of focus. Such deconvolution operations are described, for example, in International Publication No. WO 2009 / 120718 (A1), which is hereby incorporated by reference in its entirety. However, certain methods of providing equally spaced and / or weighted exposure contributions over the entire depth of focus of the EDOF image are not fast enough, not accurate (with respect to the sharpness and quality of the EDOF image), or not repeatable.

[0102] As outlined previously, when using a variable focus lens (such as a TAG lens) that is very fast and periodically modulated, significant problems can arise in an actual system with respect to timing, control, and "exposure amount" because the focus position can change very rapidly. In particular, during any given modulation, the imaging system focus position may sequentially pass through adjacent pairs of desired focus positions within a period of tens of nanoseconds, and separate exposures at such sequentially adjacent focus positions can be non - practical and / or inaccurate. To provide a practical solution to such problems, each controlled timing used to acquire a separate image exposure increment EI at the desired equally spaced focus positions FP (e.g., Z1 - Z8) is configured as shown in FIG. 8A.

[0103] In FIG. 800A of FIG. 8A, a plurality of separate image exposure increments EI 1 ~EI 8 are each determined by an instance of an illumination source strobe operation of a light source 330 (having, for example, a short strobe pulse within a range of 5 ns to 100 ns) or a camera shutter strobe operation having respective controlled timings that define respective focus positions FP (e.g., one of equally spaced focus positions Z1 to Z8). Each of the respective controlled timings (such as represented by respective increment times T1 to T8) is distributed over a plurality of cycles of a periodically modulated focus position MFP within the image integration time.

[0104] To clarify the operation with respect to FIG. 800A, during the periodically modulated focus position MFP, a separate image exposure increment EI 1 is obtained at an increment time T1 having a phase timing tz1 corresponding to a desired focus position FP = Z1. Next, the periodically modulated focus position MFP continues through two reversals in the direction of focus position change during its periodic modulation following the increment time T1. Even when the periodic modulation has a very high frequency, it is practical to obtain a separate image exposure increment EI 2 at a time T2 having a phase timing tz2 corresponding to a desired focus position FP = Z2 adjacent to Z1. The acquisition of the separate image exposure increments EI continues similarly through the acquisition of a separate image exposure increment EI8 obtained at an increment time T8 having a phase timing tz8 corresponding to a desired focus position FP = Z8. At this point, the separate image exposure increments EI 1 ~EI 8 have been obtained at each of the desired equally spaced focus positions Z1 to Z8 (forming 810A) during the image integration time. In this example, the image integration time ends at this point. It will be understood that this maintains an equal "image weighting" for each of the focus positions Z1 to Z8 in the overall preliminary EDOF image exposure, including the separate image exposure increments and / or the set 810A of equally spaced focus positions.

[0105] FIG. 800B of FIG. 8B has a certain similarity to FIG. 800A of FIG. 8A and will be understood by analogy, except as otherwise described below. The main difference between FIG. 800B and FIG. 800A is the utilization of the "double pulse" principle (e.g., correspondingly, it can provide more light for imaging at each of the focusing positions Z1 to Z8). More specifically, in FIG. 800B, the separate image exposure increment EI 1 is obtained at an increment time T1 having a phase timing tz1 corresponding to the desired focusing position FP = Z1. Then, the periodically modulated focusing position MFP continues through one reversal in the direction of the focusing position change during its periodic modulation following the increment time T1 before a "mirrored" separate image exposure increment having a phase timing corresponding to the desired focusing position FP = Z1 is obtained (note that for this, the increment time and the phase timing are not labeled for simplicity of the figure). Thus, two separate image exposure increments are provided at the increment and phase timing corresponding to the desired focusing position FP = Z1. The acquisition of the separate image exposure increment EI 1 ~EI 8 , and the corresponding mirrored separate image exposure increments are obtained at each of the desired equally spaced focusing positions Z1 to Z8 (forming 810B) during the image integration time. In this example, the image integration time ends at this point. It will be understood that this maintains an equal "image weighting" for each of the focusing positions Z1 to Z8 in the overall preliminary EDOF image exposure including the separate image exposure increments and / or the set 810B of equally spaced focusing positions. In various implementations, the image integration time may be referred to further or alternatively as the image acquisition time.

[0106] FIG. 9A and FIG. 9B have a certain similarity to FIG. 8A and FIG. 8B and will be understood by analogy, except as otherwise described below. Similar to FIG. 8A, FIG. 9A utilizes a single pulse (e.g., a single optical pulse provided by light source 330 in FIG. 5), and similar to FIG. 8B, FIG. 9B utilizes a double pulse (e.g., a double optical pulse provided by light source 330 in FIG. 5). The implementations of FIG. 9A and FIG. 9B provide separate image exposure increments for four focus positions Z1 to Z4 (e.g., compared to the eight focus positions of FIG. 8A and FIG. 8B). FIGS. 9A and 9B also show the "image integration time / EDOF cycle" periods that occur in relation to the camera trigger pulse (e.g., three camera trigger pulses are shown for each image, and three "image integration time / EDOF cycle" periods are obtained). For each "image integration time / EDOF cycle" period, in FIG. 9A, four separate image exposure increments are shown (e.g., one for each of the four focus positions Z1 to Z4), and in FIG. 9B, eight separate image exposure increments are shown (e.g., two for each of the four focus positions Z1 to Z4). In the examples of FIGS. 9A and 9B, a comparison of the three illustrated "image integration time / EDOF cycle" periods shows that they each contain the same number of separate image exposure increments for the four focus positions Z1 to Z4. However, they may sometimes start with separate image exposure increments for different focus positions (e.g., the first, second, and third periods in the examples of FIGS. 9A and 9B are shown to start with separate image exposure increments corresponding to the second, fourth, and first focus positions).

[0107] FIG. 10 is a diagram showing the dependence of image blur caused by motion on the scanning speed and the image acquisition time. As shown, the first set of data points is for an image acquisition time of 70 μs, and the second set of data points is for an image acquisition time of 130 μs. As a general principle, as the scanning speed increases (e.g., with respect to the motion speed along the inspection scanning path for acquiring an EDOF image at an inspection point on the workpiece surface), the blur distance increases accordingly. For a given scanning speed, a faster image acquisition time of 70 μs results in a smaller blur distance than a slower image acquisition time of 130 μs. For a desired maximum blur distance for operation, FIG. 10 shows how the system can be configured using a certain image acquisition time and scanning speed to achieve such characteristics. For example, if the system is configured with a maximum blur distance of 1.0 μm and an image acquisition time of 70 μs, it may be desirable to have a maximum scanning speed of 14 mm / s, and for an image acquisition time of 130 μs, it may be desirable to have a maximum scanning speed of 7 mm / s.

[0108] To support a high inspection speed, the machine vision system may move continuously (i.e., for relative movement between the inspection unit 300 and the workpiece 20). In certain implementations, it may be desirable to limit the image acquisition time / image integration time (i.e., the time during which the inspection unit 300 acquires an EDOF image of the inspection point) as much as possible to avoid resolution loss due to image blur caused by motion (e.g., as shown in FIG. 10, which shows the dependence of image blur caused by motion on the scanning speed and image acquisition time, as described above, which limits the scanning speed of the inspection unit 300 for a particular image resolution). The inspection unit light source 330 must also provide sufficient exposure for image acquisition / image integration within a short time frame (e.g., within the microsecond range). FIGS. 8A, 8B, 9A, and 9B described above show implementations for providing such exposure within a short time frame (e.g., where the light source 330 emits short light pulses in the range of 5 ns to 100 ns for each of the light pulses shown at the indicated times). FIGS. 8B and 9B show implementations that increase the illumination intensity by a factor of 2 during scanning (e.g., when following the inspection scan path ISP) compared to the implementations of FIGS. 8A and 9A.

[0109] FIG. 11 shows a machine vision system 10 / 100 similar to that of FIG. 1, including a vision measuring machine 12. The vision measuring machine 12 includes a vision component unit 200 that performs autofocus processing, an inspection unit 300 that performs inspection processing, and an X movement mechanism, a Y movement mechanism, and a Z movement mechanism. The vision component unit 200 includes an optical imaging system 34' (e.g., including at least an optical assembly unit 205 and one or more light sources) and a workpiece stage 32 / 210. The Z movement mechanism (e.g., the movement mechanism 294Z) is for causing Z movement of the optical assembly unit 205 and the inspection unit 300 with respect to the movable workpiece stage 32 / 210 (e.g., for changing the distance between the inspection unit 300 and the surface of the workpiece disposed on the movable workpiece stage 32 / 210).

[0110] In the illustrated configuration, the Y movement mechanism (e.g., movement mechanism 294XY) provides the y-axis movement of the movable workpiece stage 32 / 210, and the X movement mechanism (e.g., as part of the vision measuring machine 12) provides the x-axis movement of the optical assembly unit 205 and the inspection unit 300 with respect to the movable workpiece stage 32 / 210. In the illustrated configuration, the optical assembly unit 205 may be included in the turret (e.g., the main turret) of the optical imaging system 34 or attached in other ways, and the inspection unit 300 is attached to the side of the main turret and / or the optical assembly unit 205, whereby it may move up and down in the z-axis direction and laterally in the x-axis direction together with the optical assembly unit 205. The vision measuring machine 12 is operably connected to exchange data and control signals with the control computer system 14. The control computer system 14 is further operably connected to exchange data and control signals with a monitor or display 16.

[0111] As described above (e.g., with respect to FIG. 5), the inspection unit 300 includes a variable focal length (VFL) lens such as a TAG lens (not shown), a VFL lens controller 380 configured to control the VFL lens, and an inspection unit light source 330 configured to provide light source light for illuminating the workpiece disposed on the movable workpiece stage 32 / 210 (e.g., via an optical delivery cable). As shown in the previous figures and described above, the vision component unit 200 and the inspection unit 300 may include additional or alternative configurations or components.

[0112] FIG. 12A shows the z-height of a plurality of sampling points SP on the surface of the workpiece 20 and is a schematic diagram showing the use of the vision component unit 200 together with the optical assembly unit 205 of FIG. 11 to perform an autofocus process in which a sampling scan path SSP is followed to determine the rough surface profile data of the surface of the workpiece. In the illustrated autofocus process, the sampling scan path SSP involves moving the vision component unit 200 on the workpiece 20 (e.g., at least partially in the x-axis direction) to quickly determine the rough surface profile data of the surface of the workpiece 20. Specific operations / alternatives of such an autofocus process have been described above (e.g., with respect to FIGS. 3, 4A, and 4B), and for that reason, the details of the autofocus process being executed and the variations within the workpiece determine the x-axis movement, y-axis movement, and z-axis movement along the sampling scan path SSP. In some cases, the z-axis movement along the sampling scan path SSP may be relatively limited (e.g., mainly performed at relatively sparsely located sampling points SP), and / or the total time for movement along the sampling scan path SSP may be relatively short, and / or may otherwise be performed relatively quickly.

[0113] FIG. 12B is a schematic diagram showing the use of the inspection unit 300 of FIG. 11 for performing an inspection process that includes acquiring an extended depth of field (EDOF) image for each inspection point IP of a plurality of inspection points IP on the surface of the workpiece 20, and for this purpose, the inspection scan path ISP is followed. As shown, the inspection scan path ISP includes an adjustment regarding the distance between the inspection unit 300 and the surface of the workpiece 20 in the z-axis direction, and is determined based at least in part on the rough surface profile data from the autofocus process of FIG. 12A. The adjustment in the z-axis direction keeps the surface of the workpiece 20 at approximately the correct distance relative to the inspection unit 300 so that variations in the workpiece surface height remain within the focus range corresponding to the operation of the VFL lens. In the example of FIG. 12B, the inspection unit 300 is shown as being above the inspection point IP, and the inspection point IP is within the shown focus range of the inspection unit 300 (e.g., similar to the focus range Refp described above with respect to FIG. 5). It will be understood that the movement of the inspection unit 300 along the inspection scan path ISP (e.g., including the corresponding movement / adjustment of the inspection unit 300 along the z-axis direction) is configured to keep the inspection point IP on the surface of the workpiece 20 within the shown scan range of the inspection unit 300 (e.g., near the center of the scan range) (e.g., enabling the acquisition of the desired EDOF image at the inspection point IP). Generally, the number of inspection points IP used in the inspection process is greater (e.g., denser / more frequent) than the number of sampling points SP used in the autofocus process. Further, as shown, the inspection points IP need not be equally spaced and may be selected by the user, for example, to inspect specific locations / features on the surface of the workpiece 20, as will be described in more detail below with respect to FIG. 13.

[0114] FIG. 13 is a top view of one exemplary arrangement of the sampling points SP used in the autofocus process and the inspection points IP used in the inspection process on the surface of the workpiece 20. The example shown includes 32 sampling points SP and 2160 (2160 = 9×10×24) inspection points IP.

[0115] In an exemplary embodiment, the plurality of sampling points SP are determined based at least in part on a plurality of inspection points IP specified by a user. For example, when the plurality of inspection points IP are received, the machine vision system 10 can determine a plurality of sampling points SP that is less than the number of inspection points IP, for example, to provide a coarse sampling of the region of the workpiece surface where the inspection points IP are located to determine a general surface profile of that region. Regions of the workpiece surface that do not have inspection points IP may correspondingly not have designated sampling points SP, and in some cases, at least some of the sampling points SP may or may not be in the same location as the inspection points IP. In some instances, the sampling points SP may be determined according to a pattern and / or (as illustrated, for example, in FIG. 13) generally evenly spaced to sample a specified region. On the other hand, the inspection points IP may be for inspecting specific workpiece features and may be configured or patterned to have a shape similar to the shape of the workpiece feature being inspected, such as the circular / annular pattern of FIG. 13 where the inspection points IP are for inspecting circular / annular workpiece features.

[0116] To perform an autofocus process at each of the sampling points SP, the optical assembly unit 205 can move along a sampling scan path SSP with respect to the workpiece surface. For example, in FIG. 13, in one implementation form, the sampling scan path SSP moves from left to right along the top row of 8 sampling points SP, then moves from right to left along the next row of 8 sampling points SP, then moves from left to right along the next row of 8 sampling points SP, and then moves from right to left along the next row of 8 sampling points SP (i.e., relative movement between the optical assembly unit 205 and the workpiece 20). In contrast, in FIG. 13, in one implementation form of the inspection scan path ISP, the relative movement (i.e., relative movement between the inspection unit 300 and the workpiece 20) may include movement according to a pattern of inspection points specified by the user. As described above, the inspection scan path ISP includes movement along the z-axis direction in order to maintain the distance between the inspection unit 300 and the workpiece 20 within the scan range corresponding to the operation of the VFL lens of the inspection unit 300 (e.g., to enable the corresponding acquisition of the EDOF image by the inspection unit 300 at each inspection point IP on the workpiece surface, etc.).

[0117] As described above, the movement in the z-axis direction along the inspection scanning path ISP can be determined based at least in part on the rough surface profile data from the sampling process. In some implementations, for some or all of the inspection points, the estimated z-height of each inspection point can be determined based on the rough surface profile data. For example, for an inspection point IP having XY coordinates that fall between the XY coordinates of nearby sampling points SP, interpolation or other techniques can be used to estimate the z-height of the inspection point (or a location near the inspection point). The determination of the inspection scanning path ISP based on such an estimated z-height of the inspection point may include a z-axis / z-height adjustment when approaching the inspection point IP such that the estimated z-height of the inspection point IP is within the focusing range of the operation of the VFL lens of the inspection unit 300 (i.e., the operation for acquiring the EDOF image at the inspection point IP). The inspection scanning path ISP may be determined using such z-height adjustments that are performed as needed based on the estimated z-height (e.g., the estimated z-height of some or all of the inspection points and / or locations near the inspection points). As a result, since the inspection scanning path ISP includes XY movement over a portion of the workpiece surface having z-height variations and the distance between a particular inspection point on the workpiece surface and the inspection unit 300 varies accordingly, movement in the z-height direction (e.g., movement based on the estimated z-height of the inspection point) can be performed to adjust the distance to maintain / return it within the focusing range of the operation of the VFL lens of the inspection unit 300.

[0118] In various implementation forms, any difference between the estimated z-height of the inspection point IP and the actual z-height is intended / configured to be small enough so that the actual z-height of the inspection point falls within the focusing range of the operation of the VFL lens of the inspection unit 300, so that the desired EDOF image can be obtained at each of the inspection points IP. In relation to such desired accuracy, the number of sampling points SP (e.g., determined by the system and included as part of the sampling scan path SSP) may be more or less depending on the specific surface of the workpiece to be inspected. For example, in order to improve the accuracy of rough surface profile data (e.g., to correspondingly improve the accuracy of the estimated z-height of the inspection point IP used to determine the inspection scan path ISP), relatively fewer sampling points may be used for a workpiece surface with less z-height variation or relatively minimal z-height variation, and relatively more sampling points may be used for a workpiece surface with greater and / or more frequent z-height variation. In various implementation forms, a balance is determined between the desired speed for performing the autofocus process (e.g., the number of sampling points corresponds to the speed at which the autofocus process can be performed) and the desired accuracy of the rough surface profile data (e.g., a sufficient number of sampling points are determined and used to achieve the desired accuracy). In various implementation forms, the number of inspection points IP may exceed twice, or five times, or ten times, or twenty times the number of sampling points SP (see, for example, the example in FIG. 13).

[0119] FIG. 14 is a flowchart showing a method 1400 for operating the machine vision system 10, including steps of using the vision component unit 200 to perform an autofocus process and using the inspection unit 300 to perform an inspection process.

[0120] Specifically, step 1410 shows the z-height of a plurality of sampling points SP on the surface of the workpiece 20, and for this purpose, an autofocus process is executed following a sampling scan path SSP, including using the vision component unit 200 to determine the rough surface profile data of the surface of the workpiece 20.

[0121] Step 1420 includes executing an inspection process using the inspection unit 300. The inspection process includes obtaining an extended depth of field (EDOF) image for each inspection point IP of a plurality of inspection points IP (for example, those specified by the user) on the surface of the workpiece 20, and for this purpose, an inspection scan path ISP is followed. The inspection scan path ISP includes adjustments regarding the distance between the inspection unit 300 and the surface of the workpiece 20 (for example, adjustments using the movement mechanism 294Z), and is determined at least partially based on the rough surface profile data from the autofocus process of step 1410.

[0122] FIG. 15 is a flowchart showing an embodiment of a routine 1500 for operating the machine vision system 10 to execute an autofocus process and an inspection process. The inspection process includes obtaining an extended depth of field (EDOF) image for each inspection point IP along the inspection scan path ISP.

[0123] The machine vision system 10 may be configured as described above. In an exemplary embodiment, the machine vision system 10 enables high-speed and high-resolution inspection of complex workpieces with height variations and may include the following. i) The ability for stop-and-shoot imaging or continuous scan imaging (for example, using QUICK VISION (registered trademark)), ii) A vision component unit used for an autofocus process for high-speed and rough (low-density) surface profile measurement to account for surface height (z-axis) variations beyond the range of a variable focus imaging system, iii) An inspection unit for performing an inspection process, comprising a high-speed variable focus element (VFL / TAG) capable of acquiring an extended depth of field (EDOF) image within any range between 30 and 500 microseconds (which can be faster for higher TAG operating frequencies, for example, compared to a TAG operating frequency of 70 kHz), and illuminating the workpiece during EDOF image acquisition without or with very limited motion blur and focus blur caused by motion and the workpiece, within tens or hundreds of microseconds (for example, the actual illumination time can be less than 1 microsecond or close to 800 nanoseconds), and a light source (for example, white light, or a selected wavelength, or wavelength range) that enables recording an image of the workpiece with a good signal-to-noise ratio (SNR) and minimal motion blur, and iv) A function (for example, QVSTREAM (registered trademark)) that enables continuous workpiece scanning and image alignment during motion at precisely defined points in 3D.

[0124] In some implementations, certain workpieces (e.g., dark, highly scattering, etc.) may require relatively more light (e.g., 10 times, 100 times, etc.) for imaging, especially for high throughput. Thus, in certain implementations, the light source 330 may be configured to include a brighter light source technology (e.g., laser, supercontinuum laser, superluminescent diode, etc.). Using such technology, the light source 330 may be enabled to have a high repetition rate (e.g., a repetition rate of 1 MHz+ such that all EDOF planes can be captured in a single VFL / TAG cycle). As some related concepts, it may also be desirable to utilize relatively bright sensor technology in relation to the AF and TAF sensors / processes as described above with respect to FIG. 3 and FIGS. 4A and 4B. For example, a TAF sensor using a laser may be included in such an implementation.

[0125] The routine 1500 for operating the machine vision system 10 generally includes six steps. Step 1510 includes fixing the workpiece 20 on the machine vision system 10 (e.g., on the movable workpiece stage 32 / 210). Step 1520 includes executing a program to determine the XYZ partial coordinate system of the machine vision system 10 (e.g., to support the precise adjustment of the relative position between the optical imaging system 34 and the surface of the workpiece 20). Step 1530 includes receiving user input specifying the inspection points IP on the workpiece 20 for use when executing the inspection process. In an exemplary embodiment, when multiple inspection points IP are received, the machine vision system 10 determines a plurality of sampling points SP that is less than the number of inspection points IP and is determined to provide a rough sampling of the area of the workpiece surface where the inspection points IP are located. Step 1540 includes using the vision component section 200 and executing an autofocus process using the determined plurality of sampling points SP. Step 1550 includes executing the inspection process using the inspection points IP that may be input in step 1530 above. For example, the inspection process may include moving from one inspection point IP to the next and acquiring and storing EDOF images. Step 1560 includes processing the stored EDOF images and reporting the results of the inspection process. For example, features of the workpiece of interest are extracted, measured, and / or detected using various algorithms and may be reported in various forms.

[0126] The following describes various exemplary embodiments of the present disclosure having various features and elements annotated with reference numbers found in FIGS. 1-15. It should be understood that these reference numbers are added to illustrate the exemplary embodiments and that the features and elements are not limited to the specific embodiments shown in FIGS. 1-15.

[0127] In various implementation forms, the moving mechanism 294Z (for example, as shown in FIG. 2) is configured to move the inspection unit 300 in the z-axis direction as part of the movement along the inspection scanning path for adjusting the distance between the inspection unit 300 and the surface of the workpiece 20. The moving mechanism 294Z may also be configured to move the optical assembly unit 205 in the z-axis direction, and for this purpose, the inspection unit 300 is attached to the optical assembly unit 205 (for example, attached to a frame).

[0128] In various implementation forms, the periodic modulation of the focus position of the inspection unit 300 covers the focus range Refp, and the adjustment of the distance between the inspection unit 300 and the surface of the workpiece 20 (for example, the adjustment using the moving mechanism 294Z) is performed during the inspection process so that each inspection point IP on the workpiece surface is within the focus range Refp of the inspection unit 300 when an EDOF image is acquired for the inspection point IP. As shown in FIG. 5, by driving the VFL lens 370 using the periodic modulation of the signal, the effective focus position EFP of the imaging system (that is, the focus position in front of the objective lens 350) can be quickly moved within a range Refp (for example, a focus range or an autofocus search range, etc.) surrounded by the effective focus position EFP1 (or EFPmax or peak focal distance Z1max+) corresponding to the maximum positive refractive power of the VFL lens 370 combined with the objective lens 350 and the effective focus position EFP2 (or EFPmin or peak focal distance Z1max-) corresponding to the maximum negative refractive power of the VFL lens 370 combined with the objective lens 350.

[0129] In various implementation forms, the sampling scanning path SSP is different from the inspection scanning path ISP. The inspection scanning path ISP may include movement to coordinates where the sampling scanning path SSP does not include movement (for example, the XYZ coordinates for imaging the inspection point IP).

[0130] In various implementation forms, the optical assembly unit 205 includes an optical assembly objective lens 250 having an optical axis OA. The optical assembly unit 205 may further include an optical assembly light source 230 configured to provide the light source light 232 for illuminating the workpiece 20. The optical assembly objective lens 250 may be configured to input the workpiece light 255 generated from the surface of the workpiece 20 illuminated by the light source light 232 and transmit the workpiece light 255 along the imaging optical path OPATH of the optical assembly unit. The optical axis OA corresponds to the z-axis direction, where the surface point on the surface of the workpiece 20 has a corresponding z-height along the z-axis direction. The optical assembly unit 205 may further include an optical assembly camera (e.g., of the camera system 260) configured to receive the workpiece light 255 transmitted along the imaging optical path OPATH and provide an image of the workpiece surface.

[0131] In various implementation forms, the autofocus process configured to be utilized for the optical assembly unit 205 of the vision component unit includes at least one of collecting an image stack or utilizing a tracking autofocus sensor (such as included in the tracking autofocus unit 1000 of FIG. 3). To collect the image stack, in various implementation forms, at each sampling point SP, the movement mechanism 294Z can be utilized to move the optical assembly unit 205 along the z-axis direction to capture an image stack including images captured at different focus positions. For each image, the focus metric may be calculated based on the image and may be associated with the corresponding position along the z-axis (e.g., of a camera or other part of the optical assembly unit 205 or a reference point) at the time the image was captured. The focus curve may be determined as corresponding to the calculated focus metric, and the peak of the focus curve may correspond to the best focus position along the z-axis.

[0132] In various implementations, the acquisition of each extended depth of field (EDOF) image by the inspection unit 300 (i.e., for each of the inspection points IP) includes operating the VFL lens controller 380 to periodically modulate the focus position of the inspection unit 300, exposing a first preliminary image using an image exposure including a plurality of separate image exposure increments acquired at respective separate focus positions during an image integration time including a plurality of periods of the periodically modulated focus position, and processing the first preliminary image to remove the blurred image contribution occurring within the focus range during the image integration time to provide an extended depth of field (EDOF) image that is substantially in focus over the entire depth of field greater than that provided by the inspection unit 300 at a single focus position. In various implementations, each of the plurality of separate image exposure increments is determined by a respective instance of the strobe operation of the inspection unit light source 330. In various implementations, the EDOF image can be further processed. For example, the image may be segmented, the blurred image contribution removed, and analyzed (e.g., features of interest may be extracted, measured, and / or detected using various algorithms, etc.).

[0133] In various implementations, the machine vision system 10 / 100 includes a vision measuring machine 12 having a movable workpiece stage 32 / 210 and an optical imaging system 34 (e.g., including the vision component unit 200, the moving mechanism 294Z, etc.). In the examples of FIGS. 12A and 12B, it will be understood that the workpiece 20 can be disposed on the stage 32 / 210 (e.g., to be moved by the moving mechanism 294XY).

[0134] It will be understood that the principles disclosed and claimed herein can be readily and desirably combined with various features disclosed in the incorporated references. The various implementations described above can provide further implementations by combining them. All U.S. patents and U.S. patent applications referenced herein are hereby incorporated by reference in their entirety. Aspects of the implementations can be modified, as needed, to provide further implementations using concepts from various patents and applications. In light of the above detailed description, these and other changes can be made to the implementations. Generally, the terms used in the following claims should not be construed as limiting the claims to the specific implementations disclosed in the specification and claims, but should be construed to include all possible implementations along with the full scope of equivalents that give rise to such claims.

Claims

1. A vision component unit including an optical assembly unit, a moving mechanism, and an inspection unit coupled to the optical assembly unit, a variable focal length (VFL) lens, a VFL lens controller configured to control the VFL lens so as to periodically modulate the refractive power of the VFL lens over a range of refractive powers at an operating frequency in order to periodically modulate the focusing position of the inspection unit, an inspection unit light source configured to provide light source light for illuminating a workpiece, an inspection unit objective lens having an optical axis, inputting workpiece light generated from the workpiece, and configured to transmit the workpiece light along an imaging optical path passing through the VFL lens, wherein the optical axis corresponds to the z-axis direction, and a surface point on the surface of the workpiece has a corresponding z-height along the z-axis direction, an inspection unit camera configured to receive the workpiece light transmitted by the VFL lens along the imaging optical path and provide a corresponding workpiece image exposure, and an inspection unit, comprising, the vision component unit is configured to be used to execute an autofocus process, the autofocus process shows the z-heights of a plurality of sampling points SP on the surface of the workpiece in order to determine rough surface profile data of the surface of the workpiece, and a sampling scanning path SSP is followed for the autofocus process, the inspection unit is configured to be used to execute an inspection process, the inspection process includes obtaining an extended depth of field (EDOF) image for each inspection point IP of a plurality of inspection points IP on the surface of the workpiece, and an inspection scanning path ISP is followed for the inspection process, the inspection scanning path ISP includes an adjustment regarding the distance between the inspection unit and the surface of the workpiece, and is determined at least partially based on the rough surface profile data from the autofocus process, a machine vision system.

2. The machine vision system according to claim 1, wherein the moving mechanism is configured to move the inspection unit in the z-axis direction for the adjustment regarding the distance between the inspection unit and the surface of the workpiece as a part of the movement along the inspection scanning path ISP.

3. The moving mechanism is also configured to move the optical assembly unit in the z-axis direction. Therefore, the inspection unit is attached to the optical assembly unit, and the machine vision system according to claim 2.

4. The periodic modulation of the focus position of the inspection unit extends over the focus range During inspection, an adjustment regarding the distance between the inspection unit and the surface of the workpiece is performed, and when acquiring the EDOF image for the inspection point IP, each inspection point IP on the surface of the workpiece is within the focus range of the inspection unit. The machine vision system according to claim 1.

5. The sampling scanning path SSP is different from the inspection scanning path ISP, and the machine vision system according to claim 1.

6. The inspection scanning path ISP includes movement to coordinates where no movement occurs in the sampling scanning path SSP, and the machine vision system according to claim 5.

7. The number of inspection points IP is more than twice the number of sampling points SP, and the machine vision system according to claim 1.

8. The optical assembly unit includes an optical assembly objective lens having an optical axis, and the machine vision system according to claim 1.

9. The optical assembly unit further includes an optical assembly light source configured to provide optical assembly light source light for illuminating the workpiece. The optical assembly objective lens inputs workpiece light generated from the surface of the workpiece illuminated by the optical assembly light source light and is configured to transmit the workpiece light along the imaging optical path of the optical assembly unit. The machine vision system according to claim 8.

10. The optical assembly unit further includes an optical assembly camera configured to receive the workpiece light transmitted along the imaging optical path and provide an image of the surface of the workpiece. The machine vision system according to claim 9.

11. The autofocus process configured to be utilized by the optical assembly unit to perform it is collecting an image stack, or using a tracking autofocus sensor, including at least one of these, and the machine vision system according to claim 1.

12. Obtaining each extended depth of field (EDOF) image by the inspection unit includes: Operating the VFL lens controller so as to periodically modulate the focus position of the inspection unit; During an image integration time including a plurality of periods of the periodically modulated focus position, exposing a first preliminary image using image exposures including a plurality of separate image exposure increments obtained at respective separate focus positions; Processing the first preliminary image to remove the blurred image contribution occurring within the focus range during the image integration time, so that the inspection unit provides an extended depth of field (EDOF) image substantially focused over an entire depth of field greater than the depth of field provided at a single focal position; The machine vision system according to claim 1, comprising: **Claim 13** The machine vision system according to claim 12, wherein each of the plurality of separate image exposure increments is determined by a respective instance of a stroboscopic operation of the inspection unit light source. **Claim 14** The machine vision system according to claim 1, wherein the VFL lens is an adjustable acoustic gradient (TAG) lens. **Claim 15** A method of operating a machine vision system, the machine vision system comprising: A vision component unit including an optical assembly unit, A moving mechanism, and An inspection unit coupled to the optical assembly unit, the inspection unit comprising: A variable focal length (VFL) lens; A VFL lens controller configured to control the VFL lens so as to periodically modulate the refractive power of the VFL lens over a range of refractive powers at an operating frequency to periodically modulate the focus position of the inspection unit; An inspection unit light source configured to provide light source light for illuminating a workpiece; An inspection unit objective lens having an optical axis, inputting workpiece light generated from the workpiece, and configured to transmit the workpiece light along an imaging optical path passing through the VFL lens, wherein the optical axis corresponds to the z-axis direction, and a surface point on the surface of the workpiece has a corresponding z-height along the z-axis direction; An inspection unit camera configured to receive the workpiece light transmitted by the VFL lens along the imaging optical path and provide a corresponding workpiece image exposure. The method comprising: Comprising: The method includes: Executing an autofocus process using the visual component unit, wherein, to determine the rough surface profile data of the surface of the workpiece, the z-height of a plurality of sampling points SP on the surface of the workpiece is indicated, and a sampling scan path SSP is followed for the autofocus process Executing an autofocus process; Executing an inspection process using the inspection unit, wherein, for each inspection point IP of a plurality of inspection points IP on the surface of the workpiece, an extended depth of field (EDOF) image is acquired, and following an inspection scan path ISP including adjustment regarding the distance between the inspection unit and the surface of the workpiece is performed, and the inspection operation path ISP is determined based at least in part on the rough surface profile data from the autofocus process Executing an inspection process; A method comprising the above.

16. The method according to claim 15, wherein the moving mechanism is used to move the inspection unit in the z-axis direction for adjustment regarding the distance between the inspection unit and the surface of the workpiece as part of the movement along the inspection scan path ISP

17. The periodic modulation of the focus position of the inspection unit is over a focus range During inspection, an adjustment regarding the distance between the inspection unit and the surface of the workpiece is performed, and when acquiring the EDOF image for the inspection point IP, each inspection point IP on the surface of the workpiece is made to fall within the focus range of the inspection unit, according to the method of claim 15

18. The sampling scan path SSP is different from the inspection scan path ISP, and the number of the inspection points IP is more than twice the number of the sampling points SP, according to the method of claim 15

19. Acquiring each extended depth of field (EDOF) image by the inspection unit comprises: Operating the VFL lens controller to periodically modulate the focus position of the inspection unit; Exposing a first preliminary image using image exposures including a plurality of separate image exposure increments acquired at respective separate focus positions during an image integration time including a plurality of periods of the periodically modulated focus position Processing the first preliminary image to remove the defocused image contribution that occurs within the depth of focus during the image integration time, and providing an extended depth of field (EDOF) image that is substantially in focus over the entire depth of field that is greater than the depth of field provided by the inspection unit at a single focal position The method according to claim 15, including this.

20. A visual component unit including an optical assembly unit A moving mechanism An inspection unit coupled to the optical assembly unit, A variable focal length (VFL) lens, and A VFL lens controller configured to control the VFL lens so as to periodically modulate the refractive power of the VFL lens over a range of refractive powers at an operating frequency in order to periodically modulate the focal position of the inspection unit An inspection unit light source configured to provide light source light for illuminating the workpiece An inspection unit objective lens having an optical axis, inputting workpiece light generated from the workpiece, and configured to transmit the workpiece light along an imaging optical path passing through the VFL lens, wherein the optical axis corresponds to the z-axis direction, and a surface point on the surface of the workpiece has a corresponding z-height along the z-axis direction An inspection unit camera configured to receive the workpiece light transmitted by the VFL lens along the imaging optical path and provide a corresponding workpiece image exposure, and an inspection unit A memory for storing programmed instructions, and One or more processors for executing the programmed instructions to perform operations, and the operations are Executing an autofocus process using the visual component unit, which shows the z-heights of a plurality of sampling points SP on the surface of the workpiece to determine rough surface profile data of the surface of the workpiece, and a sampling scan path SSP is followed for the autofocus process Executing an autofocus process Executing an inspection process using the inspection unit, wherein for each inspection point IP of a plurality of inspection points IP on the surface of the workpiece, an extended depth of field (EDOF) image is acquired, and following an inspection scanning path ISP including an adjustment regarding a distance between the inspection unit and the surface of the workpiece is performed, and the inspection scanning path ISP is determined based at least in part on the rough surface profile data from the autofocus process Executing an inspection process; A measurement system including the above.

Images