Range differentiator for autofocusing of optical imaging system

Abstract

To provide a system about a depth range differentiator to be used in autofocusing of an optical imaging system.SOLUTION: The present invention is a range differentiator 120 useful for autofocusing, and the range differentiator includes: an image generator that provides images of scenes at various physical depths; a depth differentiator that distinguishes a part of the image at the depth less than a prescribed threshold regardless of a shape of the part, and provides a depth differentiated image; and a focus distance ascertaining device that ascertains a focus distance based on the depth differentiated image. Also, the depth differentiator is configured to work so as to provide a focus score on the basis of the image at the depth equal to or more than the threshold.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] Cross - Reference to Related Applications In this document, reference is made to U.S. Provisional Patent Application No. 62 / 634,870, filed on February 25, 2018, for the invention entitled "RANGE DIFFERENTIATORS FOR AUTO - FOCUSING IN OPTICAL IMAGING SYSTEMS", the disclosure of which is hereby incorporated by reference into this document, and the priority thereof is claimed herein in accordance with 37 CFR 1.78(a)(4) and (5)(i).

[0002] This invention generally relates to optical imaging systems, and more particularly to systems and methods useful for auto - focusing (automatic focusing) in optical imaging systems.

Background Art

[0003] Various types of auto - focus systems used in optical imaging systems are known in the prior art.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] This invention provides a system and method for depth range differentiation used in auto - focusing in an optical imaging system.

Means for Solving the Problems

[0006] That is, in a preferred embodiment of the present invention, a range differentiator useful for autofocus is provided, and the range differentiator includes an image generator that provides an image of a scene at various physical depths, a depth differentiator that differentiates a portion of the image having a depth less than a predetermined threshold without regard to the shape of the portion to provide a depth differentiated image, and a focus distance ascertainer that ascertains a focus distance based on the depth differentiated image.

[0007] In a preferred embodiment of the present invention, the image generator includes a feature specific illuminator that illuminates the scene during acquisition of the image. Further, the depth differentiator operates to distinguish a portion of the image having a depth less than the predetermined threshold and a portion of the image having a depth greater than or equal to the predetermined threshold based on differences in optical properties therebetween under illumination by the feature specific illuminator.

[0008] In a preferred embodiment of the present invention, the feature specific illuminator includes a UV illumination source, and the depth differentiator operates to distinguish the portions of the image based on differences in fluorescence therebetween. In another example, the feature specific illuminator includes a dark field illumination source and a bright field illumination source, and the depth differentiator operates to distinguish the portions of the image based on differences in reflectance therebetween.

[0009] Preferably, the focus distance ascertainer operates to ascertain the focus distance based on one of the portion of the image having a depth less than the predetermined threshold and the portion of the image having a depth greater than or equal to the predetermined threshold.

[0010] In a preferred embodiment of the present invention, the range discrimination device further includes an image focus analysis device that operates to provide a focus score based on a portion of the image having a depth equal to or greater than the predetermined threshold, and the focus distance confirmation device operates to confirm the focus distance based on the focus score. Further, the image focus analysis device includes an illuminator that illuminates the scene using illumination that emphasizes an imaged texture (imaging texture) of the portion of the image having a depth equal to or greater than the predetermined threshold. In addition to this, the illuminator includes a dark field illuminator. Alternatively or in addition to this, the focus score is assigned regardless of the shape of the portion. In a preferred embodiment of the present invention, the focus score is individually assigned to each pixel corresponding to the portion of the image having a depth equal to or greater than the predetermined threshold.

[0011] Preferably, the portion of the image having a depth equal to or greater than the threshold is machine identifiable.

[0012] In a preferred embodiment of the present invention, the image generator includes a camera, and the depth discrimination image includes a two-dimensional image of the scene. In addition to or instead of this, the image generator includes a plenoptic camera, and the depth discrimination image includes a three-dimensional image of the scene. In a preferred embodiment of the present invention, the feature identification illuminator includes a dark field illuminator.

[0013] In a preferred embodiment of the present invention, the image generator includes a projector that projects a repeating pattern onto the scene, the depth discrimination device includes a phase analysis device that analyzes a phase shift of the repeating pattern and operates to derive a map of the physical depth based on the phase shift, and the map forms the depth discrimination image. In addition to this, the focus distance confirmation device operates to confirm the focus distance based on at least one of the physical depths.

[0014] In a preferred embodiment of the present invention, the repeating pattern includes at least one of a sinusoidal repeating pattern and a binary repeating pattern. Further, the repeating pattern has a sufficiently low spatial frequency, and the phase analyzer operates to uniquely correlate the phase shift with the physical depth. In addition or alternatively, the map of the physical depth is one of a two-dimensional map and a three-dimensional map.

[0015] Also, a range discrimination device useful for autofocus provided by another preferred embodiment of the present invention includes an image generator that provides images of scenes at various physical depths, a depth discrimination device that discriminates portions of the images at depths less than a predetermined threshold, an image focus analyzer that operates to provide a focus score based on portions of the images at depths greater than or equal to the predetermined threshold, and a focus distance confirmation device that confirms the focus distance based on the focus score.

[0016] In a preferred embodiment of the present invention, the image generator includes a feature-specific illuminator that illuminates the scene during acquisition of the image. Further, the depth discrimination device includes a UV illumination source, and the depth discrimination device discriminates the portions of the image based on differences in fluorescence therebetween. In another aspect, the feature-specific illuminator includes a combined dark field and bright field illuminator, and the depth discrimination device discriminates the portions of the image based on differences in reflectivity therebetween.

[0017] In a preferred embodiment of the present invention, the image focus analyzer includes an illuminator that illuminates the scene using illumination that emphasizes the imaging texture of the portion of the image at a depth equal to or greater than the predetermined threshold. Further, the illuminator includes a dark field illuminator. In addition or alternatively, the illuminator and the feature identification illuminator share at least one common illumination component.

[0018] In a preferred embodiment of the present invention, the focus score is assigned regardless of the shape of the portion. In addition or alternatively, the focus score is individually assigned to each pixel corresponding to the portion of the image at a depth equal to or greater than the predetermined threshold.

[0019] Preferably, the portion of the image at a depth equal to or greater than the threshold is machine-identifiable.

[0020] A range discrimination device useful for autofocus, further provided by yet another preferred embodiment of the present invention, includes a target identifier including a user interface that enables a user to identify machine-identifiable features of an object in an image, a feature detector that operates to identify at least one occurrence of the machine identifiable feature in the image regardless of the shape of the feature, and a focus distance confirmation device that confirms the focus distance for the machine-identifiable feature.

[0021] Preferably, the range discrimination device further includes a feature identification illuminator that illuminates the object during acquisition of the image.

[0022] In a preferred embodiment of the present invention, the feature specifying illuminator includes a UV illumination source, and the feature identifying device identifies the machine-identifiable features based on its fluorescence. In other embodiments, the feature specifying illuminator includes a dark field and bright field combined illuminator, and the feature identifying device identifies the machine-identifiable features based on its reflection.

[0023] In a preferred embodiment of the present invention, a range ascertainer includes an illuminator that illuminates the object using illumination that emphasizes the imaging texture of the features of the object in the image. Further, the illuminator includes a dark field illuminator.

[0024] Preferably, the illuminator and the feature specifying illuminator share at least one common illumination component.

[0025] In a preferred embodiment of the present invention, the features of the object include a conductive feature. Further, the features of the object include an indentation in the conductive feature.

[0026] A range discrimination device useful for autofocus further provided by yet another preferred embodiment of the present invention includes a first imaging modality, a first image generator that provides a first image of a scene at various physical depths, a depth discrimination device that discriminates a portion of the first image having a depth less than a predetermined threshold to provide a depth discrimination image, a focus distance confirmation device that confirms a focus distance based on the depth discrimination image, and a second imaging modality, and includes a second image generator that provides a second image of the scene automatically focused at the focus distance.

[0027] In a preferred embodiment of the present invention, the first imaging modality includes bright field and dark field combined illumination, and the second imaging modality includes dark field illumination. Further, the second image generator includes a plenoptic camera.

[0028] In a preferred embodiment of the present invention, the first imaging modality includes dark-field illumination, and the second imaging modality includes bright-field and dark-field combined illumination. Further, the first image generator includes a plenoptic camera.

[0029] An autofocus range discrimination device further provided according to still another preferred embodiment of the present invention includes a projector that projects a repeating pattern onto an object including features of various physical depths, a sensor that acquires an image of the object onto which the repeating pattern is projected, a phase analysis device that analyzes a phase shift of the repeating pattern and derives a map of the physical depths of the features based on the phase shift, and a focus analysis device that confirms a focal length for at least one of the features.

[0030] In a preferred embodiment of the present invention, the repeating pattern includes at least one of a sine-wave repeating pattern and a binary repeating pattern. In addition to or instead of this, the repeating pattern has a sufficiently low spatial frequency, and the phase analysis device operates to uniquely correlate the phase shift with the physical depth.

[0031] Preferably, the map of the physical depths is either a two-dimensional map or a three-dimensional map.

[0032] The present invention will be fully understood and recognized from the following detailed description in relation to the drawings.

Brief Description of the Drawings

[0033]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Mode for Carrying Out the Invention

[0034] Referring to FIG. 1, FIG. 1 is a simplified diagram of an optical imaging system including an autofocus (automatic focusing, automatic focus adjustment) function, configured and operating in a preferred embodiment of the present invention. Referring to FIG. 2, FIG. 2 is a simplified block diagram of the elements of a system of the type shown in FIG. 1.

[0035] As shown in FIGS. 1 and 2, an optical imaging system 100 is provided that preferably includes an optical imaging head 102 mounted on a chassis 104. The chassis 104 preferably includes a table 106 on which an object 108 to be imaged is placed. For example, for the purpose of inspecting or processing the object 108, the optical imaging system 100 preferably operates to provide an image of the object 108.

[0036] The object 108 is preferably a non-planar object and has physical features (contrivances) of more than one physical depth. Here, as an example, the object 108 is embodied and shown as a PCB including a non-conductive substrate 109 having a metal trace 110 formed on the upper surface. The metal trace 110 may be embedded or may protrude from the surface of the substrate 109. It should be understood that the optical imaging head 102 can be used to acquire images of any suitable target or scene having physical features of a plurality of physical heights or depths, including but not limited to PCBs, wafers / dies, assembled PCBs, flat panel displays, and solar energy wafers.

[0037] In some cases, it may be desirable to generate a focused image of a feature of interest included in the object 108, and this feature of interest has a different physical height or depth relative to other features of the object 108. For example, in the case of the object 108, it may be desirable to generate an in-focus image focused on the metal trace 110 for the purpose of its inspection. A specific feature of the preferred embodiment of this invention is that the optical imaging system 100 provides depth differentiated images and includes a range differentiator 120 that enables autofocus on a feature of interest such as the metal trace 110 even if there is a physical depth difference between the feature of interest and other features such as the substrate 109. Further, this autofocus can be achieved by the range differentiator 120 regardless of the shape of the feature of interest.

[0038] As clearly shown in FIG. 2, the range discrimination device 120 preferably includes an image generator that operates to provide an image of a scene at various physical depths, and here it is embodied as including, for example, an illumination module 122 that illuminates the object 108. The illumination provided by the illumination module 122 is preferably directed at the object 108 through a movable lens unit 124, and this movable lens unit 124 is preferably mounted on a translation stage 126 controlled by a controller 128. The light emitted from the object 108 preferably travels through the movable lens unit 124 towards a camera sensor 130, and this camera sensor 130 is preferably coupled to a processor 132.

[0039] The range discrimination device 120 preferably operates in two modes. In the first operating mode of the range discrimination device 120, preferably, the object 108 is imaged by the camera sensor 130 under illumination conditions where the feature of interest can be clearly distinguished (discriminated) from other features of the object 108 having a physical depth different from that of the feature of interest. This imaging is preferably performed following an initial coarse focusing of the camera sensor 130 on the object 108, so that the image obtained thereby is well focused enough for subsequent processing.

[0040] Illumination that can clearly distinguish the feature of interest from other features of the object 108 having a physical depth different from that of the feature of interest is called feature specific illumination and can be provided by a feature specific illuminator 140 included in the illumination module 122. Here, by way of example only, the feature specific illuminator 140 is shown as being embodied as a UV light source and preferably provides illumination of a very short wavelength having a wavelength of about 420 nm or less.

[0041] Under the UV illumination provided by the feature identification illuminator 140, the non-conductive substrate 109 fluoresces, while the metal trace 110 does not. FIG. 3(A) shows an exemplary image of the substrate 109 and the metal trace 110 thereon under UV feature identification illumination conditions. As shown in FIG. 3(A), the non-conductive substrate 109 appears bright due to its fluorescence, while the metal trace 110 appears dark. Thus, in the image of FIG. 3(A), the non-conductive substrate 109 can be clearly distinguished from the metal trace 110. Further, as a result of the fluorescence of the substrate 109, additional features of the object 108 that may be under the surface of the substrate 109 are masked and thus do not appear in the image of FIG. 3(A), thereby simplifying subsequent image processing.

[0042] Following the generation of the initial feature identification image shown in FIG. 3(A), a preferably tagged or segmented image is generated, which is based on the initial feature identification image. FIG. 3(B) shows an example of a segmented image based on the feature identification image of FIG. 3(A). In the segmented image of FIG. 3(B), the pixels corresponding to the dark metal trace 110 are marked in gray and identified as corresponding to regions of interest, and the pixels corresponding to the bright substrate area 109 are marked in white and identified as corresponding to non-regions of interest. The non-regions of interest should be ignored in subsequent image processing steps. The pixels in the area 112 corresponding to the unclear identification area are marked in black and identified as corresponding to regions of questionable interest, and this area should also be ignored in subsequent image processing steps. Preferably, a predetermined threshold is applied to the level of brightness of the pixels in FIG. 3(A), and the dark pixels corresponding to the metal trace 110 and the bright pixels corresponding to the background substrate 109 are distinguished.

[0043] Accordingly, the segment image of FIG. 3(B) effectively forms a depth differentiated mask image, and the portion of the feature identification image of FIG. 3(A) below a given depth, for example the portion including the substrate 109, is distinguished from the portion of the feature identification image of FIG. 3(A) above the given depth, for example the portion including the metal trace 110. It is understood that the distinction between the portions of the feature identification image of FIG. 3(A) at different physical depths is based on the difference in their optical properties, more specifically the fluorescence difference under UV irradiation between them, and is independent of or unrelated to the physical shape of the features.

[0044] The generation of the segment mask image of FIG. 3(B), although merely an example, can be automatically executed by the computing function included in the system 100 embodied as the processor 132, and the processor 132 is included in the computer 114. That is, the processor 132 preferably operates as a depth differentiator that distinguishes the portions of the initial feature identification image at depths below a predetermined threshold, such as the image of FIG. 3(A), regardless of the shape of the portions, and provides a depth differentiated image such as the depth differentiated image of FIG. 3(B).

[0045] Furthermore, it is further understood that the feature identification UV illuminator 140 combined with the sensor 130 and the processor 132 constitutes a particularly preferred embodiment of an image generator that provides an image of the object 108 including the substrate 109 and the metal trace 110. Note that the image generation function of the range differentiator 120 is not limited to the specific camera and illumination components described in this document. Rather, it should be understood that any suitable components having the function of generating an image of a scene at various physical depths that can distinguish features having various physical depths based on their optical properties regardless of their shape may be provided.

[0046] The computer 144 may include a user interface that enables a user to identify (discriminate) features of interest in a feature identification image such as the metal trace 110 in FIG. 3(A). The features of interest are features that can be identified (discriminated) by the user and are preferably machine-identifiable features. It should be understood that the presence of machine-identifiable features in the feature identification image can be detected by the computer 144 based on its appearance, regardless of the shape features. Therefore, it should be understood that the computer 144 can operate as a target identifier that enables the user to identify machine-identifiable features and preferably also as a feature detector that automatically identifies machine-identifiable features.

[0047] In the second operation mode of the range discrimination device 120, following the generation of a segment image as shown in FIG. 3(B), preferably, the object 108 is imaged by the camera sensor 130 under illumination conditions that are most suitable for enhancing the imaged texture of the feature of interest, which is here embodied as the metal trace 110. Such illumination is called feature focusing illumination and is preferably provided by the feature focusing illuminator 150 included in the illumination module 122. Here, by way of example only, the feature focusing illuminator 150 is shown embodied as a bright field illuminator.

[0048] Although the feature identification illuminator 140 and the feature focus illuminator 150 are shown here as being embodied as two separate illuminators included in the illumination module 122, alternatively, the feature identification illuminator 140 and the feature focus illuminator 150 can be provided by at least partially common illumination elements having at least partially overlapping functions that provide both feature identification illumination and feature focus illumination, as illustrated below with reference to FIG. 6.

[0049] When imaging the object 108 under the illumination provided by the feature focus illuminator 150, the vertical position of the lens 124 with respect to the object 108 is preferably incrementally shifted, and the focal height of the lens 124 with respect to the object 108 is adjusted accordingly. The adjustment of the lens 124 can be controlled by the controller 128, which is preferably operable to move the stage 126, and thereby the lens 124, incrementally with respect to the object 108. In addition or alternatively, the focal height of the lens 124 with respect to the object 108 may be adjusted by adjusting the height of the table 106 and / or the overall adjustment of the optical head 102.

[0050] Preferably, an image of the object 108 is acquired by the sensor 130 for each position of the lens 124. That is, preferably, a series of images are generated in a range of focal heights of the lens 124 above the object 108. An image focus analyzer, preferably embodied as the processor 132, preferably operates to perform image focus analysis on the series of images, provides a focus score based on portions of each image having a depth equal to or greater than a predetermined depth, and confirms a focal distance based on the focus score. That is, it should be understood that the processor 132 further preferably operates as a focus distance ascertainer that confirms the focal distance based on a depth discrimination image such as the image of FIG. 3(B).

[0051] The focus score is preferably calculated for each image acquired under lighting conditions provided by the feature focus illuminator 150, and the focus score is based only on pixels identified (specified) as corresponding to the region of interest in a segment depth discrimination image such as the image of FIG. 3(B). In the case of the metal trace 110 on the substrate 109, as an example, each pixel identified as corresponding to a region of interest such as the metal trace 110 in a depth discrimination image such as the image of FIG. 3(B) is assigned a focus measure based on the local texture. This focus measure can be, for example, the gradient magnitude at the pixel neighborhood, or any other focus measure known in the art.

[0052] In a depth discrimination image such as the image of FIG. 3(B), pixels identified as corresponding to non - interested regions such as the substrate 109 are preferably assigned a focus scale of zero. The overall focus score of each image obtained under the illumination conditions provided by the feature focus illuminator 150 is preferably given by the sum of the focus scales of all the individual pixels in the image corresponding to the region of interest such as the metal trace 110. Since the focus scale of pixels corresponding to non - interested regions such as the substrate 109 is set to zero, the pixels corresponding to non - interested regions do not contribute to the overall focus score of the above - mentioned image and are effectively ignored in focus score calculation.

[0053] In the above - described embodiments, it should be understood that the focus score of each image is preferably based only on the portion of the image at a depth equal to or greater than a predetermined depth, here the depth corresponding to the depth of the metal trace 110, and the portion of the image at a depth less than the predetermined depth, here the portion of the image corresponding to the substrate 109, is not considered. Instead, for example, if the feature of interest is embedded in the substrate, the above - mentioned focus score may be calculated based only on the portion of the depth discrimination image at a depth less than the predetermined depth.

[0054] As shown in FIG. 4, the focus score obtained for each image can be plotted as a function of the height of the lens 124. The lens position at which the feature of interest is at the optimal focus is identified as the lens position corresponding to the image having the highest focus score. In the case of the data represented in FIG. 4, it can be seen that the highest focus score of 80 corresponds to a focus height of approximately 6487 μm. FIG. 3(C) shows a representative image having the highest focus score that is most in focus on the metal trace 110. As can be understood from considering the focus image of FIG. 3(C), the texture of the metal trace 110 is highly visible, while the substrate 109 appears smooth, because the image of FIG. 3(C) is obtained at a focus height that is optimally focused on the metal trace 110 without considering the substrate 109 at a different physical height.

[0055] It should be understood that the optimal focus height corresponding to the focus height of the image with the highest focus score is preferably found with a higher accuracy than the height step between consecutive images. This can be achieved by any method suitable for finding the maximum value of a function, such as fitting the data in the region close to the maximum value to a parabola, which is just an example.

[0056] It should be further understood that the feature-specific illumination preferably provided by the feature-specific illuminator 140 is not limited to UV illumination, and can be under any type of illumination where target features of various physical depths show different optical responses and can thus be distinguished in the image. As an example, the UV feature-specific illuminator 140 can be replaced by another illuminator as shown in the embodiments of FIGS. 5 and 6.

[0057] Next, referring to FIGS. 5 and 6, the optical imaging system 500 is provided in a manner generally similar to the optical imaging system 100 in its related aspects, but is different in that the UV feature-specific illuminator 140 of the illuminator 122 of the range discrimination device 120 is replaced by a combined bright and dark field illuminator or a broad angle illuminator 540 as shown in FIG. 6. The feature-specific illuminator 540 can be of the type generally described in Chinese Patent Application No. 201510423283.5 filed on July 17, 2015, or other illuminators known in the art.

[0058] Here, just as an example, the object 108 is shown embodied as a PCB 508 including a laminate region 509, on which a copper trace 510 is formed and protrudes therefrom. For example, in the case of the PCB 508, for the purpose of its inspection, it may be desirable to generate an image focused on the copper trace 510.

[0059] Under bright-field and dark-field combined illumination or wide-angle illumination provided by the feature identification illuminator 540, the laminate region 509 has a much lower reflectivity than the copper trace 510. FIG. 7(A) shows an example image of the laminate region 509 and the copper trace 510 under the feature identification reflection illumination conditions provided by the feature identification illuminator 540. As shown in FIG. 7(A), the laminate region 509 appears dark due to its low reflectivity, while the copper trace 510 appears bright. Thus, in the image of FIG. 7(A), the laminate region 509 can be clearly distinguished from the copper trace 510. Further, due to the opaque appearance of the laminate 509, additional features of the object 508 that may be under the laminate 509 are masked and thus do not appear in the image of FIG. 7(A), thereby simplifying subsequent image processing.

[0060] FIG. 7(B) shows a depth discrimination image or a segment image based on the initial feature identification image of FIG. 7(A). In the segment image of FIG. 7(B), the pixels corresponding to the bright copper trace 510 are marked white and are identified as corresponding to the regions of interest, and the pixels corresponding to the dark laminate region 509 are marked black and are identified as corresponding to the non-regions of interest, and the non-regions of interest are ignored in subsequent image processing steps. Preferably, a predetermined threshold value of the pixel brightness level is given, and the white pixels corresponding to the copper trace 510 and the black pixels corresponding to the laminate 509 are distinguished.

[0061] The segment image of FIG. 7(B) thus effectively forms a depth discrimination image, and the portion of the feature identification image of FIG. 7(A) with a depth less than a given threshold, here the portion constituting the laminate 509 as an example, is distinguished from the portion of the feature identification image of FIG. 7(A) with a depth greater than or equal to the given threshold, here the portion constituting the copper trace 510 as an example. It is understood that the discrimination between portions of the feature identification image of FIG. 7(A) with various physical depths is based on the difference in optical properties between them, more specifically the difference in reflectance between them under bright-field and dark-field combined illumination or wide-angle illumination, and does not depend on the physical shape of the features.

[0062] The generation of the segment mask image of FIG. 7(B), although just an example, can be automatically executed by the computing function included in the system 500 embodied as the processor 132, and the processor 132 can be included in the computer 144. That is, it should be understood that the processor 132 preferably operates as a depth discrimination device within the system 500 to distinguish portions of an initial feature identification image such as the image of FIG. 7(A) with a depth below a predetermined threshold, regardless of the shape of the portions, and provides a depth discrimination image such as the depth discrimination image of FIG. 7(B) based on this.

[0063] The acquisition of a series of images under the illumination conditions provided by the feature focus illumination 150, and subsequently, the automatic selection of the image that is most in focus on the copper trace 510 at the optimal focus distance, based on the comparison of focus scores assigned only to the pixels corresponding to the copper trace 510 identified in a segmented depth discrimination image such as preferably the image of FIG. 7(B), is generally as described above with reference to FIGS. 3(B)-4. As described above with reference to the schematic diagram 4, the processor 132 within the system 500 preferably additionally operates as a focus distance confirmation device to confirm the focus distance based on a depth discrimination image such as the image of FIG. 7(B).

[0064] FIG. 7(C) is an image of the object 508 to which the highest focus score is assigned, which is optimally focused on the metal trace 510. It should be understood that the focus score is preferably calculated based only on a portion of the depth discrimination image such as the image of FIG. 7(B) with a depth above a predetermined depth threshold, which here corresponds to the prominent copper trace 510. Instead of this, for example, if the feature of interest is embedded in the substrate, the focus score may be calculated based only on a portion of the depth discrimination image with a depth below the predetermined depth threshold.

[0065] Automatically focused images generated by the systems of FIGS. 1-2 and FIGS. 5-6, such as the images shown in FIGS. 3(C) and 7(C), correspond to the images obtained at such a focal distance that are optimally focused on a particular feature of interest of the object being imaged, even though the difference in physical height or depth between the particular feature of interest and other features forms part of the object being imaged.

[0066] Note that the system of this invention can alternatively be operated to automatically generate a range image of the object or scene in order to obtain a depth profile of a particular feature of interest of the object or scene being imaged, and the feature of interest preferably has a physical depth or height different from the depth or height of other features forming part of the object or scene being imaged.

[0067] In connection with the generation of a range image of the object 1108, the operation of a system of the type shown in FIGS. 5 and 6 is described below. The object 1108 includes a non-conductive substrate 1109 and has a copper region 1110 formed thereon, and FIGS. 8(A)-8(C) show images of the object 1108. The systems of FIGS. 5 and 6 are preferably operable to automatically generate a range image of the copper region 1110, where the copper region 1110 protrudes or is recessed with respect to the substrate 1109. The range image is useful, for example, for detecting and measuring the presence of the depth (etch depth) of an indent in the copper region 1110. The generation of the range image is described below with reference to the systems of FIGS. 5 and 6, but it should be understood that instead, any of the above-described systems may be configured, with appropriate modifications apparent to those skilled in the art, to provide a range image of an interesting feature.

[0068] In a first mode of operation of the range discrimination device 120 of the system 500, the object 1108 is preferably imaged by the camera sensor 130 under illumination conditions such that the interesting feature can be clearly distinguished from other features of the object 1108 having a physical depth different from the interesting feature. FIG. 8(A) shows an example image of the substrate 1109 and the copper region 1110 on the substrate under feature-specific illumination conditions. As shown in FIG. 8(A), the non-conductive substrate 1109 appears dark due to its low reflectivity, while the copper region 1110 appears bright. Thus, in the image of FIG. 8(A), the non-conductive substrate 1109 is clearly distinguishable from the copper region 1110. Further, due to the opaque appearance of the substrate 1109, additional features of the object 1108 that may be under the substrate 1109 are masked and thus do not appear in the image of FIG. 8(A), thereby simplifying subsequent image processing.

[0069] Following the generation of the initial feature identification image as shown in FIG. 8(A), preferably a depth discrimination image or a segment image is generated, and this segment image is based on the initial feature identification image. FIG. 8(B) shows an example of a segment image based on the feature identification image of FIG. 8(A). In the segment image of FIG. 8(B), the pixels corresponding to the bright copper region 1110 are marked white, and these pixels are identified as corresponding to the region of interest, and the pixels corresponding to the dark substrate region 1109 are marked black, and these pixels are identified as corresponding to the non-region of interest, and the non-region of interest is ignored in subsequent image processing steps. Preferably, a predetermined threshold value of the level of pixel brightness is applied to distinguish the bright pixels corresponding to the copper region 1110 from the dark pixels corresponding to the background substrate 1109.

[0070] It is understood that the segment image of FIG. 8(B) thus effectively forms a depth discrimination image, and the portion of the feature identification image of FIG. 8(A) with a depth less than a given threshold, here the portion constituting the substrate 1109 as an example, is distinguished from the portion of the feature identification image of FIG. 8(A) with a depth greater than or equal to the given threshold, here the portion constituting the copper region 1110 as an example.

[0071] It should be understood that the discrimination between the portions of the feature identification image of FIG. 8(A) with various physical depths is based on the difference in optical properties between them, more specifically the difference in reflectance under appropriate illumination between them, and does not depend on the physical shape of the features.

[0072] The generation of the segment mask image of FIG. 8(B) can be automatically executed by the processor 132 included in the computer 114. Thus, the processor 132 preferably operates as a depth discrimination device that operates to distinguish a portion of an initial feature identification image such as the image of FIG. 8(A) with a depth less than a predetermined threshold, regardless of the shape of the above portion, and provides a depth discrimination image such as the depth discrimination image of FIG. 8(B) based on this.

[0073] It should be further understood that the feature - specific illuminator 540 combined with the sensor 130 and the processor 132 constitutes a preferred embodiment of an image generator that provides an image of an object 1108 including the substrate 1109 and the copper region 1110.

[0074] The computer 144 may include an interface that allows a user to identify an interesting feature in a feature - specific image, such as the copper region 1110 in FIG. 8(A). The interesting feature is a feature that can be identified by the user and is preferably machine - distinguishable. It should be understood that the presence of a machine - distinguishable feature in the above - mentioned feature - specific image can be detected based on its appearance, regardless of the features of its shape. Thus, the computer 144 can operate as a target - specifying device that allows a user to identify machine - distinguishable features, and preferably as a feature detector that automatically identifies machine - distinguishable features.

[0075] In a second mode of operation of the range - discrimination device, following the generation of the segment - depth - discrimination image shown in FIG. 8(B), preferably, the object 1108 is imaged by the camera - sensor 130 under illumination conditions that are most suitable for generating a depth profile of an interesting feature, which is here embodied as the copper region 1110.

[0076] When imaging the object 1108 under the illumination provided by the feature - focus illuminator 150, the vertical position of the lens 124 with respect to the object 1108 is preferably shifted step - by - step, and the focal height of the lens 124 with respect to the object 1108 is adjusted accordingly. The adjustment of the lens 124 can be controlled by the controller 128, and the controller 128 is preferably operable to move the stage 126, and thereby the lens 124, step - by - step with respect to the object 1108. In addition or alternatively, the focal height of the lens 124 with respect to the object 1108 may be adjusted by adjusting the height of the table 106 and / or the overall adjustment of the optical head 102.

[0077] For each position of the lens 124, preferably an image of the object 1108 is acquired by the sensor 130. That is, preferably a series of images are generated in the focal height range of the lens 124 above the object 108. Preferably, an image focus analysis device embodied as the processor 132 operates to perform image focus analysis on the series of images, provides a focus score based on portions of each image, and confirms the focal length based on the focus score. That is, it is understood that the processor 132 operates as a focal length confirmation device to confirm the focal length preferably based on a discrimination image such as the image of FIG. 8(B).

[0078] In the case of the protruding copper region 1110, it should be understood that the focus score can be calculated based only on portions above a predetermined depth threshold in a depth discrimination image such as the image of FIG. 8(B). Alternatively, for example, in the case of the copper region 1110 embedded in the substrate 1109, the focus score may be calculated based only on portions of the depth discrimination image at a depth less than the predetermined depth threshold.

[0079] In this case, preferably, the focus score is calculated for each pixel in each image acquired under the illumination conditions provided by the feature focus illuminator 150, and the focus score is calculated only for pixels identified in a segment depth discrimination image such as the image of FIG. 8(B) corresponding to the region of interest. To generate a range image, it should be understood that the focus score is preferably calculated for each pixel and the optimal focal height corresponding to the maximum measured feature texture in that pixel is confirmed. In contrast to the focus score calculation described above for the system 100, it should be noted that an overall focus score based on the sum of the focus scores of all pixels in the region of interest of each image is preferably not calculated in this embodiment.

[0080] In the case of the copper region 1110 on the substrate 1109, as an example, for each pixel identified in the depth discrimination image such as the image of FIG. 8(B) corresponding to the copper region 1110, a focus score based on an appropriate local texture measure such as the magnitude of the gradient or some other appropriate focus measure known in the art is assigned. For pixels in the depth discrimination image such as the image of FIG. 8(B) identified as non-interested regions corresponding to the substrate 1109 in the illustrated embodiment, a zero focus score is assigned. It should be understood that no focus score is calculated for portions of each image that are below a predetermined brightness threshold that is a portion corresponding to the substrate 1109.

[0081] As shown in FIG. 9, the focus scores obtained for each pixel can be plotted as a function of the focus height of the lens 124. Referring to FIG. 9, the first trace 1202 represents the variation of the focus score with focus height for the case of the pixels corresponding to the first indentation 1204 seen in FIG. 8(A), and it is shown that the highest 100 focus scores correspond to an absolute focus height of about 6486 μm. Further referring to FIG. 9, the second trace 1206 represents the variation of the focus score with focus height for the case of another pixel corresponding to the second indentation 1208. In this example, the second indentation 1208 is not as deep as the first indentation 1204 represented by the first trace 1202. As can be seen by comparing the first and second traces 1202 and 1206, the height that produces the maximum focus score for the second indentation 1208 is shifted with respect to the maximum focus score of the first indentation 1204 due to the depth difference between them.

[0082] Based on the function as shown in FIG. 9, a height image can be created in which a value equal to the height of focus at which each pixel is found to have the highest focus score is assigned to each pixel. FIG. 8(C) shows this height image, where the gray scale corresponds to the pixel height in microns. Referring to FIG. 8(C), the gray pixels in region 1110 represent higher regions, and the white pixels in regions 1204 and 1208 represent lower regions. The black pixels in region 1109 correspond to the pixels for which the focus score was not calculated, because based on the segment depth discrimination image like the image in FIG. 8(B), these pixels were identified as belonging to the non - interested regions.

[0083] It should be understood that by further analyzing the height or range image of FIG. 8(C), the depths of the indentations 1204 and 1208 with respect to most of the copper region 1110 may be found.

[0084] It is understood that the focal metric based on achieving autofocus in the above - described method is applied only to the features of interest and preferably limited within the boundaries of the features of interest. This is in contrast to the conventional autofocus methods where the focal metric is typically derived over the entire field of view of the camera, and thus various features of shape and size as well as depth, as in the present invention, have a large influence.

[0085] Referring now to FIG. 10, FIG. 10 is a schematic diagram of an optical processing system including a depth discrimination function configured and operating according to a further preferred embodiment of this invention.

[0086] Referring to FIG. 10, an optical imaging system 1300 is provided that preferably includes an optical imaging head 1302 mounted on a chassis 1304. Preferably, the chassis 1304 includes a table 1306 on which an object 1308 to be imaged is placed. The optical imaging system 1300 preferably operates to provide a depth profile image of the object 1308, which is used, for example, for the purpose of inspecting or processing the object 1308.

[0087] The object 1308 is preferably a non-planar object and has physical features of a plurality of physical depths. Here, as an example, the object 1308 is shown embodied as a PCB and has a non-conductive substrate 1309 and metal traces 1310 formed on its upper surface. The metal traces 1310 may be embedded in or protrude from the surface of the substrate 1309. It should be understood that, although not limited, the optical imaging head 1302 can be used to acquire images of any suitable target or scene having physical features of a plurality of physical heights or depths, including PCBs, wafers / dies, assembled PCBs, flat panel displays, and solar energy wafers.

[0088] For inspection purposes, it is often desirable to generate a two-dimensional image of the object 1308, whereby the metal traces 1310 and the substrate 1309 are clearly distinguishable based on the difference in their optical properties.

[0089] In some cases, it may also be desirable to generate a three-dimensional depth profile of an interesting feature included in the object 1308, where this interesting feature has a different physical height or depth relative to other features of the object 1308. For example, in the case of the substrate 1309, it may be desirable to generate a depth profile image of the metal traces 1310 for inspection purposes.

[0090] The specific features of the preferred embodiments of this invention are that the optical imaging system 1300 includes a combined 2D spatial and 3D range differentiator 1320 that provides both a spatial segment image and a depth discrimination image of the region of interest, such as the metal trace 1310, regardless of the difference in physical depth between the feature of interest and other features such as the substrate 1309. Particularly preferably, the range differentiator 1320 includes a 3D plenoptic camera 1321 for generating a depth profile image of the feature of interest.

[0091] The range differentiator 1320 preferably includes an image generator that operates to provide images of the scene at various physical depths, and in this example, is embodied to include an illumination module 1322 that illuminates the object 1308. The illumination provided by the illumination module 1322 is preferably directed towards the object 1308 via the lens unit 1324. The light emitted from the object 1308 is preferably directed towards the two-dimensional imaging camera 1330 and the plenoptic camera 1321 via the beam splitter 1332.

[0092] The illumination module 1322 preferably operates in two modes: a 2D mode and a 3D mode. In the operation of the 2D mode, the object 1308 is preferably imaged by the two-dimensional imaging camera 1330 under illumination conditions that clearly distinguish the object 1308 with the feature of interest from other features of the object 1308 having a physical depth different from that of the feature of interest. This illumination is called feature-specific illumination and can be provided, for example, by the bright-field illuminator 1340 and the dark-field illuminator 1342 included in the illumination module 1322. The combination of the bright-field illuminator 1340 of the illumination module 1322 and the dark-field illuminator 1342 of the illumination module 1322 can be considered to constitute the first part of the image generator by delivering combined bright field and dark field illumination modalities.

[0093] Under the combination of bright-field illumination and dark-field illumination provided by the bright-field illuminator 1342 and the dark-field illuminator 1342, the non-conductive substrate 1309 exhibits a lower reflectivity compared to the reflectivity indicated by the metal trace 1310. FIG. 11(A) is an example image of the substrate 1309 and the metal trace 1310 on the substrate under feature-specific dark-field and bright-field illumination conditions. As shown in FIG. 11(A), due to its low reflectivity, the non-conductive substrate 1309 appears dark relative to the metal trace 1310, while the metal trace 1310 appears bright relative to the substrate 1309. Thus, the non-conductive substrate 1309 can be clearly distinguished from the metal trace 1310 in the image of FIG. 11(A). Further, as a result of the opacity of the substrate 1309, additional layers of the PCB 1308 that may be located under the substrate 1309 become unclear and thus do not appear in the image of FIG. 11(A), thereby simplifying subsequent image processing.

[0094] Following the generation of the initial feature identification image shown in FIG. 11(A), preferably a depth discrimination or segment image is generated, and this segment image is based on the initial feature identification image. An example of a segment image based on the feature identification image of FIG. 11(A) is shown in FIG. 11(B). In the segment image of FIG. 11(B), the pixels corresponding to the bright metal trace 1310 are marked white and are distinguished from the pixels corresponding to the dark substrate area 1309 marked black. Preferably, a predetermined threshold of the level of pixel brightness is applied, and the bright pixels corresponding to the metal trace 1310 are distinguished from the dark pixels corresponding to the background substrate 1309.

[0095] Thus, it is understood that the segment image of FIG. 11(B) effectively forms a depth discrimination image, and the portion of the feature identification image of FIG. 11(A) with a depth less than a given threshold, here for example the portion corresponding to the substrate 1309, is distinguished from the portion of the feature identification image of FIG. 11(A) with a depth exceeding the given threshold, here for example the portion corresponding to the metal trace 1310. The discrimination between portions of the feature identification image of FIG. 11(A) at different physical depths is based on the difference in their optical properties, more specifically the difference in reflectivity under dark field and bright field illumination between them, and is understood not to depend on the features of the physical shape.

[0096] The generation of the segment mask image of FIG. 11(B) can be automatically executed by the computing function included in a processor (not shown) forming part of the system 1300. That is, the processor operates as a depth discrimination device that discriminates, regardless of the shape of the above portion, a portion of an initial feature identification image such as the image of FIG. 11(A) obtained under illumination by a first imaging modality, at a depth less than a predetermined threshold, and provides a depth discrimination image such as the depth discrimination image of FIG. 11(B).

[0097] The features of interest may be identifiable (specifiable) by a user in the feature identification images of FIGS. 11(A) and 11(B), and are preferably features that can be machine-identified. It should be understood that the presence of machine-identifiable features in the feature identification images can be detected based on their appearance, regardless of the shape of the features.

[0098] In the operation of the 3D mode of system 1300, following the generation of a segment image as shown in FIG. 11(B), preferably the object 1308 is imaged by the plenoptic camera 1321 under illumination conditions that are most suitable for emphasizing the imaged texture of the feature of interest, which is here embodied as the metal trace 1310. This illumination is called feature focusing illumination and is preferably provided here by the dark field illuminator 1342. The dark field illuminator 1342 can be considered to constitute a second part of the image generator that transmits a dark field illumination modality to the object 1308.

[0099] FIG. 11(C) is an example image showing the appearance of the metal trace 1310 under only dark field illumination, and the enhanced texture (quality) of the metal trace 1310 is visible.

[0100] Here, the dark field illuminator 1342 is described as contributing to both feature identification illumination and feature focusing illumination. However, it should be understood that instead, feature identification illumination and feature focusing illumination can be provided by different illumination elements having non-overlapping functions.

[0101] Furthermore, the image generation function of the range discrimination device 1320 is not limited to the specific camera and illumination components described in this document. Rather, it should be understood that it can be equipped with any suitable components that function to generate images of scenes at various physical depths that can distinguish features at various physical depths based on their optical properties regardless of their shape.

[0102] In one embodiment, preferably, the plenoptic camera 1321 provides a depth profile image of portions identified as suspected defects based on a 2D segment image such as the image of FIG. 11(B). It should be understood that the true nature and importance of suspected defects are often revealed only when identifying their 3D profile, and the nature of certain suspected defects identifiable in a 2D segment image of the type shown in FIG. 11(B) can be well confirmed by the depth profile image. Efficient 2D segmentation typically requires suppressing the texture of the metal traces in addition to generating a luminance difference between the substrate 1309 and the metal traces 1310. This is achieved by an appropriate combination and careful balance of both bright field and dark field illumination. In contrast, 3D profiling by the plenoptic camera 1321 strongly depends on the surface texture, for example when deriving stereo parallax between adjacent micro-images. By using only dark field illumination, the contrast of the surface textures of both the metal traces 1310 and the substrate 1309 is maximized, resulting in an accurate depth rendering by the plenoptic camera 1321.

[0103] FIG. 11(D) is an example image showing the depth profile of the metal trace 1310 obtained by the plenoptic camera 1321 under the dark field illumination provided by the dark field illuminator 1342. The field of view from which the depth profile of FIG. 11(D) is obtained is larger than that of the initial and segment images of FIGS. 11(A), 11(B) and 11(C), but alternatively, the depth profile of the metal trace 1310 may be limited to a smaller portion of the metal trace 1310 such as the region of the suspected defect, whereby it should be understood that the nature of the defect is confirmed and the defect is classified. In this case, the processor can operate as a focus distance confirmation device that confirms the focus distance at each point for depth profiling of the region where suspected defects exist based on a depth discrimination image such as the image of FIG. 11(B).

[0104] In another preferred operating mode of the 2D space and 3D depth discrimination device 1320, the plenoptic camera 1321 can be used to automatically focus the 2D camera 1330 before acquiring a 2D image.

[0105] In this autofocus mode, preferably, first the object to be inspected 1308 is brought to the coarse focus of the plenoptic camera 1321 under characteristic focus illumination conditions such as dark field illumination conditions preferably provided by the dark field illuminator 1342. This preliminary coarse focus may be based on system optimization and engineering parameters and may include pre-calibration of the system 1300 known to those skilled in the art. FIG. 12(A) shows an example of a coarse focus image of the substrate 1410 acquired by the plenoptic camera 1321. In the illustrated embodiment, the substrate 1410 is a silicon wafer and includes an abrupt height step 1420 with laser inscribed pits 1430 thereon. FIG. 12(B) shows the corresponding out-of-focus 2D image received by the 2D camera 1330.

[0106] The coarse focus image acquired by the plenoptic camera 1321 is then processed by the computing functions included in the processor of the system 1300 to derive the depth profile of the instant field of view of the substrate 141. FIG. 12(C) shows an example of a depth discrimination profile image based on the coarse focus image of FIG. 12(A). In contrast to what is illustrated in FIGS. 11(A) - 11(D), in this operating mode of the range discrimination device 1320, it should be understood that the bright field illumination modality provided by the bright field illuminator 1340 preferably constitutes the first imaging illumination modality and that a depth discriminable image is preferably acquired under this illumination.

[0107] Based on the depth profile image of FIG. 12(C), the 2D camera 1330 can select the characteristic depth at which it should be optimally focused. As an example, in the case of the substrate 1410, the optimal focus depth of the 2D camera 1330 can be set to the depth corresponding to the height of the upper side 1440 of the step in the silicon wafer in the image of FIG. 12(C). As will be understood by those skilled in the art, the depth of focus of the plenoptic camera 1321 can typically straddle the depth of field of the 2D camera 1330 and be in the range of 2 to 4 times larger. As a result, the accuracy of the depth profile analysis based on the plenoptic image of FIG. 12(A) is at least as good as the accuracy achieved based on the depth of focus on the lens 1324.

[0108] The 2D camera 1330 is then automatically focused on the upper side 1440 of the silicon step at the optimal focus depth specified based on the depth profile image of FIG. 12(C), and accordingly, a focused 2D image of the substrate 1410 focused under the characteristic identification bright-field illumination conditions is acquired. Note that in this example, the focus-specific illumination is the same as the characteristic identification illumination. This is a result of the optical reflection characteristics of both the silicon wafer and the laser-formed pits on its surface. FIG. 12(D) is an example of an automatically focused 2D image acquired under the characteristic identification bright-field illumination conditions.

[0109] For example, as previously described with reference to FIGS. 11(C) and 11(D), in order to better classify the nature of the suspected defects present in the 2D autofocus image, after the automatic focus 2D imaging, additional 3D plenoptic imaging of the object 1308 may be performed as needed.

[0110] Referring now to FIG. 13, FIG. 13 schematically shows an optical processing system including an autofocus function configured and operating according to a further preferred embodiment of the present invention, and FIGS. 14(A)-14(C) are simplified examples of images generated by a system of the type shown in FIG. 13.

[0111] As shown in FIG. 13, an optical imaging system 1500 is provided that includes a projector module 1502 that operates to project a pattern onto an object 1508. The imaging system 1500 further preferably includes a camera sensor module 1510 that operates to acquire an image of the object 1508 when the pattern is projected onto the object by the projector module 1502. Preferably, the projector module 1502 and the camera module 1510 are angled (tilted) with respect to a longitudinal axis 1512 defined with respect to the object 1508. The projector module 1502 in combination with the camera module 1510 can be considered to form an image generator that operates to generate an image of the object 1508.

[0112] The object 1508 is preferably a non-planar object and includes physical features of two or more physical depths and includes, but is not limited to, PCBs, wafer dies, assembled PCBs, flat panel displays, and solar energy wafers. Alternatively, the object 1508 may be embodied as any object or scene that includes features within a certain physical depth range.

[0113] In some cases, it may be desirable to generate a focused image of an interesting feature that is at a different physical height or depth relative to other features of the object 1508 included in the object 1508. This can be automatically achieved in the system 1500 by projecting a regularly repeating pattern, such as a sinusoidal or binary moire fringe pattern, onto the surface of the object 1508 and analyzing the phase shift of the projected fringes, as detailed below.

[0114] The operation of the system 1500 can be best understood by referring to the images generated thereby, examples of which are shown in FIGS. 14(A) - 14(C).

[0115] Referring to FIG. 14(A), an image of a fringe pattern 1600 that is preferably projected onto the surface of an object 1508 by a projector module 1502 is shown. As shown in FIG. 14(A), the fringe pattern 1600 undergoes a variable phase shift according to the surface topology of the features on the object 1508, upon which the fringe pattern falls. The computing functionality included in a processor 1516 that forms part of the system 1500 preferably operates to calculate the phase shift of the fringe pattern 1600 in real time and to derive at least the height of the physical features onto which the fringe pattern is projected. The processor 1516 operates as a depth discrimination device for distinguishing portions of the image acquired by the camera module 1510 at various physical heights, regardless of its shape. The fringe phase shift analysis executed by the computing functionality included in the processor 1516 includes, for example, a windowed Fourier transform. In addition to this, the processor 1516 may control the generation of the fringe pattern projected by the projector module 1502.

[0116] The height of the physical features is preferably calculated relative to the height of a reference target incorporated into the system 1500. The height of the reference target can be calibrated against an additional imaging function (not shown) of the system 1500 that is maintained in focus with respect to the object 1508, or can be calibrated against the camera sensor 1510.

[0117] Figures 14(B) and 14(C) respectively show a two-dimensional height map and a three-dimensional height map of the object 1508 based on the projected fringe map of Figure 14(A). As shown in Figures 14(B) and 14(C), the phase shift of the projected fringe pattern can be used as a basis for segmenting the object 1508 according to the relative height of the physical features that cause the corresponding shift in the phase pattern. That is, a given feature height can be selected for optimal focusing based thereon, while features with heights other than the selected height are effectively ignored in subsequent image focusing. Thus, it should be understood that the height maps of Figures 14(B) and 14(C) constitute a segmented or depth-discriminated image, based on which the feature depth optimally focused thereon can be confirmed. Therefore, based only on height selection, regardless of the shape of the feature, the autofocus of the camera 1510 is performed for features at a given height level. The optimal focus distance can be confirmed by the processor 1516 based on the depth-discriminated images of Figures 14(B) and 14(C).

[0118] It should be understood that preferably, the optimal spatial frequency of the fringe pattern projected by the projector module 1502 is set by considering and balancing several opposing requirements. Preferably, the spatial frequency of the fringe pattern is selected to be low enough to enable good contrast projection and its imaging. More preferably, the spatial frequency of the fringe pattern is selected to be high enough to enable sufficiently high-resolution height discrimination. Further, the fringe interval in the fringe pattern is preferably selected to be large enough to encompass the entire expected depth of the object 1508 without phase ambiguity. Preferably, the fringe pattern has a sufficiently low spatial frequency such that its phase shift can be uniquely correlated with the physical depth that causes such a shift without phase ambiguity.

[0119] Preferably, at least a balance of these various factors is achieved, and an optimal spatial frequency of the moiré pattern for a specific imaging application is derived.

[0120] System 1500 is particularly suitable for use in a closed-loop tracking autofocus mode, in which object 1508 is preferably scanned continuously. In the continuous scanning mode, projector module 1502 is preferably strobe-controlled and preferably operates in a pulse mode in synchronization with the operation of camera module 1510. Alternatively, projector module 1502 may be operated continuously, preferably in cooperation with global shutter camera module 1510.

[0121] In the use of system 1500 for continuous closed-loop autofocus operation, preferably, various operating parameters of system 1500 are optimized. The temporal rate at which the height of object 1508 is sampled by the projection of moiré pattern 1600 and the subsequent analysis of its phase shift are preferably selected to be high enough to be suitable for the scanning speed of object 1508 and the rate of variation of its height. The operating frame rate of camera module 1510 is preferably set according to the height sampling rate.

[0122] In addition, the elapsed time between the acquisition of the moiré image by camera module 1510 and the acquisition of the analysis height map (this time delay is sometimes referred to as system latency) is preferably optimized. The system latency mainly depends on the computing power of the system controller of system 1500. Preferably, the system latency is made sufficiently short to avoid excessive delay leading to focusing errors in the imaging function during the operation of the autofocus function after the moiré image is acquired.

[0123] In certain embodiments of the invention, the pixel resolution of camera module 1510 can be set to optimize the performance of system 1500. The fewer the imaging pixels of camera 1510, the higher the camera frame rate and the shorter the processing time. In addition or alternatively, rather than calculating the phase shift across the entire image acquired by camera module 1510, the phase shift may be calculated only within a sparsely selected region within the image frame output by camera module 1510, thereby accelerating the processing time. The number, dimensions, aspect ratio, and spacing of the regions in which the phase shift is calculated are selected by considering the physical or other characteristics of object 1508.

[0124] Those skilled in the art will understand that the invention is not limited to what is particularly claimed below. Rather, the scope of the invention includes combinations and sub - combinations of the features described above, and modifications and variations that can be recalled by those skilled in the art upon reading the above description with reference to the drawings and that are not found in the prior art.

Claims

1. 1. A range discrimination device useful for autofocus, comprising: An image generator that provides images of a scene at various physical depths, a depth discriminator for discriminating between portions of said image that are below a predetermined depth threshold, regardless of the shape of said portions, to provide a depth discriminated image; and A focal length confirmation device is provided for confirming a focal length based on the depth-discriminated image, the image generator comprising a feature specific illuminator for illuminating the scene during acquisition of the image; Range discriminator.

2. 2. The range discriminator of claim 1, wherein the depth discriminator operates to distinguish, under illumination by the feature specific illuminator, between portions of the image at depths below the predetermined threshold and portions of the image at depths above the predetermined threshold based on differences in optical properties therebetween.

3. 3. The range discriminator of claim 1 or 2, wherein the feature specific illuminator comprises a UV illumination source, and the depth discriminator is operative to distinguish the portions of the image based on fluorescence differences therebetween.

4. 3. The range discriminator of claim 1, wherein the feature specific illuminator comprises a dark field illumination source and a bright field illumination source, and the depth discriminator operates to distinguish the portions of the image based on reflectance differences therebetween.

5. The focal length confirmation device is the portion of the image at a depth less than the predetermined threshold; and the portion of the image at a depth equal to or greater than the predetermined threshold; 3. The range discriminator of claim 2, operative to ascertain said focal length based on one of:

6. 6. The range discrimination apparatus of claim 1, further comprising an image focus analysis apparatus operative to provide a focus score based on portions of the image that are at a depth greater than or equal to the predetermined threshold, the focus distance verification apparatus operative to verify the focus distance based on the focus score, the image focus analysis apparatus comprising an illuminator for illuminating the scene with lighting that enhances captured texture in the portions of the image that are at a depth greater than or equal to the predetermined threshold.

7. 7. The range discriminator of claim 6, wherein the illuminator comprises a dark field illuminator.

8. 5. The range discriminator of claim 4, wherein the image generator comprises a camera and the depth-discriminated image comprises a two-dimensional image of the scene.

9. 3. The range discriminator of claim 1 or claim 2, wherein the image generator comprises a plenoptic camera and the depth-discriminated image comprises a three-dimensional image of the scene.

10. 10. The range discriminator of claim 9, wherein the feature specific illuminator comprises a dark field illuminator.

11. 1. A range discrimination device useful for autofocus, comprising: An image generator that provides images of a scene at various physical depths, a depth discriminator for discriminating between portions of said image that are below a predetermined depth threshold; an image focus analyzer operative to provide a focus score based on portions of the image that are at or above the predetermined threshold depth; and a focal length confirmation device for confirming the focal length based on the focus score; the image generator comprising a feature specific illuminator for illuminating the scene during acquisition of the image; Range discriminator.

12. 12. The range discriminator of claim 11, wherein the depth discriminator comprises a UV illumination source, and wherein the depth discriminator distinguishes the portions of the image based on fluorescence differences therebetween.

13. 12. The range discriminator of claim 11, wherein the feature specific illuminator comprises a combined dark field and bright field illuminator, and the depth discriminator distinguishes portions of the image based on reflectance differences therebetween.

14. 14. A range discriminator as claimed in any one of claims 11 to 13, wherein the image focus analysis means comprises an illuminator for illuminating the scene with lighting which enhances captured texture in the portion of the image at a depth above the predetermined threshold.

15. 15. The range discriminator of claim 14, wherein the illuminator comprises a dark field illuminator.

16. 16. The range discriminator of claim 14 or claim 15, wherein the illuminator and the feature-specific illuminator share at least one common illumination component.

17. 1. A range discrimination device useful for autofocus, comprising: a target identification device including a user interface that enables a user to identify machine-distinguishable features of an object in an image; a feature detector operative to identify an occurrence of at least one of said machine-discernible features in said image, regardless of the shape of said feature; and A range discriminator including a focal length confirmation device that confirms a focal length for said machine-discernible feature.

18. 20. The range discriminator of claim 17, further comprising a feature specific illuminator for illuminating said object during acquisition of said image.

19. 20. The range discriminator of claim 18, wherein said feature identification discriminator comprises a UV illumination source, and said feature identification device identifies said machine-discernible features based on their fluorescence.

20. 20. The range discriminator of claim 18, wherein the feature specific illuminator comprises a combined dark field and bright field illuminator, and the feature identification device identifies the machine-discernible features based on their reflectance.

21. 21. Apparatus according to any of claims 18 to 20, wherein the range verification apparatus comprises an illuminator for illuminating the object with lighting which enhances an imaged texture of the features of the object in the image.

22. 22. The range discriminator of claim 21, wherein the illuminator comprises a dark field illuminator.

23. 23. The range discriminator of claim 21 or claim 22, wherein the illuminator and the feature-specific illuminator share at least one common illumination component.

24. 24. A range discriminator according to any of claims 17 to 23, wherein the features of the object include conductive features.

25. 25. The range discriminator of claim 24, wherein the feature of the object comprises an indentation in the conductive feature.

26. 1. A range discrimination device useful for autofocus, comprising: a first image generator having a first imaging modality for providing first images of the scene at different physical depths; a depth discriminator for distinguishing portions of the first image that are below a predetermined threshold depth to provide a depth discriminated image; a focal length confirmation device for confirming a focal length based on the depth-discriminated image; and a second image generator comprising a second imaging modality for providing a second image of the scene automatically focused at the focal distance; The range discriminator, wherein the first imaging modality includes combined bright field and dark field illumination and the second imaging modality includes dark field illumination.

27. 27. The range discriminator of claim 26, wherein the second image generator comprises a plenoptic camera.

28. 27. The range discriminator of claim 26, wherein the first imaging modality includes dark field illumination and the second imaging modality includes combined bright field and dark field illumination.

29. 30. The range discriminator of claim 28, wherein the first image generator comprises a plenoptic camera.

30. 1. A range discrimination device useful for autofocus, comprising: A projector that projects a repeating pattern onto an object with various physical depth features; a sensor for acquiring an image of the object on which the repeating pattern is projected; a phase analyzer for analyzing a phase shift of the repeating pattern and deriving a map of the physical depth of the feature based on the phase shift; and A range discriminator including a focus analyzer for ascertaining a focal distance for at least one of said characteristics.

31. 31. The range discriminator of claim 30, wherein the repeating pattern comprises at least one of a sinusoidal repeating pattern and a binary repeating pattern.

32. 32. A range discriminator according to claim 30 or claim 31, wherein the repeating pattern has a sufficiently low spatial frequency such that the phase analyser is operative to uniquely correlate the phase shift to the physical depth.

33. 33. A range discriminator according to any of claims 30 to 32, wherein the map of the machine-discernible characteristics of the physical depth is one of a two-dimensional map or a three-dimensional map.

Images