Range discrimination device for autofocus in optical imaging systems - Patents.com

Abstract

To provide a system about a depth range differentiator to be used in autofocusing of an optical imaging system.SOLUTION: The present invention is a range differentiator 120 useful for autofocusing, and the range differentiator includes: an image generator that provides images of scenes at various physical depths; a depth differentiator that distinguishes a part of the image at the depth less than a prescribed threshold regardless of a shape of the part, and provides a depth differentiated image; and a focus distance ascertaining device that ascertains a focus distance based on the depth differentiated image. Also, the depth differentiator is configured to work so as to provide a focus score on the basis of the image at the depth equal to or more than the threshold.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS Reference is made herein to U.S. Provisional Patent Application No. 62 / 634,870, filed February 25, 2018, entitled "RANGE DIFFERENTIATORS FOR AUTO-FOCUSING IN OPTICAL IMAGING SYSTEMS," the disclosure of which is incorporated herein by reference, and priority thereto is claimed pursuant to 37 CFR 1.78(a)(4) and (5)(i).

[0002] This invention relates generally to optical imaging systems, and more particularly to systems and methods useful for autofocusing optical imaging systems. [Background technology]

[0003] Various types of autofocus systems for use in optical imaging systems are known in the art. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Special Publication No. 2007-518357 Summary of the Invention [Problem to be solved by the invention]

[0005] The present invention provides a system and method for depth range differentiation for use in autofocusing of optical imaging systems. [Means for solving the problem]

[0006] That is, in a preferred embodiment of the present invention, there is provided a range differentiator useful for autofocus, the range differentiator including an image generator for providing images of a scene at various physical depths, a depth discriminator for distinguishing portions of the images that are below a predetermined threshold depth regardless of the shape of the portions to provide a depth differentiated image, and a focus distance ascertainer for ascertaining a focus distance based on the depth discriminated image.

[0007] In a preferred embodiment of the invention, the image generator includes a feature specific illuminator for illuminating the scene during acquisition of the image, and the depth discrimination device is operative to distinguish, under illumination by the feature specific illuminator, between portions of the image at a depth below the predetermined threshold and portions of the image at a depth equal to or greater than the predetermined threshold based on differences in optical properties therebetween.

[0008] In a preferred embodiment of the invention, the feature specific illuminator includes a UV illumination source and the depth discrimination device is operative to distinguish the portions of the image based on differences in fluorescence therebetween. In another embodiment, the feature specific illuminator comprises a dark field illumination source and a bright field illumination source and the depth discrimination device is operative to distinguish the portions of the image based on differences in reflectance therebetween.

[0009] Preferably, the focus distance confirmation device is operative to confirm the focus distance based on one of said portion of the image having a depth below the predetermined threshold and said portion of the image having a depth equal to or greater than the predetermined threshold.

[0010] In a preferred embodiment of the invention, the range discriminator further comprises an image focus analyzer operative to provide a focus score based on a portion of the image that is at a depth equal to or greater than the predetermined threshold, and the focus distance verification device operative to verify the focus distance based on the focus score. Furthermore, the image focus analyzer includes an illuminator for illuminating the scene with lighting that enhances an imaged texture of the portion of the image that is at a depth equal to or greater than the predetermined threshold. In addition, the illuminator includes a dark field illuminator. Alternatively or additionally, the focus score is assigned without regard to the shape of the portion. In a preferred embodiment of the invention, the focus score is assigned individually to each pixel corresponding to the portion of the image that is at a depth equal to or greater than the predetermined threshold.

[0011] Preferably, said portions of said image at a depth equal to or greater than said threshold are machine identifiable.

[0012] In a preferred embodiment of the invention, the image generator comprises a camera and the depth-discriminated image comprises a two-dimensional image of the scene. Additionally or alternatively, the image generator comprises a plenoptic camera and the depth-discriminated image comprises a three-dimensional image of the scene. In a preferred embodiment of the invention, the feature specific illuminator comprises a dark field illuminator.

[0013] In a preferred embodiment of the invention, the image generator comprises a projector for projecting a repeating pattern onto the scene, the depth discrimination unit comprises a phase analyzer operative to analyze a phase shift of the repeating pattern and derive a map of the physical depths based on the phase shifts, the map forming the depth-discriminated image, and the focus distance identification unit operative to identify the focus distance based on at least one of the physical depths.

[0014] In a preferred embodiment of the invention, the repeating pattern comprises at least one of a sinusoidal repeating pattern and a binary repeating pattern. Furthermore, the repeating pattern has a sufficiently low spatial frequency such that the phase analyzer is operative to uniquely correlate the phase shift to the physical depth. Additionally or alternatively, the map of the physical depth is one of a two-dimensional map and a three-dimensional map.

[0015] Additionally, a range discrimination device useful for autofocus provided by another preferred embodiment of the present invention includes an image generator for providing an image of a scene at various physical depths, a depth discrimination device for distinguishing between portions of the image that have a depth below a predetermined threshold, an image focus analysis device operative to provide a focus score based on portions of the image that have a depth above the predetermined threshold, and a focus distance confirmation device for confirming the focus distance based on the focus score.

[0016] In a preferred embodiment of the invention, the image generator includes a feature specific illuminator that illuminates the scene during acquisition of the image. Additionally, the depth discriminator includes a UV illumination source, and the depth discriminator distinguishes the portions of the image based on differences in fluorescence therebetween. In another aspect, the feature specific illuminator includes a combined dark field and bright field illuminator, and the depth discriminator distinguishes the portions of the image based on reflectance differences therebetween.

[0017] In a preferred embodiment of the invention, the image focus analysis device includes an illuminator for illuminating the scene with illumination that enhances captured texture in the portion of the image at a depth above the predetermined threshold, and further, the illuminator includes a dark field illuminator. Additionally or alternatively, the illuminator and the feature specific illuminator share at least one common illumination component.

[0018] In a preferred embodiment of the invention, the focus score is assigned regardless of the shape of the portion. Additionally or alternatively, the focus score is assigned individually to each pixel corresponding to the portion of the image at a depth equal to or greater than the predetermined threshold.

[0019] Preferably, said portions of said image at a depth above said threshold are machine discernible.

[0020] A range discrimination apparatus useful for autofocus is further provided by yet another preferred embodiment of the present invention, comprising a target identifier including a user interface that enables a user to identify a machine identifiable feature of an object in an image, a feature detector operative to identify at least one occurrence of the machine identifiable feature in an image, regardless of the shape of the feature, and a focus distance confirmation apparatus that confirms a focus distance for the machine identifiable feature.

[0021] Preferably, the range discriminator further comprises a feature specific illuminator for illuminating the object during acquisition of the image.

[0022] In a preferred embodiment of the invention, the feature specific illuminator includes a UV illumination source and the feature identification device identifies the machine-discernible feature based on its fluorescence. In another aspect, the feature specific illuminator includes a combined dark field and bright field illuminator and the feature identification device identifies the machine-discernible feature based on its reflectance.

[0023] In a preferred embodiment of the invention, a range ascertainer includes an illuminator for illuminating the object with illumination that enhances an imaged texture of the features of the object in the image, and further, the illuminator includes a dark field illuminator.

[0024] Preferably, the illuminator and the feature specific illuminator share at least one common illumination component.

[0025] In a preferred embodiment of the invention, the feature of the object includes a conductive feature, and further, the feature of the object includes an indentation in the conductive feature.

[0026] According to yet another preferred embodiment of the present invention there is further provided a range discrimination device useful for autofocus comprising a first imaging modality, a first image generator for providing first images of a scene at various physical depths, a depth discrimination device for distinguishing portions of said first images at a depth below a predetermined threshold to provide depth-discriminated images, a focus distance confirmation device for confirming a focus distance based on said depth-discriminated images, and a second imaging modality, a second image generator for providing a second image of said scene automatically focused at said focus distance.

[0027] In a preferred embodiment of the present invention, the first imaging modality includes combined bright field and dark field illumination, the second imaging modality includes dark field illumination, and the second image generator includes a plenoptic camera.

[0028] In a preferred embodiment of the present invention, the first imaging modality includes dark field illumination, the second imaging modality includes combined bright field and dark field illumination, and the first image generator includes a plenoptic camera.

[0029] Yet another preferred embodiment of the present invention further provides a range discrimination device useful for autofocus, including a projector for projecting a repeating pattern onto an object including features of various physical depths, a sensor for acquiring an image of the object onto which the repeating pattern is projected, a phase analysis device for analyzing a phase shift of the repeating pattern and deriving a map of the physical depths of the features based on the phase shift, and a focus analysis device for ascertaining a focus distance for at least one of the features.

[0030] In a preferred embodiment of the invention, the repeating pattern comprises at least one of a sinusoidal repeating pattern and a binary repeating pattern, and additionally or alternatively, the repeating pattern has a sufficiently low spatial frequency such that the phase analyzer is operative to uniquely correlate the phase shift to the physical depth.

[0031] Preferably, the map of physical depth is one of a two-dimensional map or a three-dimensional map.

[0032] The present invention will be more fully understood and appreciated from the following detailed description taken in conjunction with the drawings, in which: [Brief description of the drawings]

[0033] [Figure 1] 1 is a schematic diagram of an optical imaging system including an autofocus function, constructed and operative in accordance with a preferred embodiment of the present invention; [Diagram 2] FIG. 2 is a simplified block diagram of components of a system of the type shown in FIG. 1. [Diagram 3](A), (B) and (C) are simplified images provided by a system of the type shown in Figures 1 and 2, respectively showing an initial image of an object acquired under feature-specific illumination, a segmented image of the object useful for autofocus based on the initial image, and an autofocused image of the object based on the segmented image. [Figure 4] 4 is a simplified graph showing trends in variables useful for producing an autofocus image of the type shown in FIG. 3(C). [Diagram 5] 1 is a schematic diagram of an optical imaging system including an autofocus function, constructed and operative in accordance with another preferred embodiment of the present invention; [Figure 6] FIG. 6 is a simplified block diagram of components of a system of the type shown in FIG. [Figure 7] (A), (B) and (C) are simplified images generated by a system of the type shown in Figures 5 and 6, respectively showing an initial image of an object acquired under feature-specific lighting conditions, a segmented image of the object useful for autofocus based on the initial image, and an autofocused image of the object based on the segmented image. [Figure 8] (A), (B) and (C) are simplified images generated by either of the systems of Figures 1-2 and 5-6, respectively showing an initial image of an additional feature acquired under feature-specific lighting conditions, a depth-discriminated image useful for autofocus based on the initial image, and an autofocused image based on the depth-discriminated image. [Figure 9] 8B is a simplified graph showing trends in variables useful for producing an autofocus image of the type shown in FIG. 8(C). [Figure 10] FIG. 1 is a schematic diagram of an optical processing system including an autofocus function constructed and operative in accordance with a further preferred embodiment of the present invention; [Figure 11](A), (B), (C) and (D) are simplified images generated by a system of the type shown in FIG. 10, showing, respectively, an initial image of an object acquired under feature-specific lighting conditions, a depth-discriminated image useful for autofocus based on the initial image, a two-dimensional autofocus image based on the depth-discriminated image, and a three-dimensional image. [Figure 12] (A), (B), (C) and (D) are simplified images additionally or alternatively generated by a system of the type shown in FIG. 10, showing, respectively, an initial 3D image of an object acquired under feature-focused lighting conditions, a corresponding 2D image, a depth-discriminated 3D image based on the initial image, and a 2D autofocus image based on the depth-discriminated 3D image. [Figure 13] 1 is a schematic diagram of an optical processing system including an autofocus function, constructed and operative in accordance with a further preferred embodiment of the present invention; [Figure 14] (A), (B) and (C) are simplified images generated by a system of the type shown in FIG. 13, showing, respectively, an initial image of an object acquired under a first lighting condition, a two-dimensional height map image and a three-dimensional height map image useful for autofocus based on the initial image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0034] Reference is now made to FIG. 1, which is a simplified diagram of an optical imaging system including an autofocus function as constructed and operative in accordance with a preferred embodiment of the present invention, and reference is now made to FIG. 2, which is a simplified block diagram of elements of a system of the type shown in FIG.

[0035] 1 and 2, an optical imaging system 100 is provided that preferably includes an optical imaging head 102 mounted on a chassis 104. The chassis 104 preferably includes a table 106 on which an object 108 to be imaged is positioned. The optical imaging system 100 preferably operates to provide an image of the object 108, for example for purposes of inspecting or processing the object 108.

[0036] The object 108 is preferably non-planar and has physical features at more than one physical depth. Here, by way of example, the object 108 is shown embodied as a PCB including a non-conductive substrate 109 having metal traces 110 formed on an upper surface thereof. The metal traces 110 may be embedded or may protrude from the surface of the substrate 109. It should be understood that the optical imaging head 102 may be used to acquire images of any suitable target or scene having physical features at more than one physical height or depth, including, but not limited to, PCBs, wafer dies, assembled PCBs, flat panel displays, and solar energy wafers.

[0037] In some cases, it may be desirable to generate a focused image of a feature of interest contained in the object 108, where the feature of interest has a different physical height or depth relative to other features of the object 108. For example, in the case of the object 108, it may be desirable to generate an image in focus on the metal traces 110 for inspection purposes. A particular feature of a preferred embodiment of the present invention is that the optical imaging system 100 includes a range differentiator 120 that provides depth differentiated images and allows autofocusing on a feature of interest, such as the metal traces 110, even if there is a physical depth difference between the feature of interest and other features, such as the substrate 109. Moreover, this autofocusing can be achieved by the range differentiator 120 regardless of a shape of the feature of interest.

[0038] As clearly shown in Figure 2, the range discriminator 120 preferably includes an image generator operative to provide an image of a scene at various physical depths, and is embodied here as including, for example, an illumination module 122 that illuminates the object 108. The illumination provided by the illumination module 122 is preferably directed towards the object 108 via a moveable lens portion 124, which is preferably mounted on a translation stage 126 controlled by a controller 128. Light emanating from the object 108 is preferably directed via the moveable lens portion 124 to a camera sensor 130, which is preferably coupled to a processor 132.

[0039] Range discriminator 120 preferably operates in two modes. In a first operating mode of range discriminator 120, object 108 is preferably imaged by camera sensor 130 under lighting conditions in which a feature of interest is clearly distinguishable from other features of object 108 that have a different physical depth than the feature of interest. This imaging is preferably performed following an initial coarse focusing of camera sensor 130 on object 108, so that the acquired image is sufficiently well focused for subsequent processing.

[0040] Illumination that allows a feature of interest to be clearly distinguished from other features of the object 108 that are at a different physical depth than the feature of interest is called feature specific illumination, and may be provided by a feature specific illuminator 140 included in the illumination module 122. Here, by way of example, the feature specific illuminator 140 is shown embodied as a UV light source, providing very short wavelength illumination, preferably having a wavelength of about 420 nm or less.

[0041] Under UV illumination provided by the feature specific illuminator 140, the non-conductive substrate 109 fluoresces, whereas the metal traces 110 do not. FIG. 3(A) shows an example image of the substrate 109 and the metal traces 110 thereon under UV feature specific illumination conditions. As shown in FIG. 3(A), the non-conductive substrate 109 appears bright due to its fluorescence, whereas the metal traces 110 appear dark. Thus, in the image of FIG. 3(A), the non-conductive substrate 109 can be clearly distinguished from the metal traces 110. Furthermore, as a result of the fluorescence of the substrate 109, additional features of the object 108 that may be below the surface of the substrate 109 are masked and therefore do not appear in the image of FIG. 3(A), thus simplifying subsequent image processing.

[0042] Following the generation of the initial feature specific image shown in FIG. 3(A), preferably a tagged or segmented image is generated, which segmented image is based on the initial feature specific image. FIG. 3(B) shows an example of a segmented image based on the feature specific image of FIG. 3(A). In the segmented image of FIG. 3(B), pixels corresponding to the dark metal traces 110 are marked in grey to identify these pixels as corresponding to regions of interest, and pixels corresponding to the light substrate regions 109 are marked in white to identify these pixels as corresponding to regions of non-interest, which should be ignored in subsequent image processing steps. Pixels in the regions 112 corresponding to the unclearly identified regions are marked in black to identify these pixels as corresponding to regions of questionable interest, which should also be ignored in subsequent image processing steps. Preferably, a predefined threshold is applied to the brightness levels of the pixels in FIG. 3(A) to distinguish between the dark pixels corresponding to the metal traces 110 and the light pixels corresponding to the background substrate 109.

[0043] It will thus be appreciated that the segmented image of Figure 3(B) effectively forms a depth differentiated mask image, in which portions of the feature specific image of Figure 3(A) that are below a given depth, such as by way of example including substrate 109, are distinguished from portions of the feature specific image of Figure 3(A) that are above the given depth, such as by way of example including metal trace 110. It will be appreciated that the distinction between portions of the feature specific image of Figure 3(A) at different physical depths is based on differences in optical properties between them, more specifically differences in fluorescence under UV illumination between them, and is independent of or unrelated to the physical shape of the features.

[0044] Generation of the segmented mask image of Figure 3(B) may be performed automatically, by way of example only, with computational functionality included in system 100 embodied as processor 132, which is included in computer 114. That is, it will be appreciated that processor 132 preferably operates as a depth discriminator that operates to distinguish portions of the initial feature specific image at depths below a predetermined threshold, such as the image of Figure 3(A), regardless of the shape of the portions, to provide a depth discriminated image, such as the depth discriminated image of Figure 3(B).

[0045] It is further understood that the feature specific UV illuminator 140 in combination with the sensor 130 and the processor 132 constitute a particularly preferred embodiment of an image generator that provides an image of the object 108 including the substrate 109 and the metal traces 110. It should be understood that the image generation functionality of the range discriminator 120 is not limited to the particular camera and lighting components described herein, but rather may comprise any suitable components capable of generating an image of a scene at various physical depths, in which features having various physical depths can be distinguished based on their optical properties regardless of their shape.

[0046] The computer 144 may be provided with a user interface that allows a user to identify a feature of interest in the feature identification image, such as the metal trace 110 in FIG. 3A. It is to be understood that a feature of interest is a user identifiable and preferably machine identifiable feature, the presence of which in said feature identification image can be detected by the computer 144 based on its appearance, regardless of its shape characteristics. It is therefore to be understood that the computer 144 can operate both as a target identifier, allowing a user to identify a machine identifiable feature, and also as a feature detector, preferably to automatically identify machine identifiable features.

[0047] In a second mode of operation of the range discriminator 120, following generation of the segmented images as shown in Fig. 3(B), the object 108 is preferably imaged by the camera sensor 130 under lighting conditions best suited to enhance the imaged texture of the features of interest, here embodied as metal traces 110. Such lighting is called feature focusing illumination and is preferably provided by a feature focusing illuminator 150 included in the illumination module 122. Here, by way of example only, the feature focusing illuminator 150 is shown embodied as a bright field illuminator.

[0048] While the feature specific illuminator 140 and the feature focus illuminator 150 are shown here embodied as two separate illuminators contained within the illumination module 122, it should be understood that the feature specific illuminator 140 and the feature focus illuminator 150 may alternatively be provided by an at least partially common illumination element having at least partially overlapping functionality to provide both feature specific illumination and feature focus illumination, as illustrated below with reference to FIG. 6.

[0049] When imaging the object 108 under the illumination provided by the feature focus illuminator 150, the vertical position of the lens 124 relative to the object 108 is preferably incrementally shifted and the focal height of the lens 124 relative to the object 108 is correspondingly adjusted. The adjustment of the lens 124 may be controlled by a controller 128, which is preferably operable to incrementally move the stage 126, and thereby the lens 124, relative to the object 108. Additionally or alternatively, the focal height of the lens 124 relative to the object 108 may be adjusted by adjusting the height of the table 106 and / or by adjusting the entire optical head 102.

[0050] An image of the object 108 is preferably acquired by the sensor 130 for each position of the lens 124. That is, a series of images is generated, preferably at a range of focal heights of the lens 124 above the object 108. An image focus analyzer, preferably embodied as a processor 132, is preferably operative to perform an image focus analysis on the series of images, provide a focus score based on a portion of each image at or above a predetermined depth, and ascertain a focus distance based on said focus score. That is, it should be appreciated that the processor 132 further preferably operates as a focus distance ascertainer, ascertaining a focus distance based on a depth-discriminated image, such as the image of FIG. 3(B).

[0051] The focus score is calculated for each image, preferably under lighting conditions provided by the feature focus illuminator 150, and the focus score is based only on pixels identified as corresponding to a region of interest in a segmented depth-discriminated image, such as the image in Fig. 3B. For example, in the case of metal traces 110 on substrate 109, each pixel identified as corresponding to a region of interest, such as metal traces 110, in a depth-discriminated image, such as the image in Fig. 3B, is assigned a focus measure based on the local texture. This focus measure can be, by way of example, the gradient magnitude at the pixel neighborhood, or any other focus measure known in the art.

[0052] In a depth-discriminated image, such as the image of Figure 3(B), pixels identified as corresponding to regions of non-interest, such as the substrate 109, are preferably assigned a focus metric of zero. The overall focus score for each image acquired under the lighting conditions provided by the feature-focused illuminator 150 is preferably given by the sum of all the focus metrics of the individual pixels in the image that correspond to regions of interest, such as the metal traces 110. Because the focus metrics of pixels corresponding to regions of non-interest, such as the substrate 109, are set to zero, the pixels corresponding to regions of non-interest do not contribute to the overall focus score for that image and are effectively ignored in the focus score calculation.

[0053] It should be appreciated that in the above described embodiment, the focus score for each image is preferably based only on portions of the image at depths equal to or greater than a predetermined depth, here corresponding to the depth of the metal traces 110, and does not take into account portions of the image below the predetermined depth, here corresponding to the substrate 109. Alternatively, the focus score may be calculated based only on portions of the depth-discriminated image at depths less than the predetermined depth, for example if the feature of interest is embedded within the substrate.

[0054] As shown in FIG. 4, the focus scores obtained for each image can be plotted as a function of the height of the lens 124. The lens position at which the feature of interest is in optimal focus is identified as the lens position corresponding to the image with the highest focus score. For the data represented in FIG. 4, it can be seen that the highest focus score of 80 corresponds to a focus height of about 6487 μm. FIG. 3(C) shows a representative image with the highest focus score, where the metal traces 110 are most in focus. As can be seen from considering the focused image in FIG. 3(C), the texture of the metal traces 110 is highly visible, while the substrate 109 appears smooth, since the image in FIG. 3(C) is acquired at a focus height where the focus is optimally on the metal traces 110, without taking into account the substrate 109 being at a different physical height than the metal traces 110.

[0055] It should be appreciated that the optimal focus height, which corresponds to the focus height of the image with the highest focus score, is preferably found with greater precision than the height step between consecutive images. This can be achieved by any method suitable for finding the maximum of a function, such as, by way of example only, fitting the data in the region close to the maximum to a parabolic function.

[0056] It should be further appreciated that the feature specific illumination preferably provided by feature specific illuminator 140 is not limited to UV illumination, but can be under any type of illumination under which target features at various physical depths exhibit correspondingly different optical responses and thus can be distinguished in the image. As an example, UV feature specific illuminator 140 can be replaced by another illuminator such as that shown in the embodiment of Figures 5 and 6.

[0057] 5 and 6, an optical imaging system 500 is provided which is generally similar in relevant respects to optical imaging system 100, except that the UV feature specific illuminator 140 of the illuminator 122 of the range discriminator 120 is replaced with a combined bright and dark field illuminator or broad angle illuminator 540, as shown in Figure 6. The feature specific illuminator 540 may be of the type generally described in Chinese Patent Application No. 201510423283.5, filed on July 17, 2015, or any other illuminator known in the art.

[0058] Here, by way of example only, object 108 is shown embodied as a PCB 508 including a laminate area 509 having copper traces 510 formed on and protruding therefrom. For example, in the case of PCB 508, it may be desirable to generate an image in which copper traces 510 are in focus for purposes of inspection thereof.

[0059] Under combined bright field and dark field illumination or wide angle illumination provided by feature specific illuminator 540, laminate region 509 is significantly less reflective than copper trace 510. FIG. 7(A) shows example images of laminate region 509 and copper trace 510 under feature specific reflective illumination conditions provided by feature specific illuminator 540. As shown in FIG. 7(A), laminate region 509 appears dark due to its low reflectivity, while copper trace 510 appears bright. Thus, laminate region 509 can be clearly distinguished from copper trace 510 in the image of FIG. 7(A). Furthermore, due to the opaque appearance of laminate 509, additional features of object 508 that may be underneath laminate 509 are masked and therefore do not appear in the image of FIG. 7(A), thus simplifying subsequent image processing.

[0060] Figure 7(B) shows a depth-discriminated or segmented image based on the initial feature identification image of Figure 7(A). In the segmented image of Figure 7(B), pixels corresponding to the bright copper traces 510 are marked in white to identify these pixels as corresponding to regions of interest, and pixels corresponding to the dark laminate regions 509 are marked in black to identify these pixels as corresponding to regions of non-interest, which are ignored in subsequent image processing steps. Preferably, a pre-defined threshold of pixel brightness level is provided to distinguish between the white pixels corresponding to the copper traces 510 and the black pixels corresponding to the laminate 509.

[0061] It will be appreciated that the segmented image of Figure 7(B) thus effectively forms a depth-discriminated image, in which portions of the feature specific image of Figure 7(A) at depths below a given threshold, here by way of example constituting laminate 509, are distinguished from portions of the feature specific image of Figure 7(A) at depths above a given threshold, here by way of example constituting copper trace 510. It will be appreciated that the discrimination between portions of the feature specific image of Figure 7(A) at various physical depths is based on differences in optical properties therebetween, more specifically reflectance differences therebetween under combined bright field and dark field illumination or wide angle illumination, and is independent of the physical shape of the features.

[0062] Generation of the segmented mask image of Figure 7(B) may, by way of example only, be performed automatically by computational functionality included in system 500 embodied as processor 132, which may be included in computer 144. That is, it should be appreciated that processor 132 preferably operates as a depth discriminator within system 500 that operates to distinguish portions of an initial feature-specific image, such as the image of Figure 7(A), that are below a predetermined threshold depth, regardless of the shape of the portions, and provides a depth-discriminated image thereon, such as the depth-discriminated image of Figure 7(B).

[0063] The acquisition of a series of images under lighting conditions provided by feature-focused illumination 150, and subsequent automatic selection of the image in which the copper traces 510 are most in focus at the optimal focus distance, preferably based on a comparison of focus scores assigned only to pixels corresponding to the copper traces 510 identified in a segmented depth-discriminated image, such as the image of Figure 7(B), is generally as described above with reference to Figures 3(B)-4. As generally described above with reference to Figure 4, the processor 132 in system 500 additionally operates as a focus distance verification device that preferably verify the focus distance based on a depth-discriminated image, such as the image of Figure 7(B).

[0064] Figure 7(C) is an image of object 508 with the highest focus score assigned, with the metal traces 510 in optimal focus. It should be appreciated that the focus score is preferably calculated based only on portions of a depth-discriminated image, such as the image of Figure 7(B), that are at a depth equal to or greater than a predefined depth threshold, here corresponding to the protruding copper traces 510. Alternatively, the focus score may be calculated based only on portions of a depth-discriminated image that are at a depth below a predefined depth threshold, for example if the feature of interest is embedded within the substrate.

[0065] It should be understood that automatically focused images produced by the systems of Figures 1-2 and 5-6, such as the images shown in Figures 3(C) and 7(C), correspond to images obtained at such a focus distance that a particular feature of interest of the object being imaged is best focused on, notwithstanding any physical height or depth differences between the particular feature of interest and other features that form part of the object being imaged.

[0066] It should be noted that the system of the invention may alternatively be operated to automatically generate a range image of the object or scene being imaged in order to obtain a depth profile of a particular feature of interest of the object or scene being imaged, the feature of interest preferably having a physical depth or height which differs from the depth or height of other features forming part of the object or scene being imaged.

[0067] The operation of a system of the type shown in Figures 5 and 6 is described below in relation to the generation of a range image of an object 1108. The object 1108 includes a non-conductive substrate 1109 having a copper region 1110 formed thereon, and Figures 8(A)-8(C) show images of the object 1108. The system of Figures 5 and 6 is preferably operable to automatically generate range images of the copper region 1110, where the copper region 1110 protrudes or recesses relative to the substrate 1109. The range images are useful, for example, to detect and measure the presence of an indentation depth in the copper region 1110. The generation of the range images is described below with reference to the system of Figures 5 and 6, although it should be understood that any of the above-described systems may alternatively be configured to provide a range image of a feature of interest, with appropriate modifications apparent to one skilled in the art.

[0068] In a first mode of operation of the range discriminator 120 of the system 500, the object 1108 is preferably imaged by the camera sensor 130 under lighting conditions in which the feature of interest can be clearly distinguished from other features of the object 1108 that have a different physical depth than the feature of interest. FIG. 8(A) shows an example image of the substrate 1109 and the copper regions 1110 thereon under feature-specific lighting conditions. As shown in FIG. 8(A), the non-conductive substrate 1109 appears dark due to its low reflectivity, while the copper regions 1110 appear bright. Thus, in the image of FIG. 8(A), the non-conductive substrate 1109 is clearly distinguishable from the copper regions 1110. Furthermore, due to the opaque appearance of the substrate 1109, additional features of the object 1108 that may be located below the substrate 1109 are masked and therefore do not appear in the image of FIG. 8(A), thus simplifying subsequent image processing.

[0069] Following the generation of the initial feature specific image as shown in Fig. 8(A), a depth-discriminated or segmented image is preferably generated, which is based on the initial feature specific image. Fig. 8(B) shows an example segmented image based on the feature specific image of Fig. 8(A). In the segmented image of Fig. 8(B), pixels corresponding to the bright copper regions 1110 are marked in white, which identifies these pixels as corresponding to regions of interest, and pixels corresponding to the dark substrate regions 1109 are marked in black, which identifies these pixels as corresponding to regions of non-interest, which are ignored in the subsequent image processing steps. Preferably, a predefined threshold of pixel brightness levels is applied to distinguish between bright pixels corresponding to the copper regions 1110 and dark pixels corresponding to the background substrate 1109.

[0070] The segmented image of Figure 8(B) thus effectively forms a depth-discriminated image, where it can be seen that those portions of the feature-specific image of Figure 8(A) that are at a depth below a given threshold, here constituting, by way of example, substrate 1109, are distinguished from those portions of the feature-specific image of Figure 8(A) that are above the given threshold, here constituting, by way of example, copper region 1110.

[0071] It will be appreciated that discrimination between portions of the feature-specific image of Figure 8(A) at various physical depths is based on differences in optical properties between them, more specifically, differences in reflectance between them under appropriate illumination, and is independent of the physical shape of the feature.

[0072] The generation of the segmented mask image of Figure 8(B) may be performed automatically by a processor 132 included in the computer 114. It will thus be appreciated that the processor 132 preferably operates as a depth discriminator operative to distinguish portions of an initial feature-specific image, such as the image of Figure 8(A), that are below a predetermined depth threshold, regardless of the shape of said portions, and provides a depth-discriminated image thereon, such as the depth-discriminated image of Figure 8(B).

[0073] It should be further appreciated that the feature specific illuminator 540 in combination with the sensor 130 and the processor 132 thus constitute a preferred embodiment of an image generator that provides an image of the object 1108 including the substrate 1109 and the copper regions 1110.

[0074] The computer 144 may include an interface that allows a user to identify features of interest in the feature-identified image, such as copper region 1110 in FIG. 8(A). It is to be understood that a feature of interest is a user-identifiable and preferably machine-identifiable feature, the presence of which in said feature-identified image can be detected based on its appearance, regardless of its shape characteristics. It is therefore to be understood that the computer 144 can operate both as a target identifier that allows a user to identify machine-identifiable features, and also as a feature detector that preferably automatically identifies machine-identifiable features.

[0075] In a second mode of operation of the range discrimination device, following generation of the segmented depth discriminated image shown in FIG. 8(B), the object 1108 is preferably imaged by the camera sensor 130 under lighting conditions best suited to generate a depth profile of the feature of interest, here embodied as copper region 1110.

[0076] When imaging the object 1108 under the illumination provided by the feature focus illuminator 150, the vertical position of the lens 124 relative to the object 1108 is shifted, preferably in a stepwise manner, and the focal height of the lens 124 relative to the object 1108 is correspondingly adjusted. The adjustment of the lens 124 may be controlled by a controller 128, which is preferably operable to move the stage 126, and thereby the lens 124, in a stepwise manner relative to the object 1108. Additionally or alternatively, the focal height of the lens 124 relative to the object 1108 may be adjusted by a height adjustment of the table 106 and / or an overall adjustment of the optical head 102.

[0077] For each position of the lens 124, an image of the object 1108 is preferably acquired by the sensor 130. That is, a series of images is generated, preferably at a focal height range of the lens 124 above the object 108. An image focus analyzer, preferably embodied as a processor 132, is operative to perform an image focus analysis on the series of images, provide a focus score based on portions of each image, and confirm a focus distance based on the focus scores. That is, it is understood that the processor 132 preferably operates as a focus distance confirmation device, which confirms a focus distance based on differentiated images, such as the image of FIG. 8(B).

[0078] It should be appreciated that for protruding copper regions 1110, the focus score may be calculated based only on those portions of a depth-discriminated image, such as the image of Figure 8(B), that are at or above a predefined depth threshold. Alternatively, for example, for copper regions 1110 embedded in substrate 1109, the focus score may be calculated based only on those portions of the depth-discriminated image that are at depths below a predefined depth threshold.

[0079] In this case, a focus score is preferably calculated for each pixel in each of the images acquired under the lighting conditions provided by the feature focus illuminator 150, and focus scores are calculated only for pixels identified in the segment depth discriminated image, such as the image of FIG. 8(B), that correspond to the region of interest. It should be appreciated that to generate the range image, a focus score is preferably calculated for each pixel, and an optimal focus height corresponding to the maximum measured feature texture in that pixel is identified. It should be noted that, in contrast to the focus score calculations described above for system 100, an overall focus score based on the sum of the focus scores of all pixels in the region of interest of each image is preferably not calculated in this embodiment.

[0080] For a copper region 1110 on a substrate 1109, by way of example, each pixel identified in a depth-discriminated image, such as the image of FIG. 8(B), as corresponding to the copper region 1110 is assigned a focus score based on a suitable local texture measure, such as gradient magnitude or any other suitable focus measure known in the art. Pixels in a depth-discriminated image, such as the image of FIG. 8(B), that correspond to the substrate 1109 in the illustrated embodiment and are identified as regions of no interest are assigned a focus score of zero. It should be understood that no focus score is calculated for portions of each image that are below a predetermined brightness threshold, which portions correspond to the substrate 1109.

[0081] As shown in FIG. 9, the focus score obtained for each pixel can be plotted as a function of the focal height of the lens 124. With reference to FIG. 9, a first trace 1202 represents the variation of focus score with focal height for a pixel corresponding to the first indent 1204 seen in FIG. 8(A), with the highest focus score of 100 shown to correspond to an absolute focal height of about 6486 μm. With further reference to FIG. 9, a second trace 1206 represents the variation of focus score with focal height for another pixel corresponding to a second indent 1208. In this example, the second indent 1208 is not as deep as the first indent 1204 represented by the first trace 1202. As can be seen by contrasting the first and second traces 1202 and 1206, the height resulting in the maximum focus score for the second indent 1208 is shifted relative to the maximum focus score of the first indent 1204 due to the depth difference between them.

[0082] Based on a function such as that shown in Figure 9, a height image can be created in which for each pixel a value equal to the focus height at which the pixel was found to have the highest focus score is assigned. Figure 8(C) shows this height image, where the grey scale corresponds to the pixel height in microns. With reference to Figure 8(C), the grey pixels in region 1110 represent the higher regions, and the white pixels in regions 1204 and 1208 represent the lower regions. The black pixels in region 1109 correspond to pixels for which no focus score was calculated, because based on a segment depth discrimination image such as the image in Figure 8(B), these pixels were identified as belonging to a region of no interest.

[0083] It should be appreciated that the height or range image of FIG. 8(C) may be further analyzed to find the depth of indentations 1204 and 1208 relative to the bulk of copper region 1110.

[0084] It is understood that the focal metric based on which autofocus is achieved in the above described techniques is applied only to the feature of interest, and preferably confined to the boundaries of the feature of interest. This is in contrast to conventional autofocus methods where the focal metric is typically derived over the entire field of view of the camera and is therefore heavily influenced by a variety of features of shape and size, rather than just depth as in the present invention.

[0085] Reference is now made to FIG. 10, which is a schematic diagram of an optical processing system including depth discrimination functionality, constructed and operative in accordance with a further preferred embodiment of the present invention.

[0086] 10, an optical imaging system 1300 is provided that preferably includes an optical imaging head 1302 mounted on a chassis 1304. The chassis 1304 preferably includes a table 1306 on which an object 1308 to be imaged is positioned. The optical imaging system 1300 is preferably operative to provide a depth profile image of the object 1308, which may be used, for example, for purposes of inspection or processing of the object 1308.

[0087] The object 1308 is preferably non-planar and has physical features at multiple physical depths. Here, by way of example, the object 1308 is shown embodied as a PCB having a non-conductive substrate 1309 and metal traces 1310 formed on an upper surface thereof. The metal traces 1310 may be embedded or protruding from the surface of the substrate 1309. It should be understood that the optical imaging head 1302 may be used to acquire images of any suitable target or scene having physical features at multiple physical heights or depths, including, but not limited to, PCBs, wafer dies, assembled PCBs, flat panel displays, and solar energy wafers.

[0088] For inspection purposes, it is often desirable to generate a two-dimensional image of the object 1308, whereby the metal traces 1310 and the substrate 1309 can be clearly distinguished based on the differences in optical properties between them.

[0089] In some cases, it may be desirable to generate a three-dimensional depth profile of a feature of interest contained in object 1308, where the feature of interest has a different physical height or depth relative to other features of object 1308. For example, in the case of substrate 1309, it may be desirable to generate a depth profile image of metal traces 1310 for inspection purposes thereof.

[0090] A particular feature of a preferred embodiment of the present invention is that the optical imaging system 1300 includes a combined 2D spatial and 3D range differentiator 1320 that provides both spatially segmented and depth-discriminated images of a region of interest, such as a metal trace 1310, regardless of the physical depth difference between the feature of interest and other features, such as the substrate 1309. Particularly preferably, the range discriminator 1320 includes a 3D plenoptic camera 1321 for generating a depth profile image of the feature of interest.

[0091] The range discrimination device 1320 preferably includes an image generator operative to provide images of the scene at various physical depths, and is here embodied, by way of example, to include an illumination module 1322 for illuminating the object 1308. The illumination provided by the illumination module 1322 is preferably directed towards the object 1308 via a lens portion 1324. Light emanating from the object 1308 is preferably directed towards the two-dimensional imaging camera 1330 and the plenoptic camera 1321 via a beam splitter 1332.

[0092] Illumination module 1322 preferably operates in two modes: a 2D mode and a 3D mode. In the 2D mode of operation, object 1308 is imaged by two-dimensional imaging camera 1330, preferably under illumination conditions in which features of interest are clearly distinguishable from other features of object 1308 that have a different physical depth than the features of interest. This illumination is referred to as feature specific illumination, and may be provided, by way of example only, by a bright field illuminator 1340 and a dark field illuminator 1342 included in illumination module 1322. The combination of bright field illuminator 1340 of illumination module 1322 and dark field illuminator 1342 of illumination module 1322 may be considered to constitute a first portion of an image generator, delivering combined bright field and dark field illumination modalities.

[0093] Under the combination of bright field and dark field illumination provided by bright field illuminator 1342 and dark field illuminator 1342, non-conductive substrate 1309 exhibits a low reflectance compared to the reflectance exhibited by metal traces 1310. FIG. 11(A) is an example image of substrate 1309 and metal traces 1310 thereon under feature specific dark field and bright field illumination conditions. As shown in FIG. 11(A), due to its low reflectance, non-conductive substrate 1309 appears dark relative to metal traces 1310, while metal traces 1310 appear bright relative to substrate 1309. Thus, non-conductive substrate 1309 can be clearly distinguished from metal traces 1310 in the image of FIG. 11(A). Furthermore, as a result of the opacity of substrate 1309, additional layers of PCB 1308 that may be located below substrate 1309 are obscured and therefore do not appear in the image of FIG. 11(A), thus simplifying subsequent image processing.

[0094] Following the generation of the initial feature specific image shown in Fig. 11(A), a depth discrimination or segment image is preferably generated, which segment image is based on the initial feature specific image. An example of a segment image based on the feature specific image of Fig. 11(A) is shown in Fig. 11(B). In the segment image of Fig. 11(B), pixels corresponding to the bright metal traces 1310 are marked white to distinguish them from pixels corresponding to the dark substrate areas 1309, which are marked black. Preferably, a predefined threshold of pixel brightness levels is applied to distinguish between bright pixels corresponding to the metal traces 1310 and dark pixels corresponding to the background substrate 1309.

[0095] In this manner, it will be appreciated that the segmented image of Figure 11(B) effectively forms a depth-discriminated image, in which portions of the feature specific image of Figure 11(A) at depths below a given threshold, here by way of example corresponding to substrate 1309, are distinguished from portions of the feature specific image of Figure 11(A) at depths above a given threshold, here by way of example corresponding to metal trace 1310. It will be appreciated that the discrimination between portions of the feature specific image of Figure 11(A) at different physical depths is based on differences in optical properties between them, more specifically reflectance differences under dark-field and bright-field illumination between them, and is independent of physical shape features.

[0096] The generation of the segmented mask image of Figure 11(B) may be performed automatically by computational functionality included in a processor (not shown) forming part of system 1300. That is, it will be appreciated that the processor operates as a depth discriminator operative to distinguish portions of an initial feature-specific image, such as the image of Figure 11(A) acquired under illumination by a first imaging modality, at depths below a predefined threshold, regardless of the shape of said portions, to provide a depth-discriminated image, such as the depth-discriminated image of Figure 11(B).

[0097] It will be appreciated that the features of interest may be identifiable (identifiable) by a user in the feature identification images of Figures 11(A) and 11(B), and are preferably machine-identifiable features, and that the presence of a machine-identifiable feature in the feature identification image can be detected based on its appearance, regardless of the shape of the feature.

[0098] In a 3D mode of operation of the system 1300, following generation of the segmented images as shown in Fig. 11(B), the object 1308 is preferably imaged by a plenoptic camera 1321 under lighting conditions best suited to enhance the imaged texture of the feature of interest, here embodied as a metal trace 1310. This illumination is called feature focusing illumination, and is here preferably provided by a dark field illuminator 1342. The dark field illuminator 1342 can be considered to constitute a second part of the image generator, delivering a dark field illumination modality to the object 1308.

[0099] FIG. 11(C) is an example image showing the appearance of metal traces 1310 under dark field illumination only, where the enhanced texture of metal traces 1310 is visible.

[0100] Although dark field illuminator 1342 is described herein as contributing to both feature-specific and feature-focused illumination, it should be understood that feature-specific and feature-focused illumination may alternatively be provided by different lighting elements that do not have overlapping functions.

[0101] It should further be understood that the image generation functionality of range discriminator 1320 is not limited to the particular camera and lighting components described herein, but rather may comprise any suitable components operable to generate images of scenes at various physical depths, where the features at various physical depths can be distinguished based on their optical properties regardless of their shape.

[0102] In one embodiment, the plenoptic camera 1321 preferably provides a depth profile image of the portion identified as a suspected defect based on a 2D segment image such as the image of FIG. 11(B). It should be appreciated that the true nature and importance of a suspected defect is often only revealed when identifying its 3D profile, and therefore the nature of a particular suspected defect identifiable in a 2D segment image of the type shown in FIG. 11(B) can be well confirmed by the depth profile image. Efficient 2D segmentation typically requires suppressing the texture of the metal traces in addition to creating a luminance difference between the substrate 1309 and the metal traces 1310. This is achieved by a proper combination and careful balance of both bright field and dark field illumination. In contrast, 3D profiling by the plenoptic camera 1321 strongly relies on surface texture, for example when deriving stereo disparity between adjacent micro images. By using only dark field illumination, the contrast of the surface texture of both the metal traces 1310 and the substrate 1309 is maximized, resulting in accurate depth rendering by the plenoptic camera 1321.

[0103] FIG 11(D) is an example image showing a depth profile of the metal trace 1310 acquired by the plenoptic camera 1321 under dark field illumination provided by the dark field illuminator 1342. It will be appreciated that the field of view from which the depth profile of FIG 11(D) is acquired is larger than that of the initial and segmented images of FIGS 11(A), 11(B) and 11(C), but instead the depth profile of the metal trace 1310 may be restricted to a smaller portion of the metal trace 1310, such as a region of a false defect, thereby confirming the nature of the defect and classifying the defect. In this case, the processor may operate as a focus distance checker to check the focus distance at each point for depth profiling of the region where the false defect is present, based on a depth discriminated image such as the image of FIG 11(B).

[0104] In another preferred mode of operation of the 2D spatial and 3D depth range discriminator 1320, the plenoptic camera 1321 can be used to autofocus the 2D camera 1330 prior to acquisition of the 2D image.

[0105] In this autofocus mode, the inspected object 1308 is preferably first brought to a coarse focus of the plenoptic camera 1321 under feature focus illumination conditions, such as darkfield illumination conditions provided by the darkfield illuminator 1342. This preliminary coarse focus may be based on system optimization and engineering parameters and may include pre-calibration of the system 1300 as known to those skilled in the art. FIG. 12(A) shows an example coarse focus image of a substrate 1410 acquired by the plenoptic camera 1321. In the illustrated embodiment, the substrate 1410 is a silicon wafer and includes an abrupt height step 1420 having laser inscribed pits 1430 thereon. FIG. 12(B) shows a corresponding out-of-focus 2D image received by the 2D camera 1330.

[0106] The coarsely focused image acquired by the plenoptic camera 1321 is then processed by a computational function included in the processor of the system 1300 to derive a depth profile of the instant field of view of the substrate 141. Figure 12(C) shows an example of a depth discrimination profile image based on the coarsely focused image of Figure 12(A). It should be understood that in this mode of operation of the range discriminator 1320, in contrast to what is illustrated in Figures 11(A)-11(D), the brightfield illumination modality provided by the brightfield illuminator 1340 preferably constitutes the first imaging illumination modality, and under which illumination the depth-discriminable image is preferably acquired.

[0107] Based on the depth profile image of FIG. 12(C), a feature depth at which the 2D camera 1330 should be optimally focused can be selected. As an example, for substrate 1410, the optimal depth of focus of the 2D camera 1330 can be a depth corresponding to the height of the top side 1440 of the step in the silicon wafer in the image of FIG. 12(C). As will be appreciated by those skilled in the art, the depth of focus of the plenoptic camera 1321 typically straddles the depth of field of the 2D camera 1330 and can range from 2 to 4 times larger, so that the accuracy of the depth profile analysis based on the plenoptic image of FIG. 12(A) is at least as good as the accuracy achieved based on the depth of focus on the lens 1324.

[0108] The 2D camera 1330 is then automatically focused on the top side 1440 of the silicon step at the optimum focus depth determined based on the depth profile image of FIG. 12(C), and an in-focus 2D image of the substrate 1410 focused under feature-specific brightfield illumination conditions is acquired accordingly. Note that in this example the focus-specific illumination is the same as the feature-specific illumination. This is a result of the optical reflectance properties of both the silicon wafer and the laser-formed pits on its surface. FIG. 12(D) is an example of an automatically focused 2D image acquired under feature-specific brightfield illumination conditions.

[0109] For example, as described above with reference to Figures 11(C) and 11(D), following the autofocus 2D imaging, additional 3D plenoptic imaging of object 1308 may be performed, if necessary, to better classify the nature of any spurious defects present in the 2D autofocus image.

[0110] Reference is now made to FIG. 13, which illustrates generally an optical processing system including an autofocus function constructed and operative in accordance with a further preferred embodiment of the present invention, and FIGS. 14(A)-14(C) are simplified examples of images produced by a system of the type illustrated in FIG. 13.

[0111] As shown in Fig. 13, an optical imaging system 1500 is provided that includes a projector module 1502 operative to project a pattern onto an object 1508. The imaging system 1500 preferably further includes a camera sensor module 1510 operative to capture an image of the object 1508 when the pattern is projected by the projector module 1502 onto the object. Preferably, the projector module 1502 and the camera module 1510 are angled (tilted) with respect to a longitudinal axis 1512 defined with respect to the object 1508. The projector module 1502 in combination with the camera module 1510 may be considered as forming an image generator operative to generate an image of the object 1508.

[0112] Object 1508 is preferably a non-planar object that includes physical features at two or more physical depths, including, but not limited to, PCBs, wafer dies, assembled PCBs, flat panel displays, and solar energy wafers, although object 1508 may alternatively be embodied as any object or scene that includes features at a range of physical depths.

[0113] In some cases, it may be desirable to generate a focused image of a feature of interest contained in object 1508 that is at a different physical height or depth relative to other features of object 1508. This can be achieved automatically in system 1500 by projecting a regularly repeating pattern, such as a sinusoidal or binary moire fringe pattern, onto the surface of object 1508 and analyzing the phase shift of the projected fringes, as described in more detail below.

[0114] The operation of system 1500 can best be understood with reference to the images it produces, examples of which are shown in Figures 14(A)-14(C).

[0115] Referring to FIG. 14(A), an image of a fringe pattern 1600 is shown, preferably projected by the projector module 1502 onto the surface of the object 1508. As shown in FIG. 14(A), the fringe pattern 1600 undergoes a variable phase shift depending on the surface topology of the features on the object 1508 upon which the fringe pattern falls. Computational functionality included in the processor 1516 forming part of the system 1500 is operative to calculate, preferably in real time, the phase shift of the fringe pattern 1600 and to derive at least the height of the physical feature onto which the fringe pattern is projected. The processor 1516 operates as a depth discriminator to distinguish image portions acquired by the camera module 1510 at various physical heights, independent of their shape. The fringe phase shift analysis performed by the computational functionality included in the processor 1516 may, for example, include a windowed Fourier transform. In addition, the processor 1516 may control the generation of the stripe pattern projected by the projector module 1502.

[0116] The height of the physical feature is calculated relative to the height of a reference target that is preferably incorporated into the system 1500. The height of the reference target may be calibrated to an additional imaging feature (not shown) of the system 1500 that is kept in focus on the object 1508, or may be calibrated to the camera sensor 1510.

[0117] 14(B) and 14(C) show two-dimensional and three-dimensional height maps, respectively, of the object 1508 based on the projected fringe map of FIG. 14(A). As shown in FIG. 14(B) and 14(C), the phase shift of the projected fringe pattern can be used as a basis for segmenting the object 1508 according to the relative heights of the physical features that cause the corresponding shift in the phase pattern. That is, a given feature height can be selected for optimal focusing thereon, while features at heights other than the selected height are effectively ignored in focusing the image thereafter. It should be appreciated that the height maps of FIG. 14(B) and 14(C) thus constitute a segmented or depth-discriminated image, based on which a feature depth selected for optimal focus thereon can be ascertained. Thus, based solely on the height selection, and independent of the shape of that feature, autofocus of the camera 1510 is performed for features at a given height level. The optimum focus distance can be ascertained by processor 1516 based on the depth-discriminated images of Figures 14(B) and 14(C).

[0118] It should be appreciated that the optimal spatial frequency of the stripe pattern projected by projector module 1502 is preferably set by considering and balancing several competing requirements. Preferably, the spatial frequency of the stripe pattern is selected to be sufficiently low to allow good contrast projection and imaging thereof. More preferably, the spatial frequency of the stripe pattern is selected to be sufficiently high to allow sufficiently high resolution height discrimination. Furthermore, the fringe spacing in the stripe pattern is preferably selected to be sufficiently large to encompass the entire expected depth of object 1508 without phase ambiguity. Preferably, the stripe pattern has a sufficiently low spatial frequency such that its phase shift can be uniquely correlated, without phase ambiguity, with the physical depth causing such shift.

[0119] Preferably at least these various factors are balanced to derive an optimum spatial frequency of the fringe pattern for a particular imaging application.

[0120] System 1500 is particularly suitable for use in a closed-loop tracking autofocus mode, in which object 1508 is preferably scanned continuously. In the continuous scan mode, projector module 1502 is preferably strobed controlled and preferably operates in a pulsed mode in synchronization with the operation of camera module 1510. Alternatively, projector module 1502 may be operated continuously, preferably in coordination with a global shutter camera module 1510.

[0121] In using the system 1500 for continuous closed-loop autofocus operation, various operating parameters of the system 1500 are preferably optimized. The temporal rate at which the height of the object 1508 is sampled by the projection of the fringe pattern 1600 and the subsequent analysis of its phase shift is preferably selected to be sufficiently high to suit the scanning speed of the object 1508 and the rate of its height variation. The operating frame rate of the camera module 1510 is preferably set according to the height sampling rate.

[0122] In addition to this, the time elapsed between the acquisition of the fringe image by the camera module 1510 and the acquisition of the analysis height map (this time delay is sometimes called system latency) is preferably optimized. The system latency mainly depends on the computing power of the system controller of the system 1500. Preferably, said system latency is made short enough to avoid excessive delays in the operation of the autofocus function after the acquisition of the fringe image, which could lead to focusing errors of the imaging function.

[0123] In certain embodiments of the invention, the pixel resolution of the camera module 1510 can be set to optimize the performance of the system 1500. The fewer the imaging pixels of the camera 1510, the higher the camera frame rate and the shorter the processing time. Additionally or alternatively, rather than calculating the phase shift over the entire image captured by the camera module 1510, the phase shift can be calculated only in sparsely selected regions in the image frame output by the camera module 1510, thereby accelerating processing time. The number, dimensions, aspect ratio and spacing of the regions for which the phase shift is calculated are selected by considering the physical or other characteristics of the object 1508.

[0124] Those skilled in the art will appreciate that the invention is not limited to what is specifically claimed below, but rather, the scope of the invention includes combinations and subcombinations of the features described above, as well as modifications and variations that may occur to one skilled in the art upon reading the above description in conjunction with the drawings and that are not in the prior art.

Claims

1. 1. A range discrimination device useful for autofocus, comprising: an image generator having feature specific illuminators with UV wavelengths that provide images of the scene at various physical depths; a depth discrimination device that distinguishes between portions of the image below a given depth and portions of the image above a given depth, regardless of the shape of the portions, by using fluorescence or non-fluorescence under UV wavelength illumination from the feature specific illuminator to provide a depth discriminated image; and a focus distance confirmation device for confirming a focus distance based on the depth-discriminated image, an image focus analyzer operative to provide a focus score based on a portion of the image above the given depth; the focus distance confirmation device is operable to confirm the focus distance based on the focus score; Range discriminator.

2. 2. The range discriminator of claim 1, wherein said focus scores are assigned without regard to the shape of said portion.

3. 2. The range discriminator of claim 1, wherein the focus score is assigned individually to each pixel corresponding to the portion of the image above the given depth.

4. 2. The range discriminator of claim 1, wherein said portion of said image above said given depth is machine distinguishable.

5. 2. The range discrimination apparatus of claim 1, wherein the image generator comprises a projector that projects a repeating pattern onto the scene, and the depth discrimination apparatus comprises a phase analyzer operative to analyze a phase shift of the repeating pattern and derive a map of the physical depth based on the phase shift, the map forming the depth-discriminated image.

6. 6. The range discriminator of claim 5, wherein said focus distance ascertainer is operative to ascertain said focus distance based on at least one of said physical depths.

7. 6. The range discriminator of claim 5, wherein said repeating pattern comprises at least one of a sinusoidal repeating pattern and a binary repeating pattern.

8. 8. The range discriminator of claim 7, wherein said repeating pattern has a sufficiently low spatial frequency such that said phase analyzer is operative to uniquely correlate said phase shift to said physical depth.

9. 8. The range discriminator of claim 7, wherein said map of said physical depth is one of a two-dimensional map and a three-dimensional map.

10. 1. A range discrimination device useful for autofocus, comprising: an image generator having feature specific illuminators with UV wavelengths that provide images of the scene at various physical depths; a depth discrimination device that distinguishes between portions of the image below a given depth and portions of the image above the given depth using fluorescence or non-fluorescence under UV wavelength illumination from the feature specific illuminator; an image focus analyzer operative to provide a focus score based on a portion of the image above the given depth; and a focus distance confirmation device for confirming a focus distance based on the focus score, an image focus analyzer operative to provide a focus score based on a portion of the image above the given depth, the focus distance verifying device operative to verify the focus distance based on the focus score; Range discriminator.

11. 11. The range discriminator of claim 10, wherein the focus score is assigned without regard to the shape of the portion.

12. 11. The range discriminator of claim 10, wherein the focus score is assigned individually to each pixel corresponding to the portion of the image above the given depth.

13. 11. The range discriminator of claim 10, wherein the portion of the image above the given depth is machine distinguishable.

14. 1. A range discrimination device useful for autofocus, comprising: a first image generator having a first imaging modality and having a feature specific illuminator having a UV wavelength for providing first images of the scene at various physical depths; a depth discrimination device that distinguishes between portions of the first image below a given depth and portions of the first image above the given depth using fluorescence or non-fluorescence under UV wavelength illumination from the feature specific illuminator to provide a depth discriminated image; a focus distance confirmation device for confirming a focus distance based on the depth-discriminated image; and an image focus analyzer operative to provide a focus score based on a portion of the first image above the given depth; the focus distance confirmation device is operative to confirm the focus distance based on the focus score; a second image generator including a second imaging modality and configured to provide a second image of the scene automatically focused at the focus distance; Range discriminator.

Images