Three-dimensional sensor having oppositely arranged channels - Patents.com

Abstract

To provide a method of determining dimensional information of a target surface.SOLUTION: A method includes: generating a first point group corresponding to a first plurality of reconstructed surface points of a target surface generated by a first imaging system-illumination source pair of a phase profilometry system; generating a second point group corresponding to a second plurality of reconstructed surface points of the target surface generated by a second imaging system-illumination source pair of the phase profilometry system; generating an initial estimate of the target surface on the basis of the first point group and the second point group; and refining the initial surface estimate using positions of the first point group and the second point group and geometry of the first imaging system-illumination source pair and the second imaging system-illumination source pair to generate a final point group.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] Background technology Obtaining precise dimensional information about surfaces or objects is essential to many industries and processes. For example, in the electronics assembly industry, precise dimensional information about electrical components on a circuit board can be used to determine if the components are properly positioned. In addition, dimensional information is also useful in the inspection of solder paste deposits on circuit boards prior to component assembly to ensure that the proper amount of solder paste is applied to the proper location on the circuit board. In addition, dimensional information is also useful in the inspection of semiconductor wafers and flat panel displays.

[0002] Optical phase profilometry systems have been used to accurately measure and obtain precise dimensional information about surface objects. However, some new electronic assemblies include components with reflective mirror surfaces. Conventional systems are generally configured to measure diffuse non-reflective surfaces, making it difficult to obtain precise dimensional information for such components. In addition, certain technologies are shrinking in size (e.g., circuit boards and components and / or devices on the circuit board), requiring higher magnification and higher resolution optical systems to obtain precise dimensional information. Conventional optical metrology systems experience various measurement errors from various factors as application sizes and surface reflectivities advance and change.

[0003] As the accuracy of such part dimensional information becomes increasingly important to various industries and processes, it becomes increasingly important to accurately measure and obtain such information and to correct for various sources of measurement error in the measured surface features.

[0004] The discussion above is merely provided for general background information and is not intended to be used as an aid in determining the scope of the claimed subject matter. The claimed subject matter is not limited to implementations that solve any or all of the disadvantages discussed in the background.

[0005] The discussion above is merely provided for general background information and is not intended to be used as an aid in determining the scope of the claimed subject matter. Summary of the Invention

[0006] The optical phase profilometry system includes a first moving coaxial camera-projector pair aligned at a first angle relative to the target surface to project a first illumination onto the target surface, and a second moving coaxial camera-projector pair aligned at a second angle relative to the target surface to project a second illumination onto the target surface. The first and second angles are equal and opposite to each other relative to the target surface such that the second moving coaxial camera-projector pair is configured to capture a first reflection from the first illumination and the first moving coaxial camera-projector pair is configured to capture a second reflection from the second illumination. The optical phase profilometry system further includes a controller configured to generate first and second estimates of the target surface based on the captured first and second reflections and combine them to generate a dimensional shape of the target surface.

[0007] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a schematic diagram illustrating an example of a typical optical phase profilometry system. [Figure 2A] FIG. 1 is a schematic diagram illustrating an exemplary optical phase profilometry system. [Figure 2B] FIG. 1 is a schematic diagram illustrating an exemplary optical phase profilometry system. [Figure 2C] FIG. 1 is a schematic diagram illustrating an exemplary optical inspection environment. [Figure 2D] FIG. 1 is a schematic diagram illustrating an exemplary optical inspection environment. [Figure 3A] FIG. 1 is a schematic diagram illustrating an exemplary operating coaxial imaging system-illumination source pair. [Figure 3B] FIG. 1 is a schematic diagram illustrating an exemplary operating coaxial imaging system-illumination source pair. [Figure 4A] FIG. 1 is a schematic diagram illustrating an exemplary optical phase profilometry system. [Figure 4B] FIG. 1 is a schematic diagram illustrating an exemplary optical phase profilometry system. [Figure 5A] FIG. 1 is a schematic diagram illustrating an exemplary optical phase profilometry system. [Figure 5B] FIG. 1 is a schematic diagram illustrating an exemplary optical phase profilometry system. [Figure 6] FIG. 1 is a schematic diagram illustrating an example of an optical phase profile measurement system. [Figure 7] 1 is a flow chart illustrating an exemplary method for iterative connection point cloud refinement. [Figure 8] FIG. 1 is a schematic diagram showing one iteration of iterative attachment point group refinement. [Figure 9] 1 is a flow chart illustrating an example method for merging point clouds from counter-disposed channels. [Figure 10A] FIG. 13 is a schematic diagram illustrating an example of point cloud merging from oppositely disposed channels. [Figure 10B] FIG. 13 is a schematic diagram illustrating an example of point cloud merging from oppositely disposed channels. [Figure 10C] FIG. 13 is a schematic diagram illustrating an example of point cloud merging from oppositely disposed channels. [Figure 11A] FIG. 1 is a schematic diagram illustrating an example of an optical phase profile measurement system. [Figure 11B] 1 is a schematic diagram illustrating an example generated three-dimensional shape of a spherical target. [Figure 11C] 1 is a schematic diagram illustrating an example generated three-dimensional shape of a spherical target. [Figure 12] FIG. 1 is a schematic diagram illustrating an example of an optical phase profile measurement system. [Figure 13A]FIG. 1 is a schematic diagram illustrating an exemplary optical phase profilometry system. [Figure 13B] FIG. 1 is a schematic diagram illustrating an exemplary optical phase profilometry system. [Figure 13C] FIG. 1 is a schematic diagram illustrating an exemplary optical phase profilometry system. [Figure 14] 1 is a flow chart illustrating an example method for generating a three-dimensional shape of a target surface and / or object. [Figure 15A] 1 is a schematic diagram showing an example of a size and shape generation of a target surface and / or object. [Figure 15B] 1 is a schematic diagram showing an example of a size and shape generation of a target surface and / or object. [Figure 16] FIG. 1 is a schematic diagram illustrating an example of an optical phase profile measurement system. [Figure 17] FIG. 1 is a schematic diagram illustrating an example of an optical phase profile measurement system. [Figure 18A] FIG. 1 is a schematic diagram illustrating an example of an optical phase profile measurement system. [Figure 18B] FIG. 1 is a schematic diagram illustrating an example of an optical phase profile measurement system. [Figure 19] 1 is a flow chart illustrating an exemplary method for dynamically compensating for errors in an optical phase profilometry system. [Figure 20] FIG. 2 is a schematic diagram illustrating an example of dynamic compensation. [Figure 21] 1 is a flow chart illustrating an exemplary method of dynamic compensation. [Figure 22A] FIG. 1 is a schematic diagram illustrating an example of a digital light processing projector system. [Figure 22B] FIG. 1 is a block diagram showing an example of an optical low pass filter. [Diagram 23] 1 is a flow chart illustrating an example of projecting light onto a target surface. [Figure 24] FIG. 1 is a schematic diagram illustrating an exemplary optical phase profilometry system. [Diagram 25] FIG. 1 is a schematic diagram illustrating an exemplary optical phase profilometry system. [Figure 26] FIG. 1 is a block diagram illustrating an example of an optical phase profile measurement system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Optical phase profilometry systems are often used to obtain precise dimensional information about surfaces or objects in various industries and processes. For example, these systems can be used to measure the height and position of various components on an object surface. In the electronics industry, for example, many electronic assemblies include various components and / or devices mounted on a circuit board. To ensure accurate dimensions and placement of such components and / or devices, an illumination source (e.g., a projector) projects patterned illumination onto a target surface or object. The patterned illumination is reflected from the target surface or object and captured by an imaging system (e.g., a camera) that views the target surface at a known angle relative to the illumination angle (e.g., triangulation angle). Optical phase profilometry systems calculate the dimensions of the target surface or object by measuring the phase or position of the projected illumination at a specific point (e.g., pixel) of the image captured by the imaging system and the geometry of the sensor.

[0010] In a typical optical phase profilometry system, it is common to have a single projector that illuminates a surface with a structured pattern and multiple oblique cameras observing the surface (i.e., cameras positioned at an oblique angle relative to the projector relative to the surface). Alternatively, an opposite but equivalent structure, a single camera observing a surface illuminated by multiple oblique projectors (i.e., projectors positioned at an oblique angle relative to the camera relative to the surface), is used.

[0011] An estimate (e.g., a point cloud) of the surface of the measurement object is typically generated independently from each camera-projector pair, and then these estimates are averaged together to form an approximate reconstruction of the surface. In general, the estimate may be formed by projecting a sinusoidal interference fringe onto the surface and estimating the phase of the sine wave at each pixel in the image.

[0012] FIG. 1 is a schematic diagram illustrating an example of a typical optical phase profilometry system 10. System 10 includes projector 12, oblique camera 14, oblique camera 16, illumination 18, surface 20, reflection 22, and reflection 24. As discussed above, system 10 has a single projector 12 and multiple oblique cameras 14 and 16. Projector 12 projects illumination 18 onto surface 20 that is reflected as reflections 22 and 24 that are captured by multiple oblique cameras 14 and 16. While FIG. 1 illustrates a single projector / multiple oblique cameras configuration, as discussed above, an equivalent but opposite configuration can be used in a typical phase profilometry system in which a single camera observes a surface onto which multiple oblique projectors project illumination.

[0013] These typical systems have several limitations, especially when observing objects with specular surfaces and / or difficult surface geometries (e.g., "rounded" and / or "curved" reflective surfaces, tilted target surfaces, varying reflectance, etc.).

[0014] One challenge concerns measurement inaccuracies caused by specular reflections. During optical inspection, it is common to encounter situations where there is a glint (i.e., a bright specular reflection) at a location on the target surface from the viewpoint of one imaging system. Glint occurs when the surface normal of a highly reflective surface bisects the angle subtended by the camera-projector pair. Due to the non-zero width of the imaging system's point spread function, which is a practical reality in imaging optics used by optical phase profilometry, phase estimates at neighboring pixels can be distorted by the phase observed at the glint, since the reflection at the glint is much stronger than the reflection at neighboring points on the surface. This can result in inaccurate points being added to the point cloud representation of the test surface in that phase profilometry system.

[0015] FIG. 2A is a schematic diagram illustrating an example of an optical phase profilometry system 50. FIG. 2A shows only one oblique camera viewpoint. The system 50 includes an illumination 52, a target surface 54, a reflection 56, a glint 58, a real surface point 60, a false surface point 62, a projector 64, and a first oblique camera 66. As shown in FIG. 2A, the projector 64 projects illumination 52 onto the target surface 54, which is reflected as a reflection 56 towards and captured by the first oblique camera 66. However, the target surface 54 includes at least some specular reflection that generates a glint 58 when the illumination 52 reaches the target surface 54. Due to the point spread function of the image characteristics of the camera 66 and the intensity of the reflected glint 58, the phase information of the illumination 52 appears in the image of the surface point 60 of the camera 66. The reconstructed point of surface point 60 is inaccurately measured because the information that camera 66 receives from surface point 60 via reflection 56 is a combination of phase information of surface point 60 and glint 58. As shown in FIG. 2A, the measured position surface point of camera 66 is shifted along reflected ray 56 in the direction of glint 58, resulting in, for example, erroneous surface point 62.

[0016] FIG. 2B is a schematic diagram illustrating an example of an optical phase profilometry system 50. FIG. 2B shows the perspective of the other oblique camera of the system 50 (opposite to FIG. 2A). The system 50 includes an illumination 52, a target surface 54, a reflection 57, a reconstructed surface point 63, a projector 64, and a second oblique camera 68. As shown in FIG. 2B, the projector 64 projects illumination 52 onto the target surface 54, which is reflected as a reflection 57 towards the second oblique camera 68 and captured by the second oblique camera. However, unlike FIG. 2A, no glint occurs near the surface point 63 from the perspective of the second oblique camera 68. Thus, the reconstructed surface point 63 of the camera 68 is an accurate representation of the actual surface point.

[0017] However, although the second oblique camera 68 accurately reconstructs the surface points, as described above, there will be errors due to the inaccurate points (62) measured by the first oblique camera 66 because the estimates (e.g., point clouds) produced by both camera 66 and camera 68 are combined during the reconstruction of the final surface estimate of the target surface 54. The final surface estimate will therefore be biased by errors in the point cloud of the first oblique camera 66.

[0018] Effects similar to those described in Figures 2A-B can occur when there are any intensity and / or reflectance gradients (e.g., brightness variations) on the target surface. Due to the point spread function of the imaging optics (e.g., cameras 66 and 68), the phase of darker points is influenced by the phase of nearby brighter points. This can lead to inaccuracies in the final reconstruction (e.g., the final reconstruction will be biased toward the brighter points along each reflected (imaging system) ray).

[0019] 2C is a schematic diagram illustrating an example of an optical inspection environment 70. The environment 70 includes a projection 72, a reflection 74, a target surface 76, an arrow 77, and a measurement surface point 78. The target surface 76 has an intensity and / or reflectance gradient from "high brightness" to "low brightness" from left to right, as indicated by the arrow 77. In general, this means that the target surface 76 is high reflection to low reflection and / or high specular reflection to low specular reflection from left to right. The projection 72 is projected onto the target surface 76 by a projector or other illumination source and reflected as a reflection 74 that is captured by a camera or other imaging system. However, due to the gradient, the increased brightness on the left side of the target 76 biases the measurement surface point 78 along the reflected (imaging system) ray toward the brighter portion of the target 76, so that the camera decodes the height as lower than the actual surface height, as indicated by the measurement surface point 78. Thus, the measurement output of the camera is erroneous. This error tends to occur when the normal (eg, perpendicular) to surface 76 bisects the angle between projection 72 and reflection 74 .

[0020] 2D is a schematic diagram illustrating an example of an optical inspection environment 80. Environment 80 includes projection 82, reflection 84, target surface 86, arrow 87, and measurement surface point 88. Environment 80 is similar to environment 70, except that projection 72 came from the left side (the "high brightness" / high specular side) of target surface 76, whereas projection 84 comes from the right side (the "low brightness" / low specular side) of target surface 86. Target surface 86 has an intensity and / or reflectance gradient from "high brightness" to "low brightness" from left to right, as indicated by arrow 87. In general, this means that target surface 86 is high reflection to low reflection and / or high specular reflection to low specular reflection from left to right. Projection 82 is projected onto target surface 86 by a projector or other illumination source and reflected as reflection 84 that is captured by a camera or other imaging system. However, due to the gradient, the increased brightness on the left side of the target 86 biases the measurement surface point 88 along the reflected (imaging system) ray toward the brighter portion of the target 86, causing the camera to decode the height as higher than the actual surface height, as shown by the measurement surface point 88. Thus, the measurement output of the camera will be erroneous. This error tends to occur when the normal (e.g., perpendicular) to the surface 86 bisects the angle between the projection 82 and the reflection 84.

[0021] To overcome the problems discussed above, a system is used that uses multiple coaxial illumination source / imaging system (eg, projector / camera) pairs with opposing channel configurations.

[0022] For clarity, it should be understood that the term "channel" refers to a particular illumination source-imaging system pair, and the term "oppositely-located channels" refers to a pair of channels that are identical except for the positions of the illumination source and imaging system are swapped. It should also be understood that the term "channel" can include an illumination source from one operational pair (e.g., an operational coaxial imaging system-illumination source pair) and an imaging system from another operational pair. It should also be understood that the term "channel" can include a camera from a first camera-projector pair and a projector from a second camera-projector pair.

[0023] 3A is a schematic diagram illustrating an example of an operational coaxial imaging system-illumination source pair. The system 100 includes an illumination source 102, an imaging system 104, a beam splitter 106, illumination 108, a target surface 110, and a reflection 112. The illumination source 102 (e.g., a projector) projects illumination toward the target surface 110 and hits the beam splitter 106. A portion of the illumination 108 travels toward the target surface 110 and is reflected toward the beam splitter 106 and toward the imaging system 104 as a reflection 112. By utilizing the beam splitter 106, the system 100 is composed of the illumination source 102 and the imaging system 104 sharing a common optical path. That is, the illumination source 102 and the imaging system 104 are approximately coaxial and view the target surface 110 from substantially the same perspective / perspective.

[0024] Beam splitter 106, shown as a plate beam splitter arranged at a 45° angle, consisting of a thin, flat glass plate with a coating (e.g., semi-mirror) on the surface facing illumination source 102. Beam splitter 106 "splits" illumination 108 in half, one portion traveling (e.g., traveling) toward target surface 110 (as shown) and another portion typically reflected (not shown in FIG. 3A for clarity) toward a reference surface (e.g., a mirror) in the field of view of imaging system 104, which reflects the reference beam (back through beam splitter 106) toward imaging system 104 (typically to recombine the split beams before entering imaging system 104). Although also not shown in FIG. 3A for clarity, system 100 may also include a number of lenses (e.g., collimating lenses, objective lenses, compound lens assemblies, telecentric lens assemblies, etc.), apertures, sensors, additional beam splitters, mirrors, and any other suitable components and / or devices. In addition, although a plate beam splitter is shown, other types of beam splitters can also be used, such as, but not limited to, a cube beam splitter.

[0025] Although shown separated from one another for purposes of illustration in Figure 3A, illumination source 102 and imaging system 104 are an operational pair that may be contained within a single housing. By way of example, certain future descriptions will refer to each operational illumination source / imaging system pair as a single assembly, as shown in Figure 3B.

[0026] FIG. 3B is a schematic diagram showing an example of an operational coaxial illumination source / imaging system pair. FIG. 3B is a simplified version of the system 100 shown in FIG. 3A, and is similar to FIG. 3A, so similar elements are numbered the same. The system 101 includes an operational illumination source / imaging system pair 103, which includes an illumination source 102, an imaging system 104, and a beam splitter 106, which are not shown by numbers. The system 100 further includes an illumination 108, a target 110, and a reflection 112. Although the illumination 108 and the reflection 112 share a line in FIG. 3A and FIG. 3B, it should be understood that the illumination 108 and the reflection 112 are equally represented as shown by the arrows. The illumination source 102 includes a projector in one example. The imaging system 104 includes a camera in one example.

[0027] FIG 4A is a schematic diagram illustrating an example of an optical phase profilometry system 200. System 200 includes a first illumination source / imaging system operating pair 202 and a second illumination source / imaging system operating pair 204. Operating pairs 202 and 204 are a coaxial pair as described above with reference to FIGS. 3A-3B. System 200 further includes a target surface 206, illuminations 208 and 214, and reflections 212 and 216. The configuration (e.g., alignment geometry) shown in FIG 4A utilizes two operating pairs 202 and 204, each viewing target surface 206 from a different angle, forming two counter-positioned channels (as described further below).

[0028] FIG. 4B is a schematic diagram illustrating an example of an optical phase profilometry system 200. FIG. 4B is similar to FIG. 4A, and thus like elements are numbered similarly. FIG. 4B illustratively divides system 200 into respective illumination and imaging components to show two oppositely disposed channels formed by the configuration of system 200. On the left side, an illumination source of working pair 204 projects illumination 214 onto target surface 206, which is reflected from the target surface as reflection 216 and captured by the imaging system of working pair 202. This optical path (e.g., channel) (214 and 216) forms a first oppositely disposed channel 218.

[0029] On the right side, an illumination source of operating pair 202 projects illumination 208 onto target surface 206 that is reflected from target surface 206 as reflection 212 and captured by the imaging system of operating pair 204. The optical paths (e.g., channels) (208 and 214) form a second oppositely disposed channel 220.

[0030] The use of counter-located channels is advantageous. There is no difference in optical coverage, since both channels 218 and 220 observe / expose the same field of view and share the same relative angle between illumination and reflection. More importantly, they are more robust in terms of eliminating and / or reducing the measurement errors (FIGS. 2A-2B) previously mentioned with respect to the effect of the point spread function of the imaging system in the presence of glint or intensity gradients. Specifically, when these errors occur in the system 200, corresponding points from the estimates (e.g., point clouds) of the two channels 218 and 220 move in almost equal and opposite directions. By using appropriate algorithms, the estimates of the channels 218 and 220 can be combined such that the errors mostly and / or substantially cancel each other out. In this sense, the counter-located channels 218 and 220 are complementary to each other and self-compensating. This is more fully illustrated in the following figures.

[0031] FIG. 5A is a schematic diagram of an example of an optical phase profilometry system. FIG. 5A is similar to FIG. 4A, and thus similar features are numbered the same. System 200 includes an operational illumination source / imaging system pair 202, an operational illumination source / imaging system pair 204, a surface target 206, an illumination 214, and a reflection 216, which form a first counter-disposed channel 218. System 200 further includes a glint 222, a real surface point 224, and a false surface point 226. Similar to the phenomenon of FIG. 2A, glint 222 generates a false surface point 226 in the imaging system of operational pair 202. The real surface point 224 is moved along the reflected ray 216 in the direction of glint 222.

[0032] FIG. 5B is a schematic diagram of an example of an optical phase profilometry system. FIG. 5B is similar to FIG. 4A, and thus similar features are numbered the same. System 200 includes operational illumination source / imaging system pair 202, operational illumination source / imaging system pair 204, surface target 206, illumination 208, and reflection 212, which form a second counter-disposed channel 220. System 200 further includes glint 222, actual surface point 228, and reconstructed surface point 230. Again, glint 222 causes measurement errors. The imaging system of operational pair 204 generates erroneous reconstructed surface point 230. Actual surface point 228 is pulled along the reflection 212 ray (just like the erroneous reconstructed surface point in FIG. 5A and FIG. 2A). However, because operational pair 202 and operational pair 204 are coaxial and configured to form two counter-disposed channels, the errors are equal and in opposite directions and can therefore be compensated for.

[0033] As described above, each of the counter-located channels 218 and 220 generates an estimate (point cloud) of the target surface 206. These estimates contain errors resulting from the effects of the point spread function of the imaging system in the presence of glints (e.g., 222) or intensity gradients. However, because of the counter-located channels 218 and 220, the errors in the reconstructed surface points 226 and 230 are equal and opposite along the respective reflected rays 216 and 212 and can thus compensate for each other.

[0034] 6 is a schematic diagram illustrating an example of an optical phase profilometry system 250. The system 250 includes a working pair 252 and 254, a target surface 256, illumination 258 and 254, reflection 262 and 266, counter-positioned channels 268 and 270, glint 272, erroneous reconstructed surface points 276 and 280, correct reconstructed surface point 282, and arrows 284 and 286. Errors in the reconstructed surface points 276 and 280 can be corrected to create the reconstructed point 282 by moving the erroneous points 276 and 280 along their respective reflected rays 266 and 262, as indicated by arrows 284 and 286, until they intersect and define a new, more accurate representation of the surface target position. The direction of the reflected rays is derived, in one example, by mapping the pixel positions of the imaging system in the field of view to acceptance angles using imaging system calibration techniques.

[0035] In another embodiment, an algorithm can be used to approximate the intersection of the erroneous reconstruction points of the counter-located channels. Generally speaking, the algorithm iteratively refines each estimate (point cloud) from the counter-located channels, gradually moving each point along its reflected ray (imaging system ray) towards the other point cloud.

[0036] 7 is a flow chart illustrating an exemplary method of iterative joint point cloud refinement. Method 300 begins at block 310, where an optical profilometry system is used to generate reconstructed surface points in a first counter-disposed channel point cloud and a second counter-disposed channel point cloud along a target surface. Surface points are reconstructed (estimated) for both the first counter-disposed channel point cloud and the second counter-disposed channel point cloud. In one example, an optical profilometry system such as system 250 is used.

[0037] The method proceeds to block 320 where, for each reconstructed surface point in the first channel point cloud, reconstructed surface points in the second channel point cloud near the respective reflected (imaging system) rays for each reconstructed surface point in the first channel point cloud are identified. This identification step identifies a set of "candidate" points on the second channel point cloud that are near the reflected (imaging system) rays of a selected first channel reconstructed surface point.

[0038] The method proceeds to block 330, where a projection of each of the near (candidate) points of the second channel's point cloud onto the first channel's reflected (imaging system) ray is calculated. In other words, a distance that each of the near (candidate) points of the second channel would be along the first channel's reflected (imaging system) ray is calculated. Or, in other words, where each of the near (candidate) points should be located along the first channel's reflected (imaging system) ray is calculated.

[0039] The method proceeds to block 340 where an average projection (position) of each of the calculated projections (positions) of the second channel's near (candidate) points on the first channel's reflected (imaging system) ray is calculated. In other words, the average position of the near points on the reflected ray is calculated.

[0040] The method proceeds to block 350 where the reconstructed surface point of the first channel is moved along its reflected (imaging system) ray a portion of the distance to the calculated average position of the near (candidate) point on the reflected (imaging system) ray.

[0041] The method 300 proceeds to block 360 where, for each reconstructed surface point in the second channel point cloud, reconstructed surface points in the first channel point cloud near the respective reflected (imaging system) rays for each reconstructed surface point in the second channel point cloud are identified. This identification step identifies a set of "candidate" points on the first channel point cloud that are near the reflected (imaging system) rays of a selected second channel reconstructed surface point.

[0042] The method 300 proceeds to block 370 where a projection of each of the near (candidate) points of the first channel point cloud onto the reflected (imaging system) ray of the second channel is calculated. In other words, each of the near (candidate) points is the distance that each of the near (candidate) points of the first channel would be along the reflected (imaging system) ray of the second channel.

[0043] The method proceeds to block 380 where an average projection (position) of each of the calculated projections (positions) of the near (candidate) points of the first channel on the reflected (imaging system) ray of the second channel is calculated. In other words, the average position of the near points on the reflected ray is calculated.

[0044] The method proceeds to block 390 where the reconstructed surface point of the second channel is moved along its reflected (imaging system) ray a portion of the distance to the calculated average position of the near (candidate) point on the reflected (imaging system) ray.

[0045] FIG. 8 is a schematic diagram illustrating one iteration of the method 300. In particular, FIG. 8 illustrates blocks 310-350. Iteration 301 includes an operational illumination source / imaging system 252, a reflection 266 (or a reflection / imaging system ray) forming a portion of a first counter-disposed channel 268, a first channel surface point 276, a near (candidate) point 288, an average projection point 290, a relocated (refine) point 292, and an arrow 294. A near (candidate) point 288 that is close to the reflection 266 is identified. An average projection (location) point of the near point 288 along the reflection 266 is calculated as represented by 290. Once the average projection point 290 is calculated, a relocated (refine) reconstruction point 292 is identified by moving the recalculated surface point 276 along the reflection 266 part way along the way to the average projection point 290 as indicated by arrow 294.

[0046] 9 is a flow chart illustrating an exemplary method for merging point clouds from opposing channels. The method 400 begins at block 410 with generating a first point cloud and a second point cloud using an optical phase profilometry system having at least a first opposing channel and a second opposing channel. The first opposing channel and the second opposing channel each generate a respective first point cloud and a second point cloud. Additionally, the first point cloud and the second point cloud each have a plurality of surface points corresponding to measured surface points along the target surface.

[0047] Method 400 proceeds to block 420 where the volume of the measurement object corresponding to the target surface is divided into a set of voxels. Method 400 proceeds to block 430 where for each surface point in each of the first and second point clouds, the signed distance function (SDF) of the point and corresponding weights are added to the volume of voxels along each of the reflected (imaging system) rays of the first and second counter-located channels. The signed distance is the distance from the point measured in a particular direction to a reference surface. For example, sea level is the signed distance to sea level, with positive values ​​for points above sea level and negative values ​​for points below sea level. The signed distance function (SDF) is a function that calculates this distance for a specified point, where the signed distance represents the distance from an individual voxel to a point in the point cloud.

[0048] The method 400 proceeds to block 440, where a surface shape map corresponding to the target surface within the volume of voxels is generated by identifying level sets (e.g., theoretical zero crossings) of the SDFs of each of the first and second clouds of points.

[0049] 10A-10C are schematic diagrams illustrating an example of merging point clouds from counter-disposed channels. More specifically, FIG. 10A-10C illustrate an example of the execution of method 400. It should be understood that the volumes and voxels illustrated in FIG. 10A-10C are 3D objects that define the entire measurement space of the optical phase profilometry system. For clarity of illustration, the volumes and voxels are shown in 2D.

[0050] 10A is a schematic diagram illustrating an example of a point cloud corresponding to a first counter-disposed channel in a volume of voxels. The voxel volume 300 includes a measurement volume 302, a voxel(s) 304, a surface point(s) 306, and a reflected (imaging system) ray(s) 308. In general, the voxel volume 300 represents a measurement volume 302 having a plurality of voxels 304 to which a plurality of surface points 306 are added, each with its own SDF and corresponding weight along its respective reflected (imaging system) ray 308.

[0051] 10B is a schematic diagram illustrating an example of a point cloud corresponding to a second counter-disposed channel within a volume of voxels. The voxel volume 310 includes a measurement volume 312, a voxel(s) 314, a surface point(s) 316, and a reflected (imaging system) ray(s) 318. In general, the voxel volume 310 represents a measurement volume 302 having a plurality of voxels 304 to which a plurality of surface points 316 are added, each with its own SDF and corresponding weight along its respective reflected (imaging system) ray 318.

[0052] 10C is a schematic diagram showing an example of merging point clouds from a first and a second counter-disposed channel. The voxel volume 320 includes a measurement target volume 322, voxels 324, a reflected (imaging system) ray 326 of the first counter-disposed channel(s), a reflected (imaging system) ray 328 of the second counter-disposed channel(s), and a target surface approximation 330. In general, the voxel volume 320 shows a target surface shape map generated by identifying a level set (e.g., theoretical zero crossings) of the SDFs of each of the first and second point clouds, as represented by the target surface approximation 330, which corresponds to a target surface within the measurement target volume 322.

[0053] Another particular challenge of typical optical phase profilometry systems is the accurate measurement of target surfaces and / or objects having rounded / spherical shapes (e.g., balls), especially when these target surfaces and / or objects have specular portions that can cause glint.

[0054] As mentioned above, a typical optical phase profilometry system will typically have a single projector that illuminates a surface with a structured pattern and multiple oblique cameras that observe the surface. Alternatively, an opposite but equivalent configuration with a single camera and two oblique projectors is used.

[0055] FIG. 11A is a schematic diagram illustrating an example of an optical phase shape measurement system. System 350 includes a first oblique projector 352, a second oblique projector 354, a camera 356, a spherical target 358, projections 360 and 362, a first oblique projector measurement 364, a second oblique projector measurement 366, and a height error 368. System 350 is subject to the effects of the phenomenon described with respect to FIG. 2A and FIG. 2B, namely, measurement errors due to glint. As shown in FIG. 11A, first oblique projector 352 in combination with camera 356 produces measurement 364 indicative of height error 368 (e.g., height divot error) of the top of spherical target 358. Similarly, second oblique projector 354 in combination with camera 356 produces measurement 366 indicative of height error 368 (e.g., height divot error) of the top of spherical target 358. This error is caused by the occurrence of glint due to the combination of the canted angles of each of the projectors and the viewpoint of the camera combined with its point spread function, as previously described. Each projector is subject to glint from a different location on the spherical target 358. In a triangulation system such as an embodiment of the present invention, the measured location of a feature is displaced by the glint. The magnitude of the displaced is modulated by two factors: 1) the distance to the glint and 2) the product of the glint and its point spread function (PSF). Thus, a plot of the displaced tends to have a slope over a local region corresponding to the width of the PSF. We call this a "divot" because it resembles a divot caused by an unpredictable golf swing.

[0056] 11B is a schematic diagram illustrating an example of a three-dimensional shape of a spherical target 358 generated using system 350. As shown in shape 370, a height error 368 occurs, indicating a "height divot error." As shown, there is a clear decrease in height measurement, as reflected by the height measurement unit 374 at the top of the spherical target 358, which should be at the apex of the height of the spherical target 358. For example, compare the height reading at the outside 372 of the target 358 compared to the height reading in the area shown as height error 368.

[0057] This height divot error is not resolved by using a single projector multiple oblique camera system as shown in FIG. 11C.

[0058] 11C is a schematic diagram showing an example of the three-dimensional shape of a spherical target 358. On the left is the measured Z-height of the target 358, showing the actual height shape of the target 358. As shown, the target 358 is at the apex of its height at the center of the image (corresponding to the top of the target 358), as reflected by height measurement unit 376. However, on the right is the three-dimensional shape of the spherical target 358 produced by a single projector / two oblique camera system. As shown, the height error 368 occurs at the center of the image (corresponding to the top of the target 358), as reflected by height measurement unit 378.

[0059] FIG. 12 is a schematic diagram illustrating an example of an optical phase profilometry system. The system 500 includes a working coaxial illumination source / imaging system pair 502, an oblique working coaxial illumination source / imaging system pair 504 and 506, a spherical target 508, and a channel 510. The system 500 is configured to eliminate or reduce measurement errors as described with respect to FIGS. 11A-11C. In one example, measurement errors due to glint are compensated for by the alignment geometry of the system 500, e.g., glint-induced errors tend to be approximately equal and opposite (and thus cancel each other out) when using a four-channel alignment geometry (with two pairs of oppositely positioned channels). The system 500 is configured to have an alignment geometry that produces six channels (e.g., four diffuse channels [optical channels configured to capture diffuse reflection] and two specular channels [optical channels configured to capture specular reflection]) in one example. In one example, system 500 compensates for measurement errors caused by specular or partially specular targets and / or objects by creating an alignment geometry configured to capture both specular and diffuse reflections, thereby compensating for errors caused by either, which is described in more detail in Figures 13A-13C.

[0060] FIG. 13A is a schematic diagram illustrating an example of an optical phase profilometry system. Specifically, FIG. 13A illustrates two of the six channels created by the alignment geometry of system 500. The first channel is formed by projection 512 and reflection 514. Projection 512 is projected by an illumination source of operating pair 502 and is reflected from target 508 as reflection 514 and captured by an imaging system of operating pair 504. The second channel is formed by projection 516 and reflection 518. Projection 516 is projected by an illumination source of operating pair 504 and is reflected from target 508 as reflection 518 that is captured by an imaging system of operating pair 502. The first and second channels form a first pair of opposingly disposed channels.

[0061] FIG. 13B is a schematic diagram illustrating an example of an optical phase profilometry system. Specifically, FIG. 13B illustrates two of the six channels created by the alignment geometry of system 500. A third channel is formed by projection 512 and reflection 520. Projection 512 is projected by an illumination source of operating pair 502 and is reflected from target 508 as reflection 520 and captured by an imaging system of operating pair 506. A fourth channel is formed by projection 522 and reflection 524. Projection 522 is projected by an illumination source of operating pair 506 and is reflected from target 508 as reflection 524 that is captured by an imaging system of operating pair 502. The third and fourth channels create a second pair of opposingly disposed channels.

[0062] 13C is a schematic diagram illustrating an example of an optical phase profilometry system. Specifically, FIG. 13C shows two of the six channels generated by the alignment geometry of system 500. A fifth channel is formed by projection 516 and reflection 526. Projection 516 is projected by an illumination source of motion pair 504 that is reflected as reflection 526 from target 508 and captured by the imaging system of motion pair 506. A sixth channel is formed by projection 522 and reflection 528. Projection 522 is projected by an illumination source of motion pair 506 that is reflected as reflection 528 from target 508 and captured by the imaging system of motion pair 504.

[0063] In one example, the optical configuration of the working pairs 502, 504, and 506 is telecentric. In one example, the lens assemblies of the working pairs 502, 504, and 506 include multi-element / compound lens assemblies with entrance or exit pupils at infinity. The telecentric optical configuration ensures that the nominal projection directions (as represented by 512, 516, and 522) and reflection directions (as represented by 514, 518, and 520) are the same across the field of view of the working pairs 502, 504, and 506. With the projection and reflection angles being equal across the field of view, the advantage of the opposing channels is maintained in that each reflection per channel is received by the respective imaging system for each working pair. In one example, the illumination generated by the illumination source enters the multi-element / compound telecentric lens assembly and is substantially parallel and therefore highly concentrated when it exits the working pair. Thus, nearly all the light generated by the illumination source strikes the target surface and the resulting reflection is captured by the imaging system.

[0064] 14 is a flow chart illustrating an example method for generating a geometry of a target surface and / or object. Method 600 begins at block 610, where a first geometry of the target surface and / or object is generated. This can be done by combined point cloud refinement as shown at block 612, point cloud merging from opposingly positioned channels as shown at block 614, for example by using an SDF, and / or others 616. Others 616 can include method 300 and / or method 400. Others 616 can include cross sections. Others 616 can include any other suitable technique for generating a geometry of the target surface and / or object.

[0065] Method 600 proceeds to block 620 where a second dimension of the target surface and / or object is generated. This may be done by combined point cloud refinement as shown in block 622, point cloud merging from opposingly positioned channels as shown in block 624, for example by using an SDF, and / or others 626. Other 626 may include method 300 and / or method 400. Other 626 may include cross sections. Other 626 may include any other suitable technique for generating a dimension of the target surface and / or object.

[0066] The method 600 proceeds to block 630 where a third dimensional shape of the target surface and / or object is generated. This can be done by a comparison of the first dimensional shape and the second dimensional shape, as indicated at block 632. This can be done by a combination of the first dimensional shape and the second dimensional shape, as indicated at block 634. This can be done by other techniques, as indicated at block 636. For example, other 636 can include taking an average (e.g., a weighted average) of the first dimensional shape and the second dimensional shape. Other 636 can include any other suitable technique for generating the third dimensional shape based on the first and second dimensional shapes.

[0067] 14 shows a particular order of steps for illustrative purposes, it should be understood that some or all of these steps may be performed in any number of orders, including, but not limited to, simultaneously, in parallel, sequentially, non-parallel, non-sequentially, and any combination thereof. No particular order of the steps of method 600 should be implied and / or interpreted from the illustration.

[0068] 15A-15B are schematic diagrams illustrating an example of a dimensional shape generation of a target surface and / or an object. More specifically, FIG. 15A-15B show some examples of the execution of the steps of the method 600.

[0069] 15A is a schematic diagram illustrating an example of a target surface and / or object geometry generation using iterative connection point cloud refinement. Geometry 650 is a representation of the geometry of a number of target objects 652, shown here as spherical objects. Geometry 650 represents dimensional information of a number of target objects 652, including their y coordinates (e.g., length and / or width) measured in micrometers, their x coordinates (e.g., length and / or width) measured in micrometers, and their heights (e.g., z coordinates) represented at 656. Geometry 658 is a cross-section of a selected target object as indicated by line 654. Geometry 658 represents dimensional information of a selected target object 654, including their x coordinates (e.g., length and / or width) measured in micrometers, and their heights (e.g., z coordinates) represented at 657. Geometry 650 and / or 658 can be generated by method 300 in one example.

[0070] FIG. 15B is a schematic diagram illustrating an example of target surface and / or object geometry generation using point cloud merging from opposingly positioned channels with SDF. Geometry 660 is a representation of the geometry of multiple target objects 662, shown here as spherical objects. Geometry 660 represents dimensional information of multiple target objects 662, including their y coordinates (e.g., their length and / or width) measured in micrometers, their x coordinates (e.g., their length and / or width) measured in micrometers, and their height (e.g., z coordinate) represented at 666. Geometry 668 is a cross-section of a selected target object as shown by line 664. Geometry 668 represents dimensional information of selected target object 664, including their x coordinates (e.g., their length and / or width) measured in micrometers, and their height (e.g., z coordinate) represented at 657. Geometry 660 and / or 668 can be generated by method 400 in one example.

[0071] According to Scheimpflug's principle, if the object plane is not perpendicular to the optical axis, the image plane will also be tilted. Figure 16 is a schematic diagram showing an example of a tiled object and image plane geometry. An object 708 located in a plane 716 has a normal 714 tilted at an angle 718 with respect to the optical axis 712. A bundle of rays 712 enters a lens 706 and is focused on an image sensor 704 located at an angle 720 with respect to the optical axis, denoted theta (θ). The lens 706 is perpendicular to the optical axis. According to Scheimpflug's principle,

[0072]

number

[0073] Large magnifications m can result in large image plane tilts required by the Scheimpflug principle. For example, for m=1.5, theta is 40.9°. This required angle is outside the usable range of typical imaging systems, especially those that use microlenses.

[0074] As taught in US Pat. No. 397,254, the image plane tilt 720 can be substantially reduced by introducing prisms into the optical path. Unfortunately, the reduction of the image plane tilt is accompanied by the introduction of various aberrations, especially astigmatism and lateral chromatic aberration. To avoid this problem, additional prisms can be introduced into the optical path and the performance can be optimized by a lens design program, as is well known in the art. In a preferred embodiment, three prisms are used for proper control of aberrations, and at least two types of glass are used for control of lateral chromatic aberration, similar to the need for two types of glass in an achromatic lens, as is well known in the art. The number of prisms depends on the magnification, field of view, and tilt degree and can be selected without departing from the scope of the invention.

[0075] 17 is a schematic diagram of an example of an optical phase profilometry system. System 760 includes lens 756, focal plane 754, output ray bundle 758, and prism assembly 752, which includes prisms 761, 762, 763, and wedge angles 764, 765, and 766. Light exiting the rear surface of lens 756 is refracted by prism assembly 752 and converges to focal plane 754, which is approximately perpendicular to the optical axis.

[0076] In one example, system 750 enables a high resolution and / or high magnification optical phase profilometer by using multiple prisms (e.g., 761, 762, and / or 763) to reduce the required image sensor tilt angle (e.g., Scheimpflug angle) to compensate, reduce, and / or eliminate aberrations (e.g., chromatic aberration) as light passes through the imaging system to reach the focal plane. In one example, at least one of prisms 761, 762, and / or 763 includes a different type of glass than at least one of the other prisms. In another example, each of prisms 761, 762, and 763 includes a different type of glass than each of the other prisms. In one example, prisms 761, 762, and / or 763 include wedge prisms. In one example, at least one of prisms 761, 762, and / or 763 includes a different wedge angle than at least one of the other prisms. In one example, at least one of the prisms 761, 762, and / or 763 has a wedge angle (e.g., wedge apex angle) oriented in a different direction than at least one of the other prisms. In one example, the prism assembly 760 includes a first prism (e.g., 761) having a wedge angle (e.g., wedge apex angle 764) oriented in a first direction, a second prism (e.g., 762) having a wedge angle (e.g., wedge apex angle 765) oriented in a second direction, and a third prism (e.g., 763) having a wedge angle (e.g., wedge apex angle 766) oriented in a second direction. Although three prisms are shown in FIG. 17, any number of prisms including any number of materials and having wedge angles oriented in any number of directions may be used.

[0077] Another challenge of a typical optical phase profilometry system is measurement errors due to changes to the sensor or the sensor's environment. For example, thermal scaling, mechanical drift, along with various other factors, can cause measurement output errors. As mentioned above, a typical optical phase profilometry system has multiple imaging paths that view the target surface from different perspectives (e.g., single camera / multiple projectors, single projector / multiple cameras, etc.). Each camera / projector pair forms a channel (imaging path / optical path) and provides a unique perspective of the surface to be measured. The surface is often reconstructed by each channel separately (e.g., in a point cloud), and these reconstructions are combined into the final dimensional shape (e.g., height map) of the target surface.

[0078] FIG. 18A is a schematic diagram illustrating an example of an optical phase profilometry system. The system 800 includes a camera 802, projectors 804 and 806, a target surface 808, projections 810 and 812, and a reflection 814. The projection 810 and the reflection 814 form a first channel 816, and the projection 812 and the reflection 814 form a second channel 818. The reflection 814 is shown as a single line in FIG. 18 for clarity of illustration, but it should be understood that the projection cast by the projector 804 creates a separate reflection from the projection cast by the projector 806. From each of the first and second channels, a point cloud is generated that includes a dimensional reconstruction of the target surface 808. The first channel 816 generates a first point cloud 820, and the second channel 818 generates a second point cloud 822. Due to the alignment geometry of projectors 804 and 806 and the relative height of target surface 808, there are blind spots / shadows in each point cloud, as shown at 824 and 826. Thus, the point clouds are combined to generate a complete geometry of target surface 808, as shown at 828.

[0079] Combining these separate point clouds requires that they are well aligned with each other, which is typically ensured by a calibration procedure for the imaging sensor. However, over time, changes to the sensor (e.g., thermal scaling, mechanical drift, etc.) can cause the individual point clouds to shift (e.g., become misaligned), as discussed above. In such cases, the final combined geometry can be less accurate and repeatable than the geometry would have been if the channels were still precisely aligned.

[0080] FIG. 18B is a schematic diagram illustrating an example of an optical phase profilometry system. FIG. 18B is similar to FIG. 18, and thus similar elements are numbered similarly. FIG. 18B shows a misaligned (e.g., shifted) projector 806, as represented by the fine alignment 807 compared to the actual alignment of projector 806. This results in measurement errors, as shown by surface points 819. This misalignment shifts the point cloud 823 of the second channel 818 relative to the point cloud 820 of the first channel 816. The combination of point cloud 820 and point cloud 823 results in an incorrect geometry of target surface 808, as shown at 829.

[0081] These types of errors can be resolved by recalibrating the sensor, for example by following a field calibration procedure. However, these field calibration procedures often require the system to stop measuring in order for an operator to work with the optical phase profilometry system. This can be highly disruptive to production and reduce the throughput of online inspection operations. Less disruptive techniques are needed. An example of such a technique is shown below.

[0082] Using dynamic compensation methods, errors can often be resolved on the fly (e.g., while the system continues to operate). The relative errors between the individual point clouds can be dynamically estimated and then compensated for before combining them into the final geometry of the target surface. A common method is to estimate the relative errors between the individual point clouds by computing a transformation that optimally aligns the individual point clouds in 3D. In general, the transformation can consist of a rotation, translation, scaling, or any other form of transformation that models the expected changes due to modifications to the imaging sensor (e.g., thermal scaling, mechanical drift, etc.). In one example, a 3D translation can be sufficient to model small changes to the alignment geometry of the system.

[0083] 19 is a flow chart illustrating an exemplary method for dynamically compensating for errors in an optical phase profilometry system. Method 900 begins at block 910, where a first point cloud and a second point cloud are generated using a first channel and a second channel (e.g., imaging path / optical path). Although two separate point clouds from two separate channels are described above, in other examples, any number of channels can be used and any number of point clouds can be generated. In other examples, each channel in a particular optical phase profilometry system can generate a separate point cloud for a target surface and / or object.

[0084] The method 900 proceeds to block 920 where it is determined which portions of the field of view are not empty. In one example, this determination includes determining which portions of the point cloud(s) have surface points and which do not have surface points. In one example, this determination is performed by an optical phase profilometry system, which may have a number of controllers and / or processors (e.g., microprocessors) configured to receive sensor signal(s) (e.g., from an image sensor) indicative of any number of characteristics (e.g., dimensional information about the target surface and / or object) and determine, based on the sensor signals, which portions of the field of view (e.g., respective areas, volumes of objects being measured, the environment the system is viewing, etc.) are not empty (e.g., portions that have corresponding surface points).

[0085] The method 900 proceeds to block 930, where the first and second point clouds are aligned in the non-sky portion of the determined field of view (e.g., the non-sky portion of the point clouds). This alignment can be done in a variety of ways. In one example, an optimal transformation for each point cloud is calculated as shown in block 932. In one example, a rotation is performed as shown in block 933. In one example, a translation is performed as shown in block 934. In another example, a scaling is performed as shown in block 935. In another example, a weighting is performed as shown in block 936. In one example, the weighting in block 936 includes weighting the surface points of the point cloud by their confidence value (e.g., their signal-to-noise figure). In another example, the point clouds can be aligned by performing any other number of transformations, particularly transformations that model changes to the imaging sensor (e.g., thermal scaling, mechanical drift, etc.), as shown in block 937.

[0086] Method 900 proceeds to block 940, where a relative error for each point cloud is determined. In one example, determining the relative error for each point cloud includes identifying an inverse of an optimal transformation for each point cloud. Method 900 proceeds to block 950, where a geometry of the target surface and / or object is generated. In one example, generating the geometry of the target surface and / or object includes subtracting the relative error. In one example, generating the geometry includes calculating an optimal transformation for each point cloud, identifying a relative error for each point cloud (e.g., an inverse of the optimal transformation), subtracting the respective relative error from each point cloud, and combining the compensated and / or corrected point clouds.

[0087] FIG. 20 is a schematic diagram showing an example of dynamic compensation. At the top of FIG. 20, a first point cloud 962 and a second point cloud 964 are shown. In one example, the point clouds 962 and 964 are generated by a first channel and a second channel (e.g., 816 and 818). Each of the first point cloud and the second point cloud has a plurality of respective surface points as indicated by dots. As shown in FIG. 20, both the point cloud 962 and the point cloud 964 have measurement errors due to changes to the image sensor (e.g., mechanical drift as shown in FIG. 18A). The generated point clouds do not accurately represent the target surface (e.g., 808) being measured. However, dynamic compensation can compensate and / or correct the measurement errors of each point cloud as described above (e.g., method 900) to create a target surface geometry that accurately or substantially accurately represents the actual geometry of the target surface as shown by the aligned geometry 966.

[0088] In one example, the dynamic compensation includes generating a first point cloud and a second point cloud from respective first and second channels of an optical phase profilometry system observing the target surface, the first point cloud and the second point cloud indicating dimensional information regarding a field of view of the optical phase profilometry system, determining which portions of the field of view are non-empty (e.g., which portions of the field of view have surface points), aligning the first point cloud and the second point cloud in the non-empty regions by calculating an optimal transformation for each point cloud and identifying (e.g., calculating) a relative error for each point cloud, subtracting the relative error from each point cloud, and generating a dimensional profile (e.g., a height map) by subtracting the relative error from each point cloud and combining the corrected and / or compensated first and second point clouds. In one example, calculating the optimal transformation includes weighting the surface points of each of the first and second point clouds by their confidence values ​​(e.g., signal-to-noise factor). In one example, calculating the optimal transformation includes performing a rotation, a translation, a scaling, or any other transformation or combination thereof.

[0089] In some cases, determining the relative error for each point cloud can be computationally intensive and can slow down the overall reconstruction of the target surface and / or object (e.g., generating a height map). In particular, in an online inspection environment where 3D acquisitions are made continuously (e.g., during automated optical inspection of circuit boards in an electronic assembly process), it may not be necessary or desirable to determine the relative error at each frame. In one example, the relative error is determined periodically and then applied at each acquisition (e.g., geometry, height map, etc.). In addition, because each determination of the relative error may contain some noise, the determinations can be smoothed in time by keeping a running average of the relative error determinations.

[0090] 21 is a flow chart illustrating an exemplary method of dynamic compensation. Method 1000 begins at block 1010, where for each individual three-dimensional acquisition of a target surface and / or object, it is determined whether the time since the last determination of the relative error average exceeds a threshold. If it is determined at block 1010 that the time since the last relative error average was determined does not exceed the threshold, then the most recent relative error average is applied to each point cloud. Method 1000 proceeds to block 1025, where a dimensional geometry (e.g., a height map) is generated for the target surface and / or object.

[0091] Returning to block 1010, if it is determined that the time since the last relative error average exceeds the threshold, method 1000 proceeds to block 1030 where a new relative error average is determined. In one example, the new relative error average is determined by determining an average of a transformation (e.g., an optimal transformation). In one example, the average of the transformation is determined according to:

[0092]

number

[0093] The method 1000 proceeds to block 1040, where the relative error mean (e.g., T avg ) is subtracted from each point cloud. Method 1000 proceeds to block 1050 where the corrected and / or compensated (e.g., relative error mean subtracted) point clouds are combined. Method 1000 proceeds to block 1060 where a dimensional shape (e.g., a height map) is generated based on the combination of the corrected and / or compensated point clouds. Method 1000 can continue for each three-dimensional acquisition until the optical inspection operation is completed.

[0094] In some cases, there may be a limit to the amount of error that can be compensated for. When the relative error determination begins to approach this limit, a full field or factory calibration may need to be performed. In one example, the optical phase profilometry system may determine whether the relative error determination exceeds a threshold (e.g., amount of error, number of iterations, etc.), and upon determining that the threshold has been exceeded, the optical phase profilometry system may generate a communication, warning, alert, or some other indication (e.g., display on a user interface) that calibration is required.

[0095] Another challenge faced by typical optical phase profilometry systems is caused by limitations of the system that limit the quality and accuracy of the captured images of the target surface and / or object.

[0096] Optical phase profilometry is limited by the quality of the projected sine wave. Digital light processing (DLP) systems are flexible but have a finite pixel size that creates a step-like pattern in the phase of the projected sine wave. DLP systems also have a limited dynamic range. For example, a typical DLP system uses an 8-bit image where the sine wave is quantized to 256 levels. Unfortunately, this digital-to-analog (e.g., light level) conversion is not perfectly linear. Real applications can have significant integral and / or differential linearity errors, with differential linearity errors being more difficult to address (e.g., correct, compensate, etc.). On the other hand, integral errors, along with offset and gain errors, can be removed by multi-phase reconstruction. For example, three-phase reconstruction is insensitive to offset gain errors, four-phase reconstruction is insensitive to second-order integral errors, and higher order reconstructions compensate for higher order integral errors. On the other hand, differential linearity errors add "random" (e.g., difficult to predict) phase noise across all parts of the sine wave.

[0097] DLP systems generate multiple gray levels through a time-slicing method. To generate gray levels, individual mirrors in a DLP system are toggled on and off, similar to pulse-width modulation (PWM). Because mirrors are mechanical devices, their switching time is limited, and therefore increasing the number of gray levels projected (e.g., increasing the number of bits in an image) also increases the frame time of the image. For example, a single-bit image may not look like a sine wave, but it can be projected very quickly compared to a high-bit image, thus improving the system output time.

[0098] FIG. 22A is a schematic diagram showing an example of a DLP projector system. The system 1100 includes a light source 1102, a digital micromirror device 1104, an optical low pass filter (OLPF) 1106, and a digital projection lens assembly 1108, which further includes lenses 1110 and 1112. The system 1100 further includes a source light beam 1114, a projection 1116, and a target surface 1118. In general, the operation of the system 1100 is as follows. The source light beam 1114 originates from the light source 1102 (e.g., an LED) and passes through a condenser lens 1102, which converges the source light beam 1114 to effectively illuminate the digital micromirror device 1104. After passing through the condenser lens 1102, the source light beam 1114 strikes the digital micromirror device, from which it is projected as a digital projection 1116. The digital projection 1116 passes through an OLPF 1106, as described above and in further detail below (FIG. 22B), which is configured to reduce and / or eliminate problems of typical DLP systems. The projection 1116 passes onto a projector lens assembly 1108, which includes lenses 110 and 112 arranged to focus the projection 1116 to a desired level on a target surface 1118. The projection 1116 then passes onto and is reflected from the surface target 1118, after which it is captured by an imaging system (e.g., a camera) which uses a sensor (e.g., an image sensor) to generate a geometry of the target surface 1118. Although a particular arrangement of components is shown in FIG. 22 for illustrative purposes, system 1100 may include any number of additional components and / or devices, including, but not limited to, additional digital micromirror devices, additional lenses (e.g., molded lenses) including additional lenses in projector lens assembly 1108 arranged in various manners, color filter(s), additional light sources, and other components and / or devices necessary or desirable for such a system.

[0099] FIG. 22B is a simplified block diagram illustrating an example of an OLPF. The OLPF 1106 includes birefringent material(s) 1120 and others 1128. The birefringent material(s) 1120 include quartz plate(s) 1122, wave retarder(s) 1124, and others 1126. In one example, the OLPF 1106 is made of a layer of birefringent material. In one example, the quartz plate(s) 1122 includes a first quartz plate that splits a point from the digital micromirror device into a pair of points and a second quartz plate that splits two points into four points. In one example, the four points are arranged in a square pattern. In one example, the wave retarder(s) 1124 includes a first quarter-wave retarder that converts the polarization of each copy of the image (projected by the digital micromirror device) to circular polarization, and a second quarter-wave retarder that converts the polarization of each copy of the image to circular polarization. In one example, the second quarter-wave retarder ensures that the light output by the OLPF 1106 (e.g., projection 1116) is circularly polarized. In one example, the OLPF 1106 includes four layers of birefringent material, where the first layer includes a first quartz plate, the second layer includes a first quarter-wave retarder, the third layer includes a second quartz plate, and the fourth layer includes a second quarter-wave retarder.

[0100] The birefringent material(s) 1120 may include any other number of birefringent materials, including but not limited to any other number of crystal structures. The OLPF 1106 may include any other number of materials and / or components necessary or desirable for optical low pass filtering and / or reconstruction filtering in an imaging system.

[0101] The vertical resolution of an optical phase profilometry system is limited by the highest frequency that is projected. In a DLP phase profilometry system, the frequency is typically about 4 digital micromirror device pixels per period. A frequency of 1 / 4 cycle per pixel is effectively a square wave. By applying the OLPF 1106 to a DLP phase profilometry system, the square wave can be converted to a sine wave. The advantage of starting with a square wave is that it is a binary image (e.g., only two levels). Using a binary pattern increases the speed of the system 1100 compared to other typical DLP phase profilometry systems. For example, an 8-bit DLP image requires tens of milliseconds per image, but a single-bit image, as in the case of the system 1100, can be projected well under a millisecond, thus increasing the speed of image acquisition and geometry output.

[0102] If a lower frequency sine wave is required, additional bit levels can be added to the sine wave, for example a sine wave with 6 pixels per cycle can be adequately projected with only three gray levels (e.g., 0, 0.5 and 1.0). In one example, a sine wave repeating every n pixels provides n / 2 gray levels. System 1100 is configured to maximize the signal level of the desired projection frequency and minimize undesirable artifacts (e.g., harmonics of the projection frequency and the "screen door effect").

[0103] 23 is a flow chart illustrating an example of projecting light onto a target surface. Method 1200 begins at block 1210, where light is projected onto a digital micromirror device (DMD) using a light source. Before reaching the DMD surface, the light may first pass through a lens assembly 1211, which in one example may include a focusing lens. The light may also pass through any other devices and / or components on the way to the DMD. The light source may include an LED 1214. The light source may include any other type of light source 1215 suitable for projecting light onto the DMD.

[0104] The method 1200 proceeds to block 1220 where the projected light (projection) is reflected towards a target surface and / or object forming a DMD. The method 1200 proceeds to block 1230 where the projected light is filtered after being reflected from the DMD but before reaching the target surface and / or object. This filtering can be done using an OLPF 1231. This filtering can be done using other types of filters 1232. The filter can include any number(s) of wave plates 1233, such as quartz wave plates. The filter can include any number(s) of wave retarders 1234, such as quarter wave retarders. The filter can include any other number of materials, components, and / or devices, including but not limited to birefringent materials, crystalline structures, and any other materials, components, and / or devices necessary and / or desirable for filtering and / or reconfiguration. In one example, the filter is an OLPF consisting of multiple layers, where a first layer includes a first quartz waveplate, a second layer includes a first quarter wave retarder, a third layer includes a second quartz waveplate, and a fourth layer includes a second quarter waveplate. In one example, the filter of block 1230 can include OLPF 1106.

[0105] Method 1200 proceeds to block 1240, where the filtered projection is focused onto a target surface and / or object. The filtered projection may first pass through a lens assembly 1241. In one example, the lens assembly of block 1241 is projector lens assembly 1108. The filtered projection may first pass through aperture(s) 1242. The filtered projection may pass through any number of other devices and / or components on its way to the target surface and / or object.

[0106] In one example, method 1200 includes projecting light from an LED onto a DMD configured to reflect light at a frequency of ¼ cycle per pixel (e.g., reflect the projection as a square wave). The light from the DMD is then filtered by an OLPF configured to “reconstruct” the square wave into a sine wave. The OLPF includes a first quartz waveplate configured to split points from the DMD into pairs of points, a first ¼ wave retarder configured to convert the polarization of each copy of the image to circular polarization, a second quartz waveplate configured to split two points into four points, the four points being arranged in a square pattern, and a second ¼ wave retarder configured to convert the output light (e.g., light passing through the OLPF and exiting the OLPF toward the target surface and / or object) to circular polarization. The filtered light then passes through a projector lens assembly configured to focus the filtered light onto a target surface and / or object, the lens assembly including a first lens having a first convex surface and a second lens having a second convex surface, the first convex surface and the second convex surface facing in opposite directions.

[0107] Another challenge associated with typical phase profilometry systems relates to tilting of targets, especially those with specular target surfaces. In a typical system, the illumination source has a numerical aperture that defines the boundaries of the ray bundles emanating from each point on the target surface. Typically, the system is aligned such that for non-tilted targets, the center of the illumination source pupil intersects the center of the imaging system pupil. However, tilting of the target surface disrupts this alignment, e.g., when the target surface is tilted, the ray bundles from the illumination source (reflected from the target as a reflection) pass through the receiver aperture. For example, when the target surface is tilted from an ideal plane and the triangulation angle is changed, deflectometer errors occur. This error is exacerbated when the reflections overlap at the edges, also known as vignetting. In some cases, such as when the target is extremely tilted, the reflections no longer overlap but rather are invisible to the imaging system, and thus dimensional information of the target surface cannot be generated. Errors due to target surface tilt are further described in U.S. Patent Application Publication No. 2019 / 0226835.

[0108] As an example of deflectometer error, when a specular target of non-zero height (e.g., an ideal flat surface) is tilted, the height measured by the imaging system changes. For example, in a phase profilometry system with a 60° acceptance angle, a target 1 mm from the camera's best focus tilted by 1° will have a height error of approximately 40 micrometers.

[0109] 24 is a schematic diagram illustrating an example of an optical phase profilometry system. System 1300 includes a first operative coaxial imaging system / illumination source pair 1302 including an imaging system 1106, an illumination source 1108, and a beam splitter 1308. System 1300 further includes a second operative coaxial imaging system / illumination source pair 1304 including an imaging system 1312, an illumination source 1314, and a beam splitter 1316. The operative pairs 1302 and 1304 are configured such that each imaging system (1306 and 1312) shares an optical path with each illumination source (1308 and 1314). The operating pairs 1302 and 1304 utilize respective beam splitters 1310 and 1316, which in one example are configured at a 45° angle with respect to the respective imaging systems (1306 and 1312) and respective illumination sources (1308 and 1314). In one example, the imaging systems 1306 and / or 1312 can include a camera. In one example, the imaging systems 1306 and / or 1312 can include an image sensor(s). In one example, the illumination sources 1308 and / or 1314 can include a projector. In one example, the illumination sources 1308 and / or 1314 can include a DLP projector, which can include a digital micromirror device (DMD) and an OLPF. The system 1300 further includes a target surface 1318 , illumination 1320 and 1324 , and reflections 1322 and 1326 .

[0110] The system 1300 is arranged such that the operation pairs 1302 and 1304 form an oblique pair (e.g., arranged at an oblique angle to the target surface 1318 and a normal [e.g., perpendicular]). The system 1300 is further arranged such that the operation pairs 1302 and 1304 are symmetrical with respect to each other. In other words, their oblique angles to the target surface 1318 and the normal are equal but in opposite directions (e.g., on either side of the normal). In this manner, the operation pairs 1302 and 1304 form counter-disposed channels. In one example, they form a first specular reflection channel and a second specular reflection channel. The first channel includes illumination 1320 projected from the illumination source 1308 and a reflection 1322 reflected from the target surface 1318. In one example, the reflection 1322 includes a specular reflection. The second channel includes illumination 1324 projected from the illumination source 1314 and a reflection 1326 reflected from the target surface 1318. In one example, the reflection 1326 includes a specular reflection.

[0111] The alignment geometry of system 1300 has the advantage that height reconstruction errors due to the tip or tilt of the specular target surface are compensated. The errors caused by the tilt of the target are equal and opposite for each motion pair 1302 and 1304 in system 1300. Thus, when the point clouds generated by motion pairs 1302 and 1304 are combined, the specular error caused by the tilt of the target is corrected. Furthermore, in one example, the optical apertures (not shown) of the illumination source and imaging system are equal (e.g., the numerical aperture of each channel is comparable), minimizing the specular error and making the resulting representation of the target surface more accurate.

[0112] 25 is a schematic diagram illustrating an example of an optical phase profilometry system. The system 1400 includes a first operative coaxial imaging system / illumination source pair 1402 including an imaging system 1408, an illumination source 1410, and a beam splitter 1412. The system 1400 further includes a second operative coaxial imaging system / illumination source pair 1404 including an imaging system 1414, an illumination source 1416, and a beam splitter 1418. The system 1400 further includes a third operative coaxial imaging system / illumination source pair 1406 including an imaging system 1420, an illumination source 1422, and a beam splitter 1424. The operative pairs 1402, 1404, and 1406 are configured such that each imaging system (1408, 1414, and 1420) shares an optical path with each illumination source (1410, 1416, and 1422). The operating pairs 1402, 1404, and 1406 utilize respective beam splitters 1412, 1418, and 1424, which in one example are configured at a 45° angle with respect to the respective imaging systems (1408, 1414, and 1420) and the respective illumination sources (1410, 1416, and 1422). In one example, the imaging systems 1408, 1414, and / or 1420 can include cameras. In one example, the imaging systems 1408, 1414, and / or 1420 can include image sensors (one or more). In one example, the illumination sources 1410, 1416, and / or 1422 can include projectors. In one example, the illumination sources 1410, 1416, and / or 1422 can include DLP projectors, which can include digital micromirror devices (DMDs) and OLPFs. System 1400 further includes a target surface 1426, illumination 1428, 1434, and 1438, and reflectors 1430 / 1440 / 1450, 1432 / 1442, and 1436 / 1446.

[0113] System 1400 is arranged to form six oppositely disposed channels. In one example, four diffuse channels and two specular channels. In one example, the four oppositely disposed channels are configured to capture nominal diffuse reflection and the two oppositely disposed specular channels are configured to capture specular reflection. System 1400 is arranged such that operating pairs 1402 and 1406 comprise an oblique pair (e.g., positioned / aligned at an oblique angle to target surface 1426 and a "normal" [e.g., perpendicular and / or, in one example, operating pair 1404]). System 1400 is arranged such that operating pairs 1402 and 1406 are symmetrical with respect to each other. In other words, their oblique angles to target surface 1426 and the normal are equal but in opposite directions (e.g., on either side of the normal). In this manner, operating pairs 1402 and 1406 form a first oppositely disposed specular channel and a second oppositely disposed specular channel. The first channel includes illumination 1428 projected from illumination source 1410 and a reflection 1432 reflected from target surface 1426. In one example, reflection 1426 includes a specular reflection. The second channel includes illumination 1438 projected from illumination source 1422 and a reflection 1436 reflected from target surface 1426. In one example, reflection 1436 includes a specular reflection.

[0114] The system 1400 is further arranged such that four additional counter-positioned channels are formed, i.e., two counter-positioned channels between each diagonal pair (1402 and 1406) and the operating pair 1404. The operating pair 1404 is positioned / aligned approximately perpendicular (e.g., perpendicular) to the target 1426. The third counter-positioned channel includes illumination 1428 projected by the illumination source 1410 and a reflection 1440 reflected from the target surface 1426. In one example, the reflection 1440 is a diffuse reflection. The fourth counter-positioned channel includes illumination 1434 projected from the illumination source 1416 and a reflection 1446 reflected from the target surface 1426. In one example, the reflection 1446 is a diffuse reflection. The fifth counter-positioned channel includes illumination 1438 projected from the illumination source 1422 and a reflection 1450 reflected from the target surface 1426. In one example, the reflection 1450 is a diffuse reflection. A sixth oppositely disposed channel includes illumination 1434 projected from illumination source 1416 and a reflection 1442. In one example, reflection 1442 is a diffuse reflection.

[0115] In one example, the timing of the individual imaging system-illumination source pairs can be combined to minimize the time required to acquire all six channels of system 1400. Typically, imaging systems 1408, 1414, and 1420 are constructed using CMOS area array detectors. These types of imagers have the ability to control the exposure time such that the imager exposure time is a small fraction of the imager frame time (e.g., the time to acquire and read out and image). For example, if an imaging system (e.g., 1408, 1414, and 1420) can acquire images at 50 frames per second, the time between acquisitions is 1 / 50th of a second or 20 milliseconds. To acquire at least one image from each of the six channels in a serial mode that acquires at least two images from one imaging system, the time to acquire a full set of images is 120 milliseconds (6 images x 20 milliseconds). However, if the reflections (e.g., 1436, 1446, 1430, 1440, 1450, 1432, and 1442) are of sufficient intensity, the exposure time can be significantly shorter than 20 milliseconds, and the sequence of image acquisition can be combined between the three imaging systems. By combining the acquisition sequences of the three imaging systems, the exposure of one imaging system can be timed to be coincident with the readout time required by another imaging system to complete its frame time, resulting in an overall acquisition time of 120 / 3 or 40 milliseconds in this particular example.

[0116] Below is an example of a combined trigger sequence for acquiring images from six channels that can take advantage of the alignment geometry (eg, configuration) of system 1400 to minimize acquisition time. Trigger time 1: Illumination source 1410 exposes imaging system 1414; Trigger time 2: Illumination source 1416 exposes imaging system 1420; Trigger time 3: Illumination source 1422 exposes imaging system 1408; Trigger time 4: Illumination source 1422 exposes imaging system 1414; Trigger time 5: Illumination source 1410 exposes imaging system 1420; Trigger time 6: Illumination source 1416 exposes imaging system 1408

[0117] In this example, the time between trigger events is 20 / 3 = 6.6 ms. The time between triggering any single imaging system is 20 ms, but the total acquisition time for all six images is 6.6 ms x 6 = 40 ms. It should be understood that this is just one example of a sequence combination and that other sequence combinations can be used without departing from the spirit and scope of the present invention.

[0118] 26 is a simplified block diagram illustrating an example of an optical phase profilometry system. System 1500 includes illumination source(s) 1502, imaging system(s) 1504, electronics 1550, alignment geometry 1562, power source(s) 1564, memory 1565, user interface(s) 1566, beam splitter(s) 1567, remote device(s) 1568, housing(s), display(s) 1570, etc. 1571. System 1500 can include any of the embodiments and can incorporate any of the methods described herein.

[0119] The illumination source(s) 1502 include illumination generator(s) 1512, lens assembly 1514, filter(s) 1515, aperture(s) 1516, housing(s) 1518, etc. 1522. The illumination source(s) 1502 may include any of the embodiments described herein. The illumination generator(s) 1512 are configured to generate illumination (e.g., structured or patterned illumination) to be projected onto a target. The illumination generator(s) 1512 may include a spatial light modulator, a structured light generator, a DLP projector, transmissive liquid crystal, liquid crystal on silicon (LCOS), or any other suitable technology for projecting a structured light pattern, a digital micromirror device (DMD), or any other number of suitable illumination generator(s).

[0120] The lens assembly 1514 is generally configured to direct illumination from the illumination source(s) 1402 toward a target and may include telecentric lens assemblies, entrance and exit lenses, infinity entrance or exit pupils, two or more lenses, and lenses made from a variety of materials, including, but not limited to, polycarbonate, plastic, polymer, glass, liquid lens material, and any other suitable material. The lens assembly 1514 may include a condenser lens, a projector lens assembly including a first lens having a first convex surface and a second lens having a second convex surface, the first convex surface and the second convex surface facing in opposite directions.

[0121] The filter(s) 1515 are generally configured to filter and / or reconfigure the illumination. The filter(s) 1515 may include an OLPF, which includes any number of birefringent materials. The OLPF may include four layers, where a first layer includes a waveplate (e.g., a quartz waveplate), a second layer includes a quarter wave retarder, a third layer includes a second waveplate (e.g., a quartz waveplate), and a fourth layer includes a second quarter wave retarder. The filter(s) 1515 may include any number of other materials, components, and / or devices, including, but not limited to, crystalline structures, plates, and the like.

[0122] The aperture(s) 1516 are configured to direct illumination from the illumination source 1502 towards the target surface. The aperture(s) 1516 can include any size variation across an optical phase profilometry system, such as the systems and embodiments described herein. In one example, the numerical apertures of each operative coaxial imaging system-illumination source pair are comparable. In one example, the numerical apertures are of different sizes. In one example, for a channel (e.g., facing, specular, diffuse, etc.), the numerical aperture of the receiver (e.g., imaging system) is smaller than the numerical aperture of the light source (e.g., illumination source). In one example, for a channel (e.g., facing, specular, diffuse, etc.), the numerical aperture of the receiver (e.g., imaging system) is larger than the numerical aperture of the light source (e.g., illumination source). In one example, for a channel (e.g., facing, specular, diffuse, etc.), the numerical aperture of the receiver (e.g., imaging system) and the numerical aperture of the light source (e.g., illumination source) are comparable.

[0123] The housing(s) 1518 define a body of the illumination source(s) 1502 and are configured to house the components of the illumination source(s) 1502. The housing(s) 1518 can comprise any number of materials, including but not limited to plastic, polymer, metal, or any other suitable material. The housing(s) 1518 can include any of the embodiments described herein. Miscellaneous 1522 can include any other components suitable for use by the illumination source to project structured illumination onto a target.

[0124] The imaging system(s) 1504 include a lens assembly 1528, an aperture(s) 1530, a camera(s) 1532, an image plane(s) 1534, an adjustment mechanism(s) 1540, a housing(s) 1542, a sensor(s) 1546, and the like 1548. The imaging source(s) 1504 are configured to receive illumination projected from the illumination source(s) 1402 that reflects off of a target. The lens assembly 1528 is configured to direct illumination reflected from the target towards the internal components of the imaging system(s) 1504 (e.g., the camera(s) 1532, the image plane(s) 1534, the sensor(s) 1546) and can include telecentric lens assemblies, entrance and exit lenses, infinity entrance or exit pupils, two or more lenses, adjustable lenses, and lenses made from a variety of materials, including but not limited to polycarbonate, plastic, polymer, glass, liquid lens material, and any other suitable material. The lens assembly 1528 can include a prism assembly (e.g., 760) that can include any number of prisms (e.g., 761, 762, and / or 763).

[0125] The aperture(s) 1530 are configured to direct illumination reflected from the target toward an internal component(s) of the imaging system(s) 1504. The aperture(s) 1530 can include any size variation across an optical phase profilometry system, such as the systems and embodiments described herein. In one example, the numerical apertures of each operational coaxial imaging system-illumination source pair are comparable. In one example, the numerical apertures are of different sizes. In one example, for a channel (e.g., facing, specular, diffuse, etc.), the numerical aperture of the receiver (e.g., imaging system) is smaller than the numerical aperture of the light source (e.g., illumination source). In one example, for a channel (e.g., facing, specular, diffuse, etc.), the numerical aperture of the receiver (e.g., imaging system) is larger than the numerical aperture of the light source (e.g., illumination source). In one example, for a channel (e.g., facing, specular, diffuse, etc.), the numerical aperture of the receiver (e.g., imaging system) is comparable to the numerical aperture of the light source (e.g., illumination source).

[0126] In one example, the numerical apertures of aperture(s) 1516 and aperture(s) 1530 are equal and are thus configured to reduce, compensate, and / or eliminate measurement errors due to target tilt (e.g., deflectometer error, vignetting, etc.).

[0127] The camera(s) 1532 are configured to receive illumination projected by the illumination source(s) 1502 and reflected from the target towards the imaging system(s) 1504. The camera(s) 1532 can include sensor(s) 1546 (e.g., image sensor) configured to generate a sensor signal based on the received illumination indicative of an image of the target. The image plane(s) 1534 are part of the camera(s) 1532 and define a camera surface on which the reflected illumination is focused after passing through internal components of the imaging system(s) 1504 (e.g., lens assembly 1528, aperture(s) 1530, etc.).

[0128] The adjustment mechanism(s) 1540 are devices configured to change the position or characteristics of the lens assembly 1528 or another component of the imaging system(s) 1504. The adjustment mechanism(s) 1540 can include a mechanical device configured to change the position of a lens such that the focus of the lens is changed. The adjustment mechanism(s) 1540 can include an electro-optic lens that changes its shape between image captures such that its focus position is changed. In such a system, the curvature of the lens is adjusted by applying an electric current. The adjustment mechanism(s) 1540 can include a variable magnification lens, e.g., a liquid lens assembly. The adjustment mechanism(s) can include a device configured to change the position of the image plane(s) 1534. The adjustment mechanism(s) 1540 can include a device configured to change the position of the camera(s) 1532. The adjustment mechanism(s) 1540 can include any other suitable device, component, and / or technique such that the focus position of the imaging system may be changed.

[0129] The housing(s) 1542 define a body of the imaging system(s) 1504 and are configured to house the components of the imaging system(s) 1504. The housing(s) 1542 can include any number of materials, including but not limited to plastic, polymer, metal, or any other suitable material. The housing(s) 1542 can include any of the embodiments described herein. The sensor(s) 1546 can include any number of sensors configured to generate signals indicative of characteristics of received illumination, target dimensional information, captured images, and the like. The miscellaneous 1548 can include any other suitable components configured to enable the imaging system(s) 1504 to receive illumination or obtain dimensional information about the target.

[0130] The electronics 1550 includes communications circuitry 1552, processor(s) 1554, controller(s) 1556, and others 1560. The communications circuitry 1552 is configured to communicate with other components of the system 1500 (e.g., illumination source(s) 1502 and imaging system(s) 1504), external components (e.g., user interface(s) 1566, remote device(s) 1568, and display(s) 1570), and other components of the electronics 1550. The communications circuitry 1552 can include wired (e.g., wired loop) and / or wireless (WiFi, Bluetooth, etc.) circuitry. The processor(s) 1554 are configured to receive signals (e.g., sensor signals from the sensor(s) 1546) and other inputs regarding the target and, based on those signals and inputs, determine, calculate, and / or generate characteristic and / or dimensional information (e.g., height, tilt, x-position, y-position, z-position, etc.) regarding the target. The processor(s) 1554, in one example, is configured to generate a point cloud having a plurality of surface points captured by respective channels (e.g., optical paths) of the system for the target, the channels including at least one illumination source and at least one imaging system. The processor(s) 1554 can be adapted, via hardware, software, or a combination thereof, to receive acquired images from the imaging system(s) 1504 and perform a number of calculations, methods, and / or techniques, including those described herein. For example, the processor(s) 1554 may perform point cloud merging, iterative combined point cloud refinement, signed distance functions, weighted averaging, dynamic compensation, combining, comparing, and any other number of calculations, methods and / or techniques, or any combination of those described herein.

[0131] The controller(s) 1556 are configured to receive signals from the processor(s) 1554 and other components (e.g., the user interface(s) 1566) and generate control signals to the components of the system 1500. In one example, the controller(s) 1556 can include any number of processor(s) or microprocessor(s), including the processor(s) 1554. For example, the controller(s) 1556 can receive an output from the processor(s) 1554 indicating the need to initiate a calibration process (e.g., if a relative error determination is out of bounds and a field or factory calibration is required). The controller(s) 1556 can then generate control signals to cause the surface of an external component (e.g., user interface(s) 1566, remote device(s) 1568, and / or display(s) 1570) to issue an indication, warning, alarm, or any other indication of the status of the system 1500 (e.g., that calibration is required). In other examples, the controller(s) 1556 can generate control signals to cause the processor(s) 1554 to determine a new relative error and control signals to cause the communication circuitry 1552 to store the new relative error in the memory 1565. The controller(s) 1556 can generate any number of control signals, including control signals for any of the methods and / or techniques described herein.

[0132] In another example, the controller(s) 1556 are configured to manipulate the timing of the system 1500 (e.g., timing of projection and acquisition, timing of exposure, etc.). In one example, the timing of the system 1500 is combined (e.g., combined trigger sequences). In one example, the exposure of one imaging system can be timed to be the same as the readout time required by another imaging system to complete its frame time. In one example, the combined trigger sequence can provide the system 1500 with an overall acquisition time of 40 milliseconds. In one example, the time between trigger events is 6.6 milliseconds.

[0133] The alignment geometry 1562 is an alignment structure of the system 1500. The alignment geometry 1562 can include a vertical or horizontal position of the illumination source(s) 1502 and / or the imaging system(s) 1504. The alignment geometry 1562 can include an azimuth angle or an optical axis of the illumination source(s) 1502 and / or the imaging system(s) 1504. The alignment geometry 1562 can include any of the alignment geometries described in any of the systems, methods, techniques, or implementations described herein, such as, but not limited to, those described in FIGS. 3A-3B, 4A-4B, 12, 13A-13C, 17, 24, and 25.

[0134] The power source(s) 1564 are configured to provide power to the components of the system 1500. The power source(s) 1564 may include a batter, a wired connection to an electrical circuit, or any other suitable technique by which the components of the system 1500 are powered. In addition, each of the individual subsystems of the system 1500 (e.g., the illumination source(s) 1502, the imaging system(s) 1504, and the electronics 1550) may include their own power source(s) (e.g., a battery or a separate connection to an electronic circuit) such that they are powered independently of one another. The power source(s) 1564 may also include any combination of these.

[0135] Memory 1565 is configured to store data (e.g., dimensional information regarding targets, calculations, decisions, instructions, etc.), calibration information, system status information, etc. Memory 1565 may include RAM, ROM, cache, dynamic RAM, static RAM, flash memory, virtual memory, video memory, BIOS, or any other suitable form of memory. Memory 1565 is preferably electrically coupled to system 1500.

[0136] The beam splitter(s) 1567 are configured to "split" illumination from the illumination source(s) 1502 and reflections reflected from the target such that the illumination source(s) 1502 and the imaging system(s) 1504 can, in one example, comprise an operational coaxial pair as described herein. The housing(s) 1569 can be configured to house both the illumination source(s) 1502 and the imaging system(s) 1504 within a single housing, particularly when an operational coaxial illumination source / imaging system pair is utilized as in certain embodiments described herein. The housing(s) 1569 define the body of the illumination source(s) 1502 and the imaging system(s) 1504 and are configured to house the internal components of each, as well as, in one example, the beam splitter(s) 1567. The housing(s) 1569 can comprise any number of materials, including but not limited to a plastic, a polymer, a metal, or any other suitable material. The system 1500 can include any other number of suitable components and / or devices, such as those shown at 1571.

[0137] The user interface(s) 1566 are configured to receive user or operator input. The user interface(s) may include a touch screen display, switches, levers, electronic control boards, buttons, or any other suitable technology for receiving user or operator input. The remote device(s) 1568 may include devices that are electronically coupled to, but remote from, the system 1500, such as a computer in a control room over a wired loop. The remote device(s) 1568 may also include devices wirelessly coupled to the system 1500, such as handheld devices, laptops, tablets, off-site computers, etc. The remote device(s) 1568 may be configured to display, receive, and transmit information about the system 1500 (e.g., dimensional information about targets, performance analysis, warnings, alerts, notifications, system status, etc.). The display(s) 1570 are configured to display information about the system 1500. The display(s) 1500 may include a visual display, such as a screen display, or a light (e.g., a warning light) configured to indicate a status of the system 1500. The display(s) 1570 may include an audible indication configured to generate a noise (e.g., an audible alarm) to communicate information regarding the system 1500.

[0138] Any and all of the components and / or devices of system 1500 may include any or any combination of the components, devices, techniques, methods, and / or embodiments described herein.

[0139] Certain implementations described herein are also configured, in one example, to reduce, eliminate, compensate, and / or correct errors (e.g., measurement errors), such as those described herein, including, but not limited to, errors due to target surface reflectance gradients, glint, target tilt, and the like.

[0140] Although a particular order of steps is shown in the methods described herein for purposes of illustration, it should be understood that some or all of these steps can be performed in any number of orders, including, but not limited to, simultaneously, in parallel, sequentially, non-parallel, non-sequentially, and any combination thereof. No particular order of steps for the methods described herein should be implied and / or interpreted from the illustrations.

[0141] Although the present invention has been described with reference to preferred embodiments, those skilled in the art will recognize that changes may be made in form and detail without departing from the spirit and scope of the present invention. It should be noted that the different examples described herein may be combined in different ways; that is, parts of one or more examples may be combined with parts of one or more other examples. All of this is contemplated herein.

[0142] Although the subject matter has been described in terms of specific structural features and / or method acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. 1. A method for merging point clouds, comprising the steps of: generating reconstructed surface points in a point cloud of a first counter-disposed channel; generating reconstructed surface points in the point cloud of a second opposing channel; for a selected reconstructed surface point in the point cloud of the first counter-disposed channel, identifying a reconstructed surface point in the point cloud of the second counter-disposed channel that is near an imaging system ray of the selected reconstructed surface point in the point cloud of the first counter-disposed channel; calculating a projection of each of the identified near points in the point cloud of the second counter-disposed channel onto the imaging system ray of the selected reconstructed surface point in the point cloud of the first counter-disposed channel; calculating an average projection of each of the identified near points in the point cloud of the second counter-disposed channel onto the imaging system rays of the selected reconstructed surface points in the point cloud of the second counter-disposed channel; moving the selected reconstructed surface point in the point cloud of the first counter-disposed channel a portion of a distance along an imaging system ray of the reconstructed surface point toward a calculated average projection of each of the identified near points in the point cloud of the second counter-disposed channel; A method comprising:

2. Moving the reconstructed surface point in the first counter-disposed channel reduces the following measurement errors: Specular reflection glit measurement error, Reflectance gradient measurement error, or Target surface tilt measurement error The method of claim 1 , configured to reduce at least one of:

3. for a selected reconstructed surface point in the point cloud of the second counter-disposed channel, identifying a reconstructed surface point in the point cloud of the first counter-disposed channel that is near an imaging system ray of the selected reconstructed surface point in the point cloud of the second counter-disposed channel; calculating a projection of each of the identified near points in the point cloud of the first counter-disposed channel onto the imaging system ray of the selected reconstructed surface point in the point cloud of the second counter-disposed channel; calculating an average projection of each of the identified near points in the point cloud of the first counter-disposed channel onto the imaging system rays of the selected reconstructed surface points in the point cloud of the second counter-disposed channel; moving the selected reconstructed surface point in the point cloud of the second counter-disposed channel a portion of a distance along an imaging system ray of the reconstructed surface point toward a calculated average projection of each of the identified near points in the point cloud of the first counter-disposed channel; The method of claim 1 further comprising:

4. 4. The method of claim 3, wherein moving the selected reconstructed surface points within the point cloud of the first opposing channel and within the point cloud of the second opposing channel is configured to reduce measurement errors caused by specular glint.

5. 4. The method of claim 3, wherein moving the selected reconstructed surface points within the point cloud of the first opposing channel and within the point cloud of the second opposing channel is configured to reduce measurement errors caused by reflectivity gradients of a target surface.

6. 4. The method of claim 3, wherein moving the selected reconstructed surface points within the point cloud of the first opposing channel and within the point cloud of the second opposing channel is configured to reduce measurement errors caused by tilting of a target surface.

7. the first opposing channel includes a first projector of a first working coaxial imaging system and illumination source pair and a first camera of a second working coaxial imaging system and illumination source pair; The method of claim 3 , wherein the second opposing channel includes a second projector of the second operating coaxial imaging system and illumination source pair and a second camera of the first operating coaxial imaging system and illumination source pair.

8. The method of claim 7 , wherein at least one of the first moving coaxial imaging system and illumination source pair or the second moving coaxial imaging system and illumination source pair is aligned at an oblique angle relative to a target surface and a target surface normal.

9. The method of claim 8 , wherein at least one of the first moving coaxial imaging system and illumination source pair or the second moving coaxial imaging system and illumination source pair is aligned substantially perpendicular to the target surface.

10. 1. A method for merging point clouds, comprising the steps of: generating a first point cloud having a first plurality of surface points corresponding to the target surface in a first opposingly disposed channel; generating, in a second opposingly disposed channel, a second point cloud having a second plurality of surface points corresponding to the target surface; determining a first non-empty portion and a second non-empty portion corresponding to the first cloud of points and the second cloud of points, respectively; identifying a transformation of each of the first cloud of points and the second cloud of points; determining a relative error for each of the first cloud of points and the second cloud of points based on the identified transformation; subtracting a relative error from each of the first cloud of points and the second cloud of points; aligning the first cloud of points and the second cloud of points in the first non-empty portion and the second non-empty portion; generating a geometry of the target surface based on the alignment; and A method comprising:

11. Determining a relative error of each of the first cloud of points and the second cloud of points comprises: determining whether the time during which the most recent relative error was calculated exceeds a threshold, and if the time exceeds the threshold, calculating a relative error average based on the most recent relative error and the newly determined relative error; The method of claim 10, comprising:

Images