Optical Imaging and Scanning of Holes

Abstract

To provide a method and an apparatus for optical imaging and scanning for a hole formed with metal processing, drilling or other processing on a substrate consisting of a composite material or a metallic material.SOLUTION: A method utilizes optical equipment 50A for imaging and scanning for a hole in combination with an image processing device which is configured (for example, programmed) to generate one conical fully-flat image without an optical distortion by post-processing image data. The optical equipment 50A comprises an optical microscope including a confocal illumination and a conical mirror 8 arranged in an axial direction in order to generate an entire 360-degree sub-image having a conical distortion. In the post-processing step, the mathematical transformation in a form of a code that can be executed by a computer is used in order to transform an unprocessed conical sub-image into the flat sub-image. The flat sub-images may be coupled in order to form a fully-flat image of a hole.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present disclosure relates generally to methods and apparatus for inspecting holes machined in a workpiece or structure, and more particularly to methods and apparatus for inspecting holes designed to receive fasteners. [Background technology]

[0002] Lightweight composite materials (such as fiber-reinforced plastic materials) are being used more and more extensively in the aerospace industry, in commercial and military aircraft, as well as other aerospace vehicles, and even in other industries. Structures using these composite materials may be formed using multiple plies or layers of material that may be layered together to form high strength structures. At least one method for fastening multiple layers of material together is to clamp the layers, drill holes, and then insert some type of fastener into the holes, thereby securing the layers together.

[0003] In manufacturing, measurements and inspections are frequently performed to ensure that manufactured parts meet design specifications. This includes inspection of holes, such as drilled holes, to ensure that the holes are of the desired shape and configuration, e.g., diameter and placement are within engineering tolerances. Ensuring that manufactured parts meet design specifications is especially important in industries such as aerospace manufacturing, where strict manufacturing standards are maintained.

[0004] It is common practice to have a quality control or quality assurance inspector inspect each hole to ensure conformance to design specifications, or to use statistical methods to analyze a sample number of holes. Inspection can be done by manually inserting a hole diameter probe, such as a capacitance probe, pneumatic hole probe, laser hole probe, or ball-type probe, into the hole to check for conformance or deviation from design specifications. Techniques for optical imaging of composite or metal holes can only collect shallow depth images at offset angles to the centerline of the hole (hereafter the "Z-axis"), because the field of view of a standard microscope along the optical axis (which is the same as the Z-axis of the hole) is extremely narrow. Such optical imaging also cannot produce a full 360 degree flat image of the hole in a single focal plane.

[0005] It would be advantageous to provide an apparatus capable of scanning along the Z axis within the pores of a structure to assess the condition of the pores. Summary of the Invention

[0006] The subject matter disclosed in detail below is directed to methods and apparatus for optical imaging and scanning of holes machined, drilled or otherwise formed in a substrate made of a composite or metallic material. The methods described herein utilize an optical instrument for imaging and scanning of the hole in combination with an image processor configured (e.g., programmed) to post-process image data to generate a single fully flattened image free of conical optical distortion. According to some embodiments, the optical instrument includes an optical microscope with axially arranged confocal illumination and a conical mirror to generate a full 360 degree sub-image with conical distortion. In the post-processing step, a mathematical transformation in the form of computer executable code is used to convert the raw conical sub-images into flat sub-images. The flattening of the sub-images is performed using a simple mathematical transformation of the image bits. The flattened sub-images may be stitched together to form a fully flattened image of the hole.

[0007] By design, a microscope uses lenses and / or mirrors to focus light onto an image sensor, such as a focal plane array of optoelectronic elements (hereinafter "array of photodetectors") that converts the impinging light into electrical signals representing pixel data of the image. As used herein, the term "photodetector" refers to a device that can emit electrons in response to a photon impinging on the surface of the device. The apparatus includes a 90 degree "cone mirror" mounted on a movable stage that is movable in a direction parallel to the Z axis of the bore, while the axis of the cone is coaxial with the Z axis of the bore. However, it should be understood that a truncated cone-shaped mirror (also referred to as a "frustoconical mirror") may also be used. A mirror having a conical or frustoconical reflective surface is referred to herein as a "cone mirror."

[0008] The subject matter disclosed in detail below is further directed to an automated, high speed method for inspecting holes in a fastened structure, as well as a computer controlled apparatus for carrying out said inspection method. In various embodiments, the apparatus comprises a multi-motion inspection head mounted on a scanning bridge, the end of a robotic arm, or a robotic trackless vehicle. The multi-motion inspection head comprises the aforementioned optical instrument, and a motorized, multi-stage probe placement head operable to move the optical instrument along the X, Y, and Z axes to achieve multiple sequential motions. The optical instrument is mounted on a mandrel that is rotatably coupled to an X-axis (or Y-axis) stage for rotation about the Z axis. A smart servo or stepper motor with feedback control is used to move the optical instrument into position and sequentially scan the interior of each hole. In one embodiment, the apparatus comprises a multi-directional motorized stage that is continuously positioned and controlled for the special motions required for inspection of a row of holes.

[0009] In some embodiments, the microscope and the optical instrument light source are housed in a housing carried by a multi-motion inspection head. The optical instrument further includes an optical probe (e.g., a conical mirror and associated support structure) extending downward from the housing. The optical probe is sized and shaped to fit inside the hole to be inspected. The multi-motion inspection head is configured to move the microscope until the optical axis of the optical probe is aligned with the centerline (Z-axis) of the hole, and then insert the probe into the hole until the optical probe reaches a starting depth inside the hole. An image sensor of the microscope collects sub-images of the inside of the hole at various depths. In one proposed implementation, the multi-motion inspection head intermittently moves the optical probe along the Z-axis of the hole, and the image sensor collects raw conical sub-images at regular time intervals while the optical probe is stationary. After one hole is fully inspected, the optical probe is removed from that hole and inserted into the next hole to be inspected. In this way, multiple holes in a row of holes can be inspected consecutively.

[0010] In some embodiments, light from the light source is directed axially (or approximately axially) during image collection. The axially propagating light is directed radially outward toward the facing portion of the bore. This radial redirection of the illumination light is performed using a conical mirror. The conical mirror has a conical (or frustoconical) surface, the apex of which is located along the optical axis of the microscope. The conical mirror receives the axially (or approximately axially) propagating light from the light source and reflects the light radially outward to illuminate the facing 360 degree portion of the bore where the optical probe is inserted.

[0011] The method proposed herein provides a simple yet complete optical inspection. Hole surface roughness and hole size variations can be fully assessed in a single scan. Instead of providing only localized coverage within the radius of curvature of the probe, the optical inspection technique disclosed herein produces a full 360 degree view.

[0012] Various embodiments of methods and apparatus for optical imaging and scanning of holes are described in some detail hereinafter, one or more of which may be characterized by one or more of the following aspects.

[0013] One aspect of the subject matter disclosed in detail below is an imaging device comprising a housing, a light source disposed within the housing, an image sensor disposed within the housing, a conical mirror disposed outside the housing and having a conical axis, a conical mirror support structure supporting the conical mirror in a fixed position relative to the housing, and an optical subassembly supported and configured by the housing such that light from the light source impinges on the conical mirror and is reflected radially outward by the conical mirror, and light that propagates radially inward and impinges on the conical mirror is directed onto the image sensor.

[0014] In some embodiments of the optical instrument described in the immediately preceding paragraph, the conical mirror support structure comprises a central post and the conical mirror is truncated and attached to one end of the central post, hi other embodiments, the conical mirror support structure comprises a cylindrical glass tube having a cylindrical axis and the conical mirror is disposed within the cylindrical glass tube such that the conical axis is coaxial with the cylindrical axis.

[0015] Another aspect of the subject matter disclosed in detail below is a method for imaging a hole in a substrate, the method including: (a) placing a conical mirror within the hole with the conical axis coaxial with a centerline of the hole and with the apex of the conical mirror or a truncated portion of the conical mirror at a first depth less than a second depth at which the bottom of the conical mirror is located; (b) illuminating the conical mirror with light focused on a focal plane within the hole; (c) using the conical mirror to reflect the light mentioned in step (b) radially outward toward the hole; (d) using the conical mirror to reflect the returning light axially upward toward an opening; (e) directing the light reflected axially upward by the conical mirror of step (d) onto an image sensor; and (f) converting the light impinging on the image sensor into electrical signals representative of pixel data of a first distorted sub-image of a first portion of the hole having a conical optical distortion. The method further includes processing the pixel data of the first distorted sub-image to generate pixel data representing the first flattened sub-image free of the conical optical distortion.

[0016] The method recited in the immediately preceding paragraph further includes (g) moving the conical mirror along the centerline of the hole to a position where an apex of the conical mirror or a cut-off portion of the conical mirror is at a third depth that is closer to the second depth than the first depth; (h) illuminating the conical mirror with light focused on a focal plane inside the hole; (i) using the conical mirror to reflect the light described in step (h) radially outward toward the hole; (j) using the conical mirror to reflect returning light described in step (i) axially upward toward the opening; (k) directing the light reflected axially upward by the conical mirror of step (j) onto an image sensor; and (l) converting the light impinging on the image sensor into electrical signals representative of pixel data of a second distorted sub-image of a second portion of the hole having a conical optical distortion. In this case, the method further includes processing the pixel data of the first distorted sub-image to generate pixel data representing a first flattened sub-image without the conical optical distortion, processing the pixel data of the second distorted sub-image to generate pixel data representing a second flattened sub-image without the conical optical distortion, stitching the first and second flattened sub-images together, and presenting a flattened image including at least the first and second flattened sub-images on a display device.

[0017] Further aspects of the subject matter disclosed in detail below include a multi-stage probe placement head comprising a block assembly, a first stage translatable relative to the block assembly along a first axis, a second stage translatable relative to the block assembly along a second axis orthogonal to the first axis, and a third stage translatable relative to the block assembly along a third axis orthogonal to the first and second axes, the third stage translatably coupled to the second stage, and the second stage translatably coupled to the first stage; and an optical instrument supported by and depending from the third stage. an optical instrument comprising: a housing coupled to a third stage and translatable; a light source disposed within the housing; an image sensor disposed within the housing; a conical mirror disposed outside the housing and having a conical axis parallel to the first axis; a conical mirror support structure supporting the conical mirror at a fixed position relative to the housing; and an optical subassembly supported by and configured such that light from the light source impinges on the conical mirror and is reflected radially outward by the conical mirror and light that propagates radially inward and impinges on the conical mirror is directed onto the image sensor.

[0018] Further aspects of the subject matter disclosed in detail below include an automated device configured to move an end effector by operation of a motor, an optical device attached to the end effector, and an image processing device that receives a conical optically distorted image collected by the optical device and processes the pixel data of the conical optically distorted image to generate pixel data representing a flattened image without conical optical distortion. The system for imaging holes in a substrate comprises an automated device configured to move an end effector by operation of a motor, an optical device attached to the end effector, and an image processing device that receives a conical optically distorted image collected by the optical device and processes the pixel data of the conical optically distorted image to generate pixel data representing a flattened image without conical optical distortion. The optical device includes a housing coupled to the end effector, a light source disposed inside the housing, an image sensor disposed inside the housing, a conical mirror disposed outside the housing and having a conical axis parallel to a first axis, a conical mirror support structure that supports the conical mirror in a fixed position relative to the housing, and an optical subassembly supported by the housing and configured such that light from the light source impinges on the conical mirror and is reflected radially outwardly toward a ring-shaped portion of a hole by the conical mirror, propagates radially inwardly, and the light impinging on the conical mirror is directed onto the image sensor.

[0019] Other aspects of methods and apparatus for optical imaging and scanning of holes are disclosed below.

[0020] The features, functions, and advantages described in the foregoing may be achieved individually in various embodiments or may be combined in yet other embodiments. For purposes of exemplifying the foregoing and other aspects, various embodiments will now be described with reference to the drawings. None of the drawings described briefly in this section are drawn to scale.

Brief Description of the Drawings

[0021] [Figure 1] FIG. is a diagram showing the operating principle of an optical device suitable for imaging a hole using a conical mirror according to an embodiment. [Diagram 2] FIG. is a diagram showing a hole being illuminated when light propagating axially downward is reflected radially outward by a conical mirror toward a ring-shaped portion of the hole. [Diagram 3]FIG. 3 is an enlarged view of a portion of FIG. 2. [Figure 4] FIG. 1 illustrates structural and functional features of an optical instrument suitable for imaging a hole using a conical mirror supported by a central rod, according to one embodiment. [Diagram 5] FIG. 13 illustrates structural and functional features of an optical instrument suitable for imaging a hole using a conical mirror supported by a glass tube, according to another exemplary embodiment. [Figure 6A] FIG. 2 shows the geometric relationship between the cylindrical reference frame of the hole and a conical mirror inserted in the hole, and further shows points on the hole surface and corresponding points on the image detected by the image sensor. [Figure 6B] FIG. 1 shows the location of the image point in a polar coordinate system centered on the cylindrical reference frame of the hole. [Figure 6C] FIG. 13 shows a light ray emanating from a point on the hole surface being reflected by a conical mirror to produce an image point on an image sensor. [Figure 7] FIG. 13 is a three-dimensional representation of an optical instrument suitable for imaging a bore using a conical mirror supported by a central rod, the conical mirror being shown inside the bore. [Figure 8] FIG. 13 shows a three-dimensional representation of an optical instrument suitable for imaging a hole using a conical mirror supported by a glass tube, according to a further exemplary embodiment. [Figure 9] FIG. 9 shows a side view of a trackless vehicle carrying an optical instrument of the type shown in FIG. 8. [Figure 10] FIG. 1 is a block diagram identifying certain components of a computer-controlled apparatus for optical imaging and scanning of holes, according to one embodiment. [Figure 11] FIG. 9 shows an elevational view of an optical instrument of the type shown in FIG. 8 mounted on a robot. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] Reference will now be made to the drawings, in which similar elements in different drawings are numbered the same.

[0023] Exemplary embodiments of methods and apparatus for optical imaging and scanning of bores are described in detail below. However, not all features of an actual implementation are described herein. Those skilled in the art will appreciate that the development of any such actual implementation will require numerous implementation-specific judgments to be made to achieve the developer's particular objectives, such as compliance with system-related and business-related constraints, which will vary from implementation to implementation. Moreover, it will be appreciated that such development efforts may be complex and time-consuming, but will nevertheless be routine for those skilled in the art having the benefit of this disclosure.

[0024] A method and apparatus for optical imaging and scanning of holes machined, drilled or otherwise formed in a substrate of composite or metallic material will now be described in some detail for illustrative purposes. The method includes inserting an optical probe having a conical mirror into the hole, taking a 360 degree sub-image of a portion of the hole, and then sending the sub-image data to an image processor configured (e.g., programmed) to generate a flattened sub-image without the conical optical distortion. Multiple flattened sub-images may be stitched together to form a complete flattened image of the hole.

[0025] FIG. 1 illustrates the operating principle of an optical instrument 50 suitable for imaging a hole 6 in a substrate 2 made of a composite material (e.g. carbon fiber reinforced plastic). The optical instrument 50 comprises a microscope 10 having a housing part 16a and a light source 18 inside the housing part 16b. Other parts of the optical instrument housing are not shown in FIG. 1. An image sensor (not shown in FIG. 1) is arranged inside the housing part 16a of the microscope 10. The light source 18 is arranged inside the housing part 16b. In some embodiments, the light source 18 is a monochromatic point light source.

[0026] The optical instrument 50 further includes a conical mirror 8 sized to fit within the bore 6. The conical mirror 8 is disposed external to the housing of the optical instrument 50 and has a conical axis. The conical mirror is supported in a fixed position relative to the housing of the optical instrument 50 by a conical mirror support structure (not shown in FIG. 1). The conical mirror 8 has a geometric shape defined by rotating about the conical axis a line disposed at an angle equal to 45 degrees relative to the conical axis. In the implementation partially shown in FIG. 1, the conical mirror 8 has an apex 8a and a base 8c. However, it should be understood that in alternative embodiments, the conical mirror 8 may be truncated.

[0027] The optical instrument 50, partially shown in FIG. 1, further includes an optical subassembly supported by the housing. The optical assembly includes an objective lens 12 supported by the housing portion 16a, a lens 20 supported by the housing portion 16b, and a dichroic mirror 14 and a mirror 22 supported by housing portions not shown. Both the dichroic mirror 14 and the mirror 22 are arranged at a 45 degree angle with respect to the optical axis of the objective lens 12. The optical subassembly is configured such that light from the light source 18 (indicated by the dashed arrow L1) impinges on the conical mirror 8. The conical mirror 8 reflects the impinging light radially outward toward the facing portion of the bore 6. Additionally, the radially inward propagating light returning from the bore 6 and impinging on the conical mirror 8 is reflected by the conical mirror 8 toward the objective lens 12. To avoid cluttering the drawing, FIG. 1 shows only one light ray L2 that is returned to the microscope 10.

[0028] In some embodiments, light L1 from the light source 18 is directed axially (or approximately axially) by mirror 22 and dichroic mirror 14 towards the conical mirror 8 during image collection. More specifically, light impinging on mirror 22 is reflected onto dichroic mirror 14. Dichroic mirror 14 then reflects light from the light source at an angle parallel to the optical axis of microscope 10. Light propagating in the axial direction impinging on conical mirror 8 is reflected radially outward towards the facing portion of hole 6. An apex 8a of conical mirror 8 is positioned along the optical axis of microscope 10. Conical mirror 8 receives axially (or approximately axially) propagating light and reflects it radially outward to illuminate a 360 degree ring-shaped portion facing hole 6. A portion of the light is scattered back towards conical mirror 8 by hole 6. Conical mirror 8 then reflects the light propagating axially upward radially inward towards dichroic mirror 14 , thereby transmitting the light to objective lens 12 .

[0029] The dichroic mirror 14 allows the transmission of light having wavelengths within a certain range and reflects light having wavelengths outside that range. The dichroic mirror 14 can be designed to reflect light from the light source 18 and transmit light received from the conical mirror 8. The dichroic mirror 14 is positioned at an angle of about 45 degrees (within a range of ±5 degrees) with respect to the optical axis of the microscope 10. The 45 degree orientation is preferred to maximize the effect of the dichroic mirror and reflect light incident at right angles. One example of a dichroic mirror is a PYREX™ substrate with a borosilicate crown glass coating. The particular wavelengths selected to be reflected or transmitted can be controlled by careful selection of the mirror and coating. Typically, after the ring-shaped hole portion 6a absorbs the light reflected radially outward by the conical mirror 8, re-emission (scattering) of light occurs at longer wavelengths. Therefore, illumination must be selected at shorter wavelengths to better distinguish the scattered light returning to the image sensor 30. For example, with a dichroic mirror set to pass red light and reflect blue light, the blue light from the light source 18 is reflected by the dichroic mirror 14 towards the cone mirror 8. The blue light illuminating the ring-shaped hole portion 6a causes the CFRP surface to generate scattered red light (having a longer wavelength than the blue light). The red color is sent by the dichroic mirror towards the image sensor 30 inside the microscope 10.

[0030] The image sensor 30 converts impinging photons into electrons and outputs image data representing a full 360 degree sub-image of the ring-shaped portion 6a of the hole 6 with a conical optical distortion. In a post-processing step, the raw conical sub-image is converted into a flattened sub-image without the conical optical distortion. The flattened image is then further processed to assess the shape and configuration of the hole 6. Thus, a system implementing the operating principle illustrated in FIG. 1 can be used to scan holes in depth and generate a flattened image of the hole that includes specific hole characteristic information. For example, the flattened image can be processed to determine whether the shape and configuration of the hole (e.g., diameter, orientation and surface roughness) are within engineering tolerances.

[0031] Figure 2 illustrates the bore 6 being illuminated when light propagating axially downward strikes and is reflected radially outward by the reflective surface of the conical mirror 8 towards the ring-shaped portion 6a of the bore 6. In the embodiment illustrated in Figure 2, the conical mirror support structure that supports the conical mirror 8 in a fixed position relative to the housing portion 16a is a central rod 24 having a linear axis and a circular cross section. In one proposed implementation, the axis of the central rod 24 is coaxial with the optical axis of the microscope 10. The XY rotational alignment is indicated by point 28 in Figure 2.

[0032] 2, the truncated portion 8b of the conical mirror 8 is attached to one end of the central rod 24. The microscope 10 further includes an image sensor 30 and an image erecting lens 32 (hereinafter "second lens 32") supported by the housing portion 16a at a location between the objective lens 12 and the image sensor 30. The image sensor 30 may comprise a steering focal plane array, such as a charge-coupled device (CCD) sensitive to visible or infrared wavelengths.

[0033] As already mentioned, the ring-shaped portion 6a of the hole 6 is illuminated by light that propagates axially into the hole, impinges on the reflective surface of the conical mirror 8, and is then reflected radially outward by said reflective surface. A portion of the light reflected on the ring-shaped portion 6a of the hole 6 is reflected or scattered back towards the reflective surface of the conical mirror 8, which then reflects the returning light towards the objective lens 12. The objective lens 12 of the microscope 10 forms an image in a first image plane perpendicular to the optical axis of the microscope 10. A second lens 32 receives the light from the objective lens 12 and focuses the light to form an image in a second image plane that coincides with the photoconductive surface of the image sensor 30. As a result, the image sensor 30 (e.g. an array of photodetectors) collects a raw conical sub-image of the ring-shaped portion 6a of the hole 6.

[0034] FIG. 3 shows an enlarged view of a portion of FIG. 2. The inclined reflective surface of the conical mirror 8 is arranged at a cone angle ω with respect to the bottom 8c. The height of the ring-shaped portion 6a of the hole 6 is illuminated by light reflected by the reflective surface of the conical mirror 8. If the cone angle ω is equal to 45 degrees, the illuminated ring-shaped portion 6a of the hole 6 is equal to the distance separating the two light rays L1 shown in FIG. 3, which in this embodiment represent the radially innermost and radially outermost light rays propagating parallel to the cone axis and impinging on the reflective surface of the conical mirror 8. In an alternative embodiment not shown in FIG. 3, the height of the illuminated ring-shaped portion 6a of the hole 6 can be equal to the height of the conical mirror (e.g. the distance between the cut-out portion 8b and the bottom 8c). The illumination light has a focal plane F (indicated by the horizontal dashed line in FIG. 3) located at a certain distance below the bottom 8c of the conical mirror 8.

[0035] In one proposed implementation, the conical mirror 8 is intermittently translated a predetermined distance along the centerline (Z-axis) of the hole 6, and each raw conical sub-image is captured at each vertical position. At time intervals following each motion, the conical mirror 8 is stationary while the image sensor 30 (see FIG. 2) collects a raw conical sub-image. After a hole has been completely inspected along the entire hole depth, the optical probe is removed from that hole and inserted into the next hole to be inspected. In this way, multiple holes in a row of holes can be inspected consecutively.

[0036] 4 illustrates structural and functional features of an optical instrument 50A suitable for imaging a hole 6 with coaxial imaging and confocal illumination using a conical mirror 8 supported by a central rod 24, according to an exemplary embodiment. The axes of the central rod 24 and the conical mirror 8 are coaxial with the optical axis of the microscope 10. The optical instrument 50A further includes a light source 18, a lens 20, and a small 45 degree mirror 42. The lens 20 is configured to focus light from the light source 18 onto the small 45 degree mirror 42. An aperture plate 40 is disposed between the lens 20 and the small 45 degree mirror 42. The small 45 degree mirror 42 reflects the light impinging thereon toward the objective lens 12 of the microscope 10. The light from the light source 18 is directed by the objective lens 12 toward the aperture 4 of the hole 6 and the reflective surface of the conical mirror 8 to illuminate the ring-shaped portion of the hole 6 being imaged.

[0037] In the exemplary embodiment shown in FIG. 4, the microscope 10 further includes a second lens 32 and a charge-coupled device image sensor 38 (hereinafter, "CCD image sensor 38") (although not shown in FIG. 4, the CCD image sensor 38 is supported by the housing portion 16a of the microscope 10). In a CCD image sensor, the photodetectors are p-type metal oxide semiconductor capacitors. When image collection begins, these capacitors are biased above a threshold for inversion, allowing the conversion of incident photons to charge at the semiconductor-oxide interface. A CCD is then used to read out these charges. The small 45-degree mirror 42 is sized so as not to block the light propagating from the objective lens 12 to the second lens 32. For example, the small 45-degree mirror 42 may be sized to fit within the shadow created by the central rod 24. Thus, light reflected upwards by the conical mirror 8 passes through the objective lens 12, around the small 45-degree mirror 42, through the second lens 32, and then impinges on the CCD image sensor 38. The objective lens 12 has a first image plane located at point f. The second lens 32 has a second image plane located at the CCD image sensor 38.

[0038] 5 illustrates structural and functional features of an optical instrument 50B suitable for imaging a hole 6 by coaxial illumination and confocal imaging using a conical mirror 8 supported by a glass support tube 36, according to another exemplary embodiment. The glass support tube 36 has anti-reflective coatings on its inner and outer surfaces, respectively. The optical instrument 50B includes a light source 18, a lens 20, an aperture plate 40, a dichroic mirror 14, and an objective lens 12. Light from the light source 18 propagates through the lens 20, the aperture plate 40, the dichroic mirror 14, and the objective lens 12 and enters the hole 6. The light from the light source 18 is directed to a reflective surface of the conical mirror 8 to illuminate a ring-shaped portion of the hole 6 to be imaged.

[0039] 5, the microscope 10 further includes a second lens 32 and a CCD image sensor 38. Light reflected upward by the conical mirror 8 passes through the objective lens 12 and is then reflected by the dichroic mirror 14 towards the second lens 32. The second lens 32 then focuses the impinging light onto the photoconductive surface of the CCD image sensor 38.

[0040] In one embodiment, the system for imaging the hole in the substrate further comprises an image processor configured (e.g. programmed) to receive the conical optically distorted image collected by the optical instrument and to process pixel data of the conical optically distorted image to generate pixel data representative of a flattened image without the conical optical distortion. In one proposed implementation, the image processor is programmed to execute a flattening algorithm comprising the steps of: converting pixel data having image sensor coordinates to pixel data having cylindrical coordinates corresponding to the hole surface; and unfolding the cylindrical shell of the hole surface into a flattened two-dimensional plane having X- and Y-axes (different from the X- and Y-axes of the XYZ coordinate system shown in FIG. 2 ; the new Y-axis of the flattened two-dimensional coordinate system is parallel to the Z-axis of the XYZ coordinate system). This transformation converts the flattened two-dimensional Cartesian coordinate system (i.e. the pixel positions of the image sensor) of the recorded reflected image of the hole surface of the conical mirror 8 into a newly unfolded flattened two-dimensional Cartesian coordinate system that represents the hole surface when flattened. The result is that a cylindrical, optically distorted image is transformed into a conical, optically distorted, flattened image.

[0041] 6A is a diagram illustrating the geometric relationship between the cylindrical reference frame 44 of the hole and a conical mirror 8 having a cone axis that is coaxial with the Z axis of the hole. The point on the imaged hole surface is represented as point P in FIG. 0 The corresponding point on the image detected by the image sensor is represented as point P 1 It is represented by (im x ,im y ) in the image sensor's reference frame. Figure 6B shows the location of the image point in a polar coordinate system centered in the bore's cylindrical reference frame 44.

[0042] FIG. 6C shows a ray of light first propagating radially inward from point P0 along a path 46 perpendicular to the Z axis, being reflected 90 degrees by a conical mirror, and then propagating upward along a path 48 until the ray strikes the image sensor, resulting in an image point P 1As best seen in FIG. 6C, the maximum radius of the conical mirror 8 is R 0 and the distance from the Z axis to the point where the light ray strikes the conical mirror 8 is equal to r. The X axis is the origin of the XY coordinate system of the image sensor and the image point P 1 The angle between the vector connecting and is indicated by θ in FIG. 6B.

[0043] The first step of the planarization algorithm is to find the point P 1 Image coordinates of (im x ,im y ) and height h into polar coordinates r and θ in FIG. 6B. TIFF0007680190000001.tif16170

[0044] The second step of the flattening algorithm is to flatten the cylindrical reference frame 44 and convert the polar coordinates r and θ into the image coordinates (im x ',im y '), where im x ' is the distance from reference point O to point P 0 Arc length θR 0 (shown in FIG. 6B), im y ' is the height of point P0 h=R 0 -r (shown in FIG. 6A). TIFF0007680190000002.tif35170

[0045] The above-described cone-to-plane transformation is performed on all image points collected from the hole to generate a flattened image of the hole. The image is displayed to a technician to enable visual inspection of the hole. The flattening algorithm disclosed herein may be employed in conjunction with optical instruments having a variety of configurations, but which share the common feature that the optical probe inserted into each hole includes a conical mirror.

[0046] FIG. 7 shows a three-dimensional image of an optical instrument 50C with a conical mirror 8 supported by a central rod 24 inside a hole 6 in a substrate 2. The double-headed arrow indicates that the optical instrument 50C can move vertically along a Z-axis that is coaxial with the axis of the hole 6. The housing of the optical instrument 50C includes (a) a housing portion 16a that houses the components of the microscope 10, and (b) a housing portion 16b that houses a light source 18, and a housing portion 16c that houses an optical subassembly including a dichroic mirror 14 and a mirror 22 (shown by a dashed oval) that are both positioned at a 45 degree angle with respect to the axis of the objective lens 12. The objective lens 12 is supported by the housing portion 16a. The lens 20 is supported by the housing portion 16b. Light from the light source 18 is transmitted by the lens 20, reflected by the mirror 22, reflected downwards by the dichroic mirror, and then strikes the conical mirror 8. The returning light from hole 6 is reflected upwards by conical mirror 8, transmitted by dichroic mirror 14 and objective lens 12, and then impinges on an image sensor (not shown in FIG. 7) inside microscope 10.

[0047] 8 is a diagram representing a three-dimensional image of an optical instrument 50D having a conical mirror 8 supported by a central glass tube 36 instead of a central rod 24. The other components shown in FIG. 8 have the same structure and function as the similarly numbered components depicted in FIG. 7 and described in the immediately preceding paragraph.

[0048] 9 is a diagram illustrating a side view of a trackless vehicle 130 having a front-end mounted multi-stage probe placement head 140 according to one embodiment. The multi-stage probe placement head 140 supports an optical instrument 50D that can be used to image a hole 6 in a substrate 2 (such as a fuselage skin). Although only one hole 6 is shown, typically multiple holes 6 are aligned. The optical instrument 50D is shown in a position where the optical probe 26 is aligned vertically with the hole 6, e.g., the optical axis of the optical probe 26 is coaxial with the centerline of the hole 6. During an inspection procedure, the optical instrument 50D is positioned at different depths within the hole and takes respective images at each depth.

[0049] The trackless vehicle 130 may take the form of a remotely operated robot that can move holonomically along non-horizontal surfaces using wheels and a suction device (e.g., a fan driven by a motor mounted on the frame of the trackless vehicle 130) and operates in a vacuum. In the embodiment shown in FIG. 9, only two wheels 122a and 122b of a set of four wheels are visible, and the suction device is not shown. The holonomic movement is achieved by rotation of Mecanum type wheels driven by respective motors mounted on the frame of the trackless vehicle 130. Holonomic movement, in which rotation and translation are separated, allows scanning in any direction in the XY plane. The trackless vehicle 130 is steerable for movement in the XY plane with the X axis parallel to the row of holes 6 to be inspected. The movement of the trackless vehicle 130 along the row of holes 6 is indicated by the long double-headed arrow labeled "Y-direction translation" in FIG. 9.

[0050] A video camera 90 is mounted on the trackless vehicle 130. The video camera 90 can be oriented so that its field of view includes the space below the multi-stage probe placement head 140. The video camera 90 captures image data and transmits the image data to a computer (not shown in FIG. 9 ). The communication channel between the video camera 90 and the computer can be via electrical cable or wirelessly. The computer uses the image feedback provided by the video camera 90 to control the precise alignment of the optical instrument 50D with the hole 6 being inspected.

[0051] Referring further to FIG. 9, the multi-stage probe placement head 140 includes a block assembly 132 attached to the trackless vehicle 130, a Z-axis stage 142 translationally coupled to the block assembly 132, an X-axis stage 144 translationally coupled to the Z-axis stage 142, and a Y-axis stage 146 translationally coupled to the X-axis stage 144. The mandrel 148 is rotatably coupled to the Y-axis stage 146. The optical device 50D is attached to the mandrel 148, and the mandrel 148 and the optical device 50D rotate integrally. The three stages of the probe placement head 140 are drivable by motors to allow the optical device 50D to move in the X, Y, or Z directions, respectively. The Z-axis stage 142 is used to raise or lower the optical device 50D. The X-axis stage 144 and the Y-axis stage 146 provide precise movement to position the optical device 50D at the center of the hole 6. The X, Y, and Z axes are axes that are perpendicular to each other in the reference coordinate frame of the trackless vehicle 130. In an ideal inspection scenario, the Z-axis of the trackless vehicle 130 is parallel to the centerline of the hole 6 being inspected. Multiple movements using a smart servo or stepper motor (not shown in FIG. 9) with feedback control (based on the image data collected by the video camera 90) are utilized to accurately position the optical device 50D with respect to the hole 6. Once the proper positioning is achieved, the optical device 50D can be lowered so that the optical probe 26 is inserted into the hole 6. The mandrel 148 can be driven to rotate by a stepper motor (not shown in FIG. 9) to allow rotation of the optical device 50D (e.g., to avoid interference between the optical device 50D and obstacles).

[0052] 9 may, for example, inspect a row of holes 6 arranged on an aircraft fuselage with the optical instrument 50D being moved successively from hole to hole. When the optical instrument 50D is in the vicinity of the next hole 6, the video camera 90 captures image data that is used to determine the position of the optical instrument 50D relative to the hole 6. X and Y stage motors (not shown) on the multi-stage probe placement head 140 are then operable to translate the optical instrument 50D in the X and / or Y directions until the optical instrument 50D and hole 6 are aligned. The glass support tube 36 may then be lowered to a starting position inside the hole 6, and the hole 6 can be scanned.

[0053] In the scenario depicted in FIG. 9, the optical instrument 50D is shown in a starting position where the centerline of the optical instrument 50D is approximately coaxial with the centerline of the hole 6. The double-headed arrows in FIG. 9 indicate the various movements that result in the scenario depicted in FIG. 9. Initially, the trackless vehicle was moved from a position where the optical instrument 50D was not in close proximity to the hole 6 to a position where the optical instrument 50D was in close proximity to the hole 6 but not aligned with the hole 6 (this position is not shown in FIG. 9). In the illustrated scenario, the trackless vehicle 130 was translated along an X-axis that is parallel to the row of holes to which the hole 6 belongs. When the hole 6 came within the field of view of the video camera 90, the trackless vehicle 130 was commanded to stop. While the trackless vehicle 130 and the optical instrument 50D were stopped, the video camera 90 was activated to collect image data representative of a field of view that included the opening 4 of the hole 6. The image data was processed by a computer (not shown in FIG. 9) using pattern recognition software to determine the location of the centerline of the hole 6 within the frame of reference of the trackless vehicle 130. The computer used the position of the bore's centrality to determine the difference between the current position of the optical instrument 50D in the reference frame of the trackless vehicle 130 and the desired starting position (shown in FIG. 9). Then, while the trackless vehicle 130 was stopped, the optical instrument 50D was moved in the X and / or Y directions from the current position to the starting position (movement in the Y direction is shown by a double-headed arrow in FIG. 9, movement in the X direction is not shown). By activating a motor (not shown) mechanically coupled to the Z-axis stage 142, the optical instrument 50D was lowered from the starting position into the bore 6 (movement in the Z direction is shown by a double-headed arrow in FIG. 9).

[0054] In the embodiment of the system shown in Figure 9, the X, Y and Z axis stages are translatably coupled by respective linear motion bearings. These translatable stages may be mechanically coupled to respective stepper motors (see probe placement head motor 54 in Figure 10) by any suitable drive mechanism known in the art. For example, each stage may have a respective attachment nut that threads onto a respective lead screw that is rotationally driven by a respective stepper motor, thereby converting rotation of the motor output shaft into translation of the stage.

[0055] 10 is a block diagram identifying several components of a computer-controlled trackless vehicle 122 platform for optical imaging and scanning of boreholes according to one embodiment. The trackless vehicle 130 includes a video camera 90 mounted on a pan-tilt device (not shown) and an optical instrument 50 mounted on a multi-stage probe placement head 140. Both the pan-tilt device and the multi-stage probe placement head 140 are mounted to a frame of the trackless vehicle 122. The optical instrument includes a microscope 10 and a light source 18, as previously described. Operation of the optical instrument 50 and the video camera 90 is controlled by a computer system 72, which may be configured with programming stored on a non-transitory, tangible computer readable storage medium (not shown).

[0056] The trackless vehicle 130 carries four wheel motors 124 that respectively drive the rotation of the four wheels 122. If the trackless vehicle is equipped with a suction device for vacuum sealing to an inclined surface, the trackless vehicle may further include a plurality of EDF motors (not shown) that drive the rotation of each of a plurality of electric ducted fans. The probe placement head 140 supports a plurality of probe placement head motors 54, three of which drive the translation of the optical instrument 50D along the X, Y and Z axes, respectively, and one of which drives the rotation of the optical instrument 50D about the Z axis. The pan-tilt apparatus includes a pan-tilt motor 76 that drives the rotation of the video camera 90 about each of the pan and tilt axes.

[0057] All motors receive power from a power source through switches (not shown) on a relay board. The state of these switches is controlled by a computer system 72 onboard the trackless vehicle 130. The computer system 72 may comprise a general purpose computer programmed with motion control application software that includes software modules for controlling the various stepper motors. The computer system 72 outputs control signals to the motor controllers 70, which selectively start / stop each motor in accordance with these control signals.

[0058] In particular, computer system 72 may be programmed to execute wireless instructions received from ground-based computer system 80. These wireless instructions are transmitted by transceiver 82 communicatively coupled to ground-based computer system 80, received by transceiver 74 onboard trackless vehicle 122, converted to a suitable digital format, and then forwarded to onboard computer system 72. Computer system 72 then controls (a) the movement of trackless vehicle 122 relative to the substrate, (b) the movement of optical instrument 50 and video camera 90 relative to the frame of trackless vehicle 122, and (3) the collection of images by optical instrument 50 and video camera 90. Thus, operation of the instruments onboard trackless vehicle 122 may be controlled by an operator in communication with ground-based computer system 80.

[0059] In particular, the probe placement head (hereinafter, "Z-axis stage motor") driving the displacement of the Z-axis stage 142 may be controlled to position the glass support tube 36 in a vertical position such that the apex or the cut-out portion of the conical mirror is located at a first depth of the hole 6. A first 360-degree image of the hole 6 is then collected while the conical mirror is parked at the first depth. The Z-axis stage motor is then controlled to position the glass support tube 36 in a vertical position such that the apex or the cut-out portion of the conical mirror is located at a second depth (different from the first depth) of the hole 6. For example, the distance separating the first and second depths may be equal to the height of the optical mirror. A second 360-degree image of the hole 6 is collected while the conical mirror is parked at the second depth. These processing steps may be repeated until the hole 6 is imaged along its entire length. At the end of this process, the collected sub-images are flattened and then the flattened sub-images are stitched together to provide a flattened image of the entire hole 6 .

[0060] In one proposed implementation, the ground computer system 80 includes a central processing unit 86 and an image processing unit 88. The central processing unit 86 is configured (e.g., programmed) to transmit second instructions to the computer system 72 via the transceiver 82 to control the movement of the optical instrument 50 and the video camera 90 and the collection of image data by the optical instrument 50 and the video camera 90. The central processing unit is further configured to receive image data collected by the optical instrument 50 and the video camera 90 via the transceiver 82 and transmit the image data to the image processing unit 88. The image processing unit 88 is configured (e.g., programmed) to process the image data. In particular, the image processing unit 88 is programmed to execute an algorithm to convert the cone-shaped optically distorted sub-images collected by the optical instrument 50 into respective flattened sub-images. The image processing unit 88 is further programmed to execute an algorithm to stitch the flattened sub-images together to form a flattened image suitable for display on the display monitor 84. The display monitor 84 includes a display processor that may be configured to display a flattened image of the hole 6 in one window and a video image of the area of ​​the substrate 2 surrounding the opening 4 in another window.

[0061] 11 is an elevation view of the optical instrument 50D mounted on the robot 100. The optical instrument 50D is attached to the robot 100 by attaching the tool side connector plate 60 to the connector 114 of the robot 100. While the optical probe 26 is parked in the hole, image data is transmitted to a data collection system for processing. Typically, the robot 100 is automatically controlled to align and move the optical probe 26 into the hole.

[0062] The robot 100 has multi-axis movement capabilities and utilizes software assistance to generate the linear profile used for scanning the hole. Specifically, the robot 100 shown in FIG. 11 includes a robot base 102, a carousel 104, a rocker 106 (i.e., pivot arm), an extendable arm 108, a robot hand 110, and a member 112 to which a connector 114 is attached. The robot base 102 and the carousel 104 are rotatably connected by a pivot connection 116. The carousel 104 and the rocker 106 are rotatably connected by a pivot connection 118. The rocker 106 and the extendable arm 108 are rotatably connected by a pivot connection 120. The rocker extendable arm 108 and the robot hand 110 are rotatably connected by a pivot connection 122. The combination of these components provides multiple degrees of freedom, which allows the optical instrument 50D to move to various positions and in various orientations. The robot 100 includes one or more position sensors (not shown) at or associated with each pivot that provide position data (X, Y, and Z in three-dimensional space) to the data collection system to accurately position the optical instrument 50D. An example of a robot 100 that may be employed with the optical instruments disclosed herein is a robot (model KR-150) manufactured by Kuka Roboter GmbH (headquartered in Augsburg, Germany), although any robot or other manipulator capable of inserting an optical instrument ultrasound inspection tool head and communicating with a data collection system may be used.

[0063] The robot 100 typically communicates with a data collection system to process image data collected by the optical instrument 50D and display the processed data. In many cases, a communication cable (not shown in FIG. 11 ) transmits data between the robot 100 and the data collection system. In other embodiments, data may be transmitted between the robot 100 and the data collection system via wireless communication. The robot 100 may be connected directly to the computer system 80 shown in FIG. 10 or indirectly via a network.

[0064] Although the method and apparatus for optical imaging and scanning of holes has been described with reference to various embodiments, those skilled in the art will recognize that various changes may be made and equivalents may be substituted for elements thereof without departing from the teachings herein. In addition, many modifications may be made to adapt the concepts and reductions to practice disclosed herein to a particular situation. Therefore, it is intended that the subject matter covered by the claims not be limited to the disclosed embodiments.

[0065] Furthermore, the present disclosure includes embodiments according to the following clauses: Clause 1. An optical instrument (50), A housing (16a, 16b, 16c); a light source (18) disposed within the housing; an image sensor (30) disposed within the housing; a conical mirror (8) disposed outside the housing and having a conical axis; a conical mirror support structure (24, 36) for supporting the conical mirror in a fixed position relative to the housing; an optical subassembly (12, 14, 20, 22) supported by the housing, the optical subassembly configured such that light from the light source impinges on the conical mirror and is reflected radially outward by the conical mirror and light propagating radially inward and impinging on the conical mirror is directed onto the image sensor; An optical instrument (50) comprising: Clause 2. The optical instrument of clause 1, wherein the conical mirror has a geometric shape defined by rotating, about the conical axis, a line disposed at an angle equal to 45 degrees relative to the conical axis. Clause 3. The conical mirror support structure comprises a central post (24); the conical mirror is truncated and attached to one end of the central post; Optical equipment as described in clause 1. Clause 4. The optical apparatus described in Clause 3, wherein the optical subassembly comprises a mirror (22), first and second lenses (12, 32) arranged along an optical axis coaxial with the conical axis, and a third lens (20) arranged between the mirror and the light source, the mirror being sized such that light propagating from the conical mirror to the image sensor passes through the first lens, around the mirror and the second lens. Clause 5. The conical mirror support structure comprises a cylindrical glass tube having a cylindrical axis; The conical mirror is disposed within the cylindrical glass tube (36) such that the conical axis is coaxial with the cylindrical axis. Optical equipment as described in clause 1. Clause 6. The optical apparatus of clause 1, wherein the optical subassembly comprises a dichroic mirror (14), first, second and third lenses (12, 20, 32), the dichroic mirror (14), first, second and third lenses (12, 20, 32) being arranged such that light propagating from the light source to the conical mirror passes through a third lens (20) and is then reflected by the dichroic mirror, and light propagating from the conical mirror to the image sensor passes through the dichroic mirror and then passes through the first and second lenses (12, 32). Clause 7. The optical apparatus of clause 1, wherein the optical subassembly comprises a dichroic mirror (14), first, second and third lenses (12, 20, 32), the dichroic mirror (14), first, second and third lenses (12, 20, 32) being arranged such that light propagating from the light source to the conical mirror passes through a third lens (20) and then passes through the dichroic mirror, and light propagating from the conical mirror to the image sensor is reflected by the dichroic mirror and passes through the first and second lenses (12, 32). Clause 8. The optical instrument of clause 1, further comprising an aperture plate (40), the third lens (20) being positioned between the aperture plate and the light source. Clause 9. A method for imaging a hole (6) in a substrate (2), comprising: (a) placing the conical mirror (8) within the hole (6) with the cone axis coaxial with the centerline of the hole and with the apex (8a) of the conical mirror or a truncated portion (8b) of the conical mirror at a first depth that is shallower than a second depth at which the base of the conical mirror is located; (b) illuminating the conical mirror with light focused onto a focal plane within the hole; (c) using the conical mirror to reflect the light described in step (b) radially outwardly towards the hole; (d) using the conical mirror to reflect the returning light axially upwards towards the opening (4) of the hole; (e) directing the light reflected axially upward by the conical mirror in step (d) onto an image sensor (30); (f) converting light impinging on the image sensor into electrical signals representing pixel data of a first distorted sub-image of a first portion of the hole having a conical optical distortion. Clause 10. The method of clause 9, further comprising storing the pixel data of the first distorted sub-image in a non-transitory, tangible computer-readable storage medium (72). Clause 11. Obtaining the pixel data of the first distorted sub-image from the non-transitory tangible computer readable storage medium; processing the pixel data of the first distorted sub-image to generate pixel data representing a first flattened sub-image free of conical optical distortion; 11. The method of claim 10, further comprising: Clause 12. The method of clause 11, further comprising displaying the flattened first sub-image on a display device (84). Clause 13. The method of clause 9, wherein the first portion of the hole is an annular surface extending from the first depth to the second depth. Clause 14. The method of clause 9, wherein the light reflected radially outward in step (c) encompasses an angle of 360 degrees. (g) moving the conical mirror along the centerline of the hole to a position where the apex of the conical mirror or a truncated portion of the conical mirror is at a third depth that is closer to the second depth than the first depth; (h) illuminating the conical mirror with light focused onto the focal plane within the hole; (i) reflecting the light described in step (h) radially outwardly toward the hole using the conical mirror; (j) reflecting the returning light described in step (i) axially upwards towards the aperture using the conical mirror; (k) directing the light reflected axially upward by the conical mirror in step (j) onto an image sensor (30); (l) converting light impinging on the image sensor into electrical signals representative of pixel data of a second distorted sub-image of a second portion of the aperture having a conical optical distortion; 10. The method of claim 9, further comprising: Clause 16. Processing the pixel data of the first distorted sub-image to generate pixel data representing a first flattened sub-image free of conical optical distortion; processing the pixel data of the second distorted sub-image to generate pixel data representing a second flattened sub-image free of conical optical distortion; stitching the first and second flattened sub-images together; and presenting a flattened image including at least the first and second flattened sub-images on a display device (84); 16. The method of claim 15, further comprising: Clause 17. An apparatus (130) for imaging a hole (6) in a substrate (2), comprising: a multi-stage probe placement head (140) comprising a block assembly (132), a first stage (144) translatable relative to the block assembly along a first axis, a second stage (146) translatable relative to the block assembly along a second axis orthogonal to the first axis, and a third stage (142) translatable relative to the block assembly along a third axis orthogonal to the first and second axes, the third stage translatably coupled to the second stage and the second stage translatably coupled to the first stage; an optical instrument (50) supported by and depending from said third stage, a translatable housing (16a, 16b, 16c) connected to the third stage; a light source (18) disposed within the housing; an image sensor (30) disposed within the housing; a conical mirror (8) disposed outside the housing and having a conical axis parallel to the first axis; a conical mirror support structure (24, 36) for supporting the conical mirror in a fixed position relative to the housing; an optical subassembly (12, 14, 20, 22) supported by and configured with the housing such that light from the light source impinges on the conical mirror, is reflected radially outward by the conical mirror, and light propagating radially inward and impinging on the conical mirror is directed onto the image sensor; an optical instrument (50) comprising: An apparatus (130) comprising: Clause 18. The conical mirror support structure comprises a central post (24); the conical mirror is truncated and attached to one end of the central post; 17. Apparatus according to clause 17. Clause 19. The conical mirror support structure comprises a cylindrical glass tube (36) having a cylindrical axis; the conical mirror is disposed within the cylindrical glass tube such that the conical axis is coaxial with the cylindrical axis; 17. Apparatus according to clause 17. Clause 20. A system for imaging holes in a substrate, comprising: an automated device (130) configured to move the end effector (60) by operation of the motor (54); An optical instrument (50) mounted on the end effector (60), a housing (16a, 16b, 16c) connected to the end effector; a light source (18) disposed within the housing; an image sensor (30) disposed within the housing; a conical mirror (8) disposed outside the housing and having a conical axis parallel to the first axis; a conical mirror support structure (24, 36) for supporting the conical mirror in a fixed position relative to the housing; an optical subassembly (12, 14, 20, 22) supported by and configured with the housing such that light from the light source impinges on the conical mirror and is reflected radially outward by the conical mirror, and light propagating radially inward and impinging on the conical mirror is directed onto the image sensor; an optical instrument comprising: an image processor (88) configured to receive the cone-shaped optically distorted image collected by the optical instrument and then process pixel data of the cone-shaped optically distorted image to generate pixel data representing a flattened image free of cone-shaped optical distortion; A system comprising:

[0066] As used herein, the term "computer system" should be interpreted broadly to include systems having at least one computer or processor, and systems that may have multiple computers or processors communicating over a network or bus. As used in the previous sentence, the terms "computer" and "processor" both refer to a device that has a processing device (e.g., a central processing unit) and some form of memory (i.e., a computer-readable medium) that stores programs that can be read by the processing device.

[0067] The methods described herein may be encoded as executable instructions embodied in a non-transitory, tangible computer-readable storage medium, including but not limited to a storage device and / or a memory device, such instructions, when executed by a processor or computer, cause the processor or computer to perform at least a portion of the methods described herein.

[0068] Unless the claim language expressly specifies or recites a condition dictating a particular order for performing some or all of the steps recited in the claim, the method claims set forth below should not be construed as requiring those steps to be performed in alphabetical order (any alphabetical order in the claims is used only to refer to steps already recited) or in the order in which those steps are recited, nor should the method claims be construed as excluding any portion of two or more steps being performed simultaneously or alternating, unless the claim language expressly specifies a condition excluding such an interpretation.

Claims

1. An optical instrument (50), comprising: A housing (16a, 16b, 16c); a light source (18) disposed within the housing; an image sensor (30) disposed within the housing; a conical mirror (8) disposed outside the housing and having a conical axis; a conical mirror support structure (24, 36) for supporting the conical mirror in a fixed position relative to the housing; an optical subassembly (12, 14, 20, 22) supported by the housing, configured such that light from the light source impinges on the conical mirror and is reflected radially outward by the conical mirror, and such that light from a bore surface that propagates radially inward and impinges on the conical mirror is directed onto the image sensor; an image processor (88) configured to convert pixel data having image sensor coordinates obtained by directing light from the hole surface impinging on the conical mirror onto the image sensor into pixel data having cylindrical coordinates corresponding to the hole surface, and to unfold a cylindrical shell of the hole surface into a flattened two-dimensional plane having an X-axis and a Y-axis; Equipped with The Y-axis is parallel to a Z-axis that is coaxial with the cone axis.

2. 2. The optical instrument of claim 1, wherein the conical mirror has a geometric shape defined by rotating, about a conical axis, a line disposed at an angle equal to 45 degrees relative to the conical axis.

3. The conical mirror support structure includes a central post (24); the conical mirror is truncated and attached to one end of the central post; 2. The optical instrument according to claim 1.

4. 4. The optical instrument of claim 3, wherein the optical subassembly comprises a mirror (22), first and second lenses (12, 32) arranged along an optical axis coaxial with the cone axis, and a third lens (20) arranged between the mirror and the light source, the mirror being sized such that light propagating from the cone mirror to the image sensor passes through the first lens, a periphery of the mirror, and the second lens.

5. The conical mirror support structure comprises a cylindrical glass tube (36) having a cylindrical axis; the conical mirror is disposed within the cylindrical glass tube such that the conical axis is coaxial with the cylindrical axis; 2. The optical instrument according to claim 1.

6. 2. The optical instrument of claim 1, wherein the optical subassembly comprises a dichroic mirror (14), and first, second and third lenses (12, 20, 32), the dichroic mirror (14), and the first, second and third lenses (12, 20, 32) are arranged such that light propagating from the light source to the conical mirror passes through a third lens (20) and is then reflected by the dichroic mirror, and light propagating from the conical mirror to the image sensor passes through the dichroic mirror and then through the first and second lenses (12, 32).

7. 2. The optical instrument of claim 1, wherein the optical subassembly comprises a dichroic mirror (14), and first, second and third lenses (12, 20, 32), the dichroic mirror (14), and the first, second and third lenses (12, 20, 32) are arranged such that light propagating from the light source to the conical mirror passes through a third lens (20) and then through the dichroic mirror, and light propagating from the conical mirror to the image sensor is reflected by the dichroic mirror and passes through the first and second lenses (12, 32).

8. The optical instrument of claim 1 , further comprising an aperture plate (40), a third lens (20) disposed between said aperture plate and said light source.

9. 2. The optical instrument of claim 1, included in an apparatus (130) for imaging a hole (6) in a substrate (2), the apparatus comprising a multi-stage probe placement head (140), the multi-stage probe placement head (140) comprises a block assembly (132), a first stage (144) translatable relative to the block assembly along a first axis, a second stage (146) translatable relative to the block assembly along a second axis perpendicular to the first axis, and a third stage (142) translatable relative to the block assembly along a third axis perpendicular to the first and second axes, the third stage translatably coupled to the second stage, the second stage translatably coupled to the first stage, The optical instrument (50) is supported by and depends from the third stage.

10. A method for imaging a hole (6) in a substrate (2), comprising the steps of: (a) placing the conical mirror (8) within the hole (6) with the conical axis coaxial with the centerline of the hole and with the apex (8a) of the conical mirror or a truncated portion (8b) of the conical mirror at a first depth that is shallower than a second depth at which the base of the conical mirror is located; (b) illuminating the conical mirror with light focused onto a focal plane within the hole; (c) reflecting the light described in step (b) radially outwardly toward the hole using the conical mirror; (d) using the conical mirror to reflect the returning light axially upwards towards the opening (4) of the hole; (e) directing the light reflected axially upward by the conical mirror in step (d) onto an image sensor (30); (f) converting light impinging on the image sensor into electrical signals representative of pixel data of a first distorted sub-image of a first portion of the aperture having a conical optical distortion; (g) converting, with an image processor, pixel data of the first distorted sub-image into pixel data having cylindrical coordinates corresponding to a hole surface; (h) unfolding, by the image processing device, the cylindrical shell of the hole surface into a flattened two-dimensional plane having an X-axis and a Y-axis, the Y-axis being parallel to a Z-axis that is coaxial with the cone axis; and The method includes:

11. The method of claim 10, further comprising storing the pixel data of the first distorted sub-image in a non-transitory, tangible computer readable storage medium (72).

12. obtaining the pixel data of the first distorted sub-image from the non-transitory tangible computer readable storage medium; processing the pixel data of the first distorted sub-image to generate pixel data representing a first flattened sub-image free of conical optical distortion; The method of claim 11 further comprising:

13. The method of claim 12, further comprising displaying the first flattened sub-image on a display device (84).

14. (i) moving the conical mirror along a centerline of the hole to a position where the apex of the conical mirror or a truncated portion of the conical mirror is at a third depth that is closer to the second depth than the first depth; (j) illuminating the conical mirror with light focused onto the focal plane within the hole; (k) reflecting the light described in step (j) radially outwardly toward the hole using the conical mirror; (l) using the conical mirror to reflect the returning light of step (k) axially upwards towards the aperture; (m) directing the light reflected axially upward by the conical mirror in step (l) onto an image sensor (30); (n) converting light impinging on the image sensor into electrical signals representative of pixel data of a second distorted sub-image of a second portion of the aperture having a conical optical distortion; 14. The method of any one of claims 10 to 13, further comprising:

15. processing the pixel data of the first distorted sub-image to generate pixel data representing a first flattened sub-image free of conical optical distortion; processing the pixel data of the second distorted sub-image to generate pixel data representing a second flattened sub-image free of conical optical distortion; stitching the first and second flattened sub-images together; presenting a flattened image including at least the first and second flattened sub-images on a display device (84); The method of claim 14 further comprising:

Images