Inspection system and method

Abstract

To provide an inspection system and method.SOLUTION: The inspection system for capturing a topography of a surface includes: a support frame disposed on the surface; at least three light sources attached to different places on the support frame; and at least two cameras each having a visual field direction and attached to different places on the support frame so that the visual field direction of each camera can be substantially directed to a common region of the surface. The inspection system further includes a processor that temporarily activates the light sources one at a time, causes all the cameras to capture a plurality of images of the common region every time the light sources are activated, and combines the images so as to realize a reconstructed surface representing the topography of the common region.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] FIELD OF THE DISCLOSURE This disclosure relates generally to inspection systems, and more particularly to systems and methods for inspecting surfaces for imperfections. [Background technology]

[0002] Surface imperfections on manufactured parts can be difficult to detect and measure. Part of the challenge in detecting surface imperfections is due to their small size, some of which can only be detected under certain lighting conditions or by tactile methods such as running a fingernail over the surface. Detecting surface imperfections early in the manufacturing process is important because it prevents defective parts from moving down the manufacturing line and potentially reaching the end customer. Additionally, early detection of surface imperfections allows for rework or replacement of the defective parts, and adjustments to the manufacturing process to prevent future occurrences of surface imperfections.

[0003] One conventional approach to inspecting surfaces involves the use of structured light scanners. Although structured light scanners are effective at detecting large surface defects such as dents, they lack the precision required to detect small surface imperfections such as scratches, gouges, and tool marks due to the inherent low frequency noise (e.g., unevenness) of the surface. Laser profilometers are another type of device conventionally used to inspect surfaces. Although laser profilometers are generally effective for their intended purposes, they have a relatively small field of view. Additionally, laser profilometers have reflection issues that affect their ability to inspect certain types of surfaces. Summary of the Invention [Problem to be solved by the invention]

[0004] As can be seen, there is a need in the art for a system for inspecting surfaces that has a relatively large field of view and is capable of detecting small imperfections on a variety of different types of surfaces. [Means for solving the problem]

[0005] The above-mentioned needs related to inspecting a surface are addressed by the present disclosure, which provides an inspection system for capturing the topography of the surface. The inspection system includes a rigid support frame disposed over the surface, at least three light sources mounted at different locations on the support frame, and at least two cameras, each having a line of sight, mounted at different locations on the support frame such that the line of sight of each camera is generally directed toward a common area of ​​the surface. The inspection system also includes a processor that temporarily activates the light sources one at a time, causes all of the cameras to capture multiple images of each of the common areas each time a light source is activated, and combines the images to provide a reconstructed surface representative of the topography of the common area.

[0006] In another example, an inspection system includes a rigid support frame and a movable platform that positions the support frame at one or more different locations on a surface. The inspection system also includes at least three light sources mounted at different locations on the support frame, and at least two cameras, each having a line of sight, mounted at different locations on the support frame such that the line of sight of each camera is generally directed toward a common area of ​​the surface. Additionally, the inspection system also includes a processor that temporarily activates the light sources one at a time, causes all of the cameras to capture multiple images of each of the common areas each time the light sources are activated, and combines the images to correct for geometric distortions and provide a reconstructed surface that represents the topography of the common area.

[0007] A method of capturing a topography of a surface is also disclosed. The method includes the step of placing a rigid support frame on the surface. The support frame has at least three light sources mounted at different locations on the support frame and at least two cameras, each having a viewing direction, mounted at different locations on the support frame such that the viewing direction of each camera is generally directed toward a common area of ​​the surface. The method further includes the step of temporarily activating the light sources one at a time and causing all of the cameras to capture multiple images of each of the common areas each time a light source is activated. Additionally, the method includes the step of combining the images to result in a reconstructed surface representative of the topography of the common area.

[0008] The discussed features, functions, and advantages can be achieved independently in various versions or may be combined in yet other versions, further details of which can be understood with reference to the following description and drawings.

[0009] The present disclosure can be better understood by referring to the following detailed description in conjunction with the accompanying drawings, which show preferred exemplary versions, but which are not necessarily drawn to scale, and which are examples and are not meant to be limitations on the description or claims. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 illustrates an example of an aircraft. [Diagram 2] 2 illustrates an example of one of the fuselage sections of the aircraft of FIG. 1 and shows a robotic device supporting an inspection system of the present disclosure for inspecting the surface of a structure such as a fuselage section. [Diagram 3] FIG. 3 is an end view of the fuselage section of FIG. 2 during inspection by an inspection system. [Figure 4]FIG. 1 is a close-up view of an example of an inspection system positioned relative to a surface of a torso section, showing inspection system components including a rigid support frame, a number of light sources supported on the support frame, and a number of cameras supported on the support frame and configured to capture multiple images of a common area of ​​the surface during sequential activation of the light sources. [Diagram 5] FIG. 1 illustrates an example of an inspection system configured to be manually held in a predetermined position relative to a surface under inspection and having a graphic user interface (GUI - e.g., a tablet) mounted on a support frame for displaying a reconstructed surface generated by the inspection system as a result of combining multiple images captured by the cameras. [Figure 6] FIG. 6 is a side view of the inspection system of FIG. 5 positioned relative to a surface of a fuselage section. [Figure 7] FIG. 2 is a side view of an inspection system showing a camera having a viewing direction that forms an angle of incidence with the surface under inspection, and further showing a light source having an illumination direction that forms an angle of incidence with the surface. [Figure 8] FIG. 1 shows an example of a camera mounted on one of the beams of the support frame. [Figure 9] FIG. 1 shows an example of a light source mounted on one of the beams of the support frame. [Figure 10] FIG. 8 is a top view of the inspection system of FIG. 7 in an arrangement having a total of three light sources and a total of three cameras mounted on a support frame. [Figure 11] FIG. 13 is an exploded view of a portion of the support frame showing multiple beams separated away from one of the connecting elements. [Figure 12] FIG. 1 illustrates an example of an inspection system in which the vertically oriented beam is omitted from the support frame, and further illustrates a total of nine light sources and a total of three cameras mounted on the support frame. [Figure 13] FIG. 1 illustrates an example of an inspection system in which the tilt beam is omitted from the support frame and further illustrates a reference target attached to the bottom end of the support frame. [Figure 14]FIG. 13 is an enlarged view of a portion of the support frame showing a reference target mounted on one of the beams. [Figure 15] FIG. 15 is an enlarged view of one of the reference targets of FIG. 14. [Figure 16] FIG. 1 illustrates an example of a reference target configured as a coded target. [Figure 17] FIG. 2 is a partial exploded view of an example inspection system having a background blocker. [Figure 18] FIG. 1 illustrates an inspection system with a background block assembled to a support frame. [Figure 19] FIG. 1 illustrates an example of a camera having a lens filter configured to filter out one or more wavelengths of background light. [Figure 20] FIG. 2 is a block diagram of an example inspection system having a processor for controlling the operation of the camera and light source and for generating a reconstructed surface based on images captured by the camera. [Figure 21] FIG. 13 illustrates an example display screen of a GUI that displays a reconstructed surface of a surface under inspection based on images captured by a camera and shows defects in the reconstructed surface in the form of scratches. [Figure 22] FIG. 1 illustrates an example of an inspection system positioned relative to a surface of a structure having a common structural feature in the form of a fastener. [Figure 23] FIG. 1 shows an example display showing a reconstructed surface in which common structural features in the image have been masked, and showing defects in the reconstructed surface in the form of gouges. [Figure 24] FIG. 1 shows an example of a display screen showing a reconstructed surface showing the texture of the inspected surface. [Diagram 25] 1 is a flowchart of operations involved in a method for capturing a topography of a surface under inspection. [Figure 26] 1 is a flowchart of operations included in a method of detecting defects in a reconstructed surface produced by the inspection system of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The figures shown in this disclosure represent various aspects of the presented versions and only the differences are described in detail.

[0012] The disclosed versions are now more fully described below with reference to the accompanying drawings, in which some, but not all, of the disclosed versions are shown. Furthermore, several different versions may be provided and should not be construed as being limited to the versions shown in this specification. Rather, these versions are provided so that this disclosure will be thorough and will fully convey the scope of the disclosure to those skilled in the art.

[0013] This specification includes references to "one version" or "a version." Occurrences of the phrases "in one version" or "in a version" do not necessarily refer to the same version. Similarly, this specification includes references to "one example" or "an example." Occurrences of the phrases "one example" or "an example" do not necessarily refer to the same example. Particular features, structures, or characteristics may be combined in any suitable manner consistent with the present disclosure.

[0014] As used herein, "comprising" is an open-ended term and, when used in the claims, the term does not exclude additional structures or steps.

[0015] As used herein, "configured to" means that various parts or components may be described or claimed as being "configured to" perform one or more tasks. In this context, "configured to" is used to suggest structure by indicating that the parts or components include structure that performs those one or more tasks during operation. Thus, a part or component may be said to be configured to perform a task even when the specified part or component is not currently operating (e.g., not on).

[0016] As used herein, an element or step listed in the singular and preceded by the word "a" or "an" should be understood not to necessarily exclude a plurality of elements or steps. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items. Also, as used herein, the term "combinations thereof" includes combinations having at least one of the associated listed items, where the combination may further include additional items, such as unlisted items.

[0017] As used herein, the phrase "at least one of," when used in conjunction with a list of items, means that one or more different combinations of the listed items may be used, and only one of each item in the list may be required. In other words, "at least one of" means that any combination of items, and some items from the list may be used, but not necessarily all items from the list are required. An item may be a particular object, thing, or category.

[0018] In this disclosure, the terms "surface of the structure," "surface of the fuselage section," "surface of the skin panel," "surface to be inspected," "surface under inspection," "surface to be inspected," and "surface" are used interchangeably.

[0019] Referring now to the drawings illustrating various examples of the present disclosure, FIG. 1 illustrates an aircraft 302 serving as one example of a structure 300 in which the inspection system 100 (FIG. 2) and method 500 (FIG. 25) of the present disclosure may be used to capture a topography of a surface 310 (i.e., a surface to be inspected) and inspect the surface 310 for defects 200 (FIGS. 21, 23, and 24). The aircraft 302 includes a fuselage 304, a pair of wing sections 314, and a tail including a tail surface 316, such as a horizontal elevator and a vertical fin. In the illustrated example, the fuselage 304 is constructed by separately manufacturing fuselage sections 306, which are then joined together at corresponding section joints 184, as shown in FIG. 1.

[0020] 2-4, the inspection system 100 includes a rigid support frame 102 (e.g., FIG. 4), at least three light sources 150 (FIG. 4) mounted at different locations on the support frame 102, at least two cameras 130 (FIG. 4) also mounted at different locations on the support frame 102, and a processor 170 (FIGS. 2 and 20). As described in more detail below, the processor 170 sequentially activates the light sources 150 in synchronization with the cameras 130 to capture multiple images 140 (e.g., FIG. 21) of the surface 310 from different viewpoints. The processor 170 performs surface reconstruction based on the images 140 using photometric stereo techniques, thereby forming a reconstructed surface 174 (e.g., FIG. 21 - three-dimensional digital model) that represents the topography of the surface 310 under inspection. Additionally, the processor 170 analyzes the reconstructed surface 174 for imperfections and defects 200, as described in more detail below.

[0021] Advantageously, multiple images 140 taken from different viewpoints allow for a relatively large field of view of the surface 310 under inspection. Additionally, multiple images 140 reduce distortion of low frequency features in the reconstructed surface 174, allowing for a high level of accuracy in the detection of small surface features such as texture 206 (FIG. 24) and defects including scratches 202 (e.g., FIG. 21), gouges 204 (e.g., FIG. 23), tool marks (not shown), and other types of discontinuities that are only detectable under certain lighting conditions or otherwise by touching the surface 310 with a finger or fingernail. Advantageously, the inspection system 100 is a relatively small, lightweight, and low-cost device that can operate to rapidly detect surface defects and imperfections that are not detectable using conventional inspection devices.

[0022] 2-3 show an example of an inspection system 100 of the present disclosure positioned relative to one of the torso sections 306 to capture the surface topography of the torso section 306 and / or to inspect the exterior surface 310. In the example shown, the torso section 306 has a generally cylindrical shape with a row of passenger windows 318 on either side of the torso section 306. Each of the windows 318 has a window surround 188 that provides localized strength reinforcement around the window opening 116 of the torso section 306. In some examples, the torso section 306 is of metal construction and the exterior surface 310 is defined by the skin panel 186. However, in other examples, the torso section 306 can be formed at least in part of a composite material.

[0023] 3 illustrates the inspection system 100 implemented on a fuselage section 306 (e.g., structure 300) statically supported on an assembly fixture 308 in a manufacturing facility. In other examples not shown, the inspection system 100 may be implemented to capture a surface topography and / or inspect a surface 310 of the structure 300 at a field operation. For example, the inspection system 100 may be implemented to capture a surface topography and / or inspect a surface 310 of a fuselage 304, wing 314, or other component of an in-service aircraft 302 at a maintenance facility during one of its regularly scheduled inspections. Although described in the context of a fuselage section 306 of a fuselage 304, the inspection system 100 and method 500 of the present disclosure may be implemented to inspect any one of a variety of different types of structures of any size, shape, configuration, and material composition, without limitation. Further in this regard, the inspection system 100 and method 500 of the present disclosure can be implemented to inspect planar and non-planar surfaces, including contoured and complex curved surfaces.

[0024] 2-14, several examples of inspection systems 100 are shown that include the support frame 102 described above. The support frame 102 has a top end 108 and a bottom end 112. The top end 108 of the support frame 102 is coupled to a movable platform 400. The bottom end 112 of the support frame 102 is configured to be positioned proximate to the surface 310 under inspection. The movable platform 400 is configured to move the inspection system 100 to any one or more locations along the surface 310 under inspection and to hold the inspection system 100 stationary at each location during activation of the light source 150 and the camera 130 as described below.

[0025] In the example of FIGS. 2-3, the mobile platform 400 is a robotic device 402 having a base that is movable along a robotic device track 406 to allow the inspection system 100 to be positioned at any point along the length of the torso section 306 or other structure 300. Although shown as a robotic device 402, the mobile platform 400 can be provided in any one of a variety of alternative configurations. For example, the mobile platform 400 can be an overhead gantry (not shown) or any other type of mobile platform 400 that can position the inspection system 100 relative to the structure 300. In some examples, the mobile platform 400 can then sequentially position the inspection system 100 at different points along the structure 300, generating a reconstructed surface 174 ( FIG. 21 ) at each point. In some examples, the processor 170 ( FIG. 20 ) can stitch the reconstructed surfaces 174 together to form a three-dimensional representation of a substantial portion of the surface 310 of the structure 300.

[0026] In the example of FIG. 2, the inspection system 100 includes a graphic user interface (GUI 210) in the form of a laptop computer 212 that includes a processor 170 communicatively coupled to the light source 150 and the camera 130. The laptop computer 212 has a display screen 216 configured to display a reconstructed surface 174 of the surface 310 under inspection. As described below, FIG. 21 shows an example of the display screen 216 of the GUI 210 displaying an image 140 captured by an inspection system 100 that includes exactly two cameras 130 (e.g., a first camera 131 and a second camera 132) and exactly three light sources 150 (e.g., a first light source 151, a second light source 152, and a third light source 153), and also displaying a reconstructed surface 174. Also shown within the reconstructed surface 174 are surface imperfections or defects 200 detected by the processor 170. The image 140 and / or the reconstructed surface 174 may be displayed in real time or shortly (eg, within a few seconds) after the image 140 is captured and processed.

[0027] 4-14, the support frame 102 is configured as a truss structure made up of a number of elongated beams 104 and a number of connecting elements 106. In the illustrated example, the beams 104 are lightweight tubes formed of a high strength, high stiffness material such as a polymer reinforced fiber matrix (e.g., carbon fiber composite). However, the beams 104 may be provided in configurations other than tubes and may be formed from any suitable material, including any metallic material (e.g., aluminum) and / or any non-metallic material, such as a non-fibrous polymeric material.

[0028] The connecting elements 106 are configured to interconnect the beams 104. In this regard, each of the connecting elements 106 has a number of openings 116, each of which is sized to closely receive an end of a beam 104, as shown in FIG. 11. In the illustrated example, the connecting elements 106 are generally spherical in shape and formed from a high strength polymeric material. However, the connecting elements 106 may be provided in any shape or size and may be formed from any type of material (e.g., composite material, metallic material) or any combination of materials.

[0029] The connection elements 106 at the bottom end 112 of the support frame 102 are referred to as frame feet 114. The frame feet 114 are interconnected by beams 104. In the illustrated example, the frame feet 114 are hemispherical and have a generally flat bottom for placement against a surface 310 during testing, as shown in FIG. 7. However, in other examples, the frame feet 114 can have the same size, shape, and configuration as the remaining connection elements 106 of the support frame 102. In some examples, the support frame 102 can include force sensors (not shown) on the frame feet 114 to provide force feedback to the movable platform 400 (e.g., robotic device 402) to limit the amount of force applied by the support frame 102 against the surface 310, thereby avoiding damage to the support frame 102 or the surface 310 during testing.

[0030] The interconnection of the beams 104 via the connecting elements 106 results in the support frame 102 having a high level of rigidity. For example, the support frame 102 is configured to limit relative movement between any two points on the support frame 102 (e.g., any two connecting elements 106). Advantageously, providing the support frame 102 as a rigid structure enables the inspection system 100 to generate a reconstructed surface 174 with a high level of accuracy, which enables a high level of precision in detecting small features in the surface 310 under inspection.

[0031] In the example of FIG. 10, the truss structure of the support frame 102 is arranged in a dome-like shape having a hexagonal cross-sectional shape when viewed from above and below. However, the multiple openings 116 (FIG. 11) in the connection elements 106 allow a high level of customization in the size and shape of the support frame 102 to accommodate surfaces of various different sizes (e.g., different radii of curvature) and / or shapes (e.g., planar, cylindrical, conical, doubly curved, etc.). For example, FIG. 12 shows an example of a support frame 102 in which the vertically directed beam 104 of FIG. 7 is omitted, thereby reducing the overall height of the support frame 102 and thereby allowing the camera 130 and light source 150 to be placed closer to the surface 310 under inspection than permitted by the support frame 102 of FIG. 7. In contrast, Figure 13 shows an example of a support frame 102 in which the angled beam 104 of Figure 7 is omitted and the vertically directed beam 104 is relatively long, increasing the overall height of the support frame 102 and thereby allowing the camera 130 and light source 150 to be positioned further from the surface 310 than permitted by the support frame 102 of Figure 7. Positioning the camera further from the surface 310 allows for a larger field of view for the camera 130, allowing a larger portion of the structure 300 to be inspected by the inspection system 100.

[0032] Although the support frame 102 is shown as a truss structure in the figures, the support frame 102 can be provided in any one of a variety of alternative configurations capable of supporting multiple cameras 130 and multiple light sources 150. For example, the support frame 102 can be provided as a rigid one-piece shell-like structure (not shown) preferably formed of a lightweight material. Such a shell-like support frame can have a dome-like shape, and the cameras 130 and light sources 150 can be mounted inside the shell-like support frame. In one example, the shell-like support frame can be shaped similar to the background light blocker 160 shown in Figures 15-16. However, the shell-like support frame can have any one of a variety of different shapes, including rounded shapes, and is not limited to the shapes shown in Figures 15-16. Advantageously, the shell-like support frame can serve the dual role of supporting the cameras 130 and light sources 150 and blocking background light.

[0033] 5-6 show an example of an inspection system 100 that, due to its light weight, can be manually held in place relative to a surface 310 under inspection. In the illustrated example, the support frame 102 includes a pair of handles 110 extending from either side of a top end 108 of the support frame 102, thereby allowing a technician to use both hands to grasp the handles 110 and hold the support frame 102 in place near or relative to the surface 310. As described in more detail below, the example of FIGS. 5-6 includes a GUI 210 in the form of a tablet 214. The tablet 214 can be attached to the handles 110 and / or the top end 108 of the support frame 102 and is configured to display (e.g., in real time) the images 140 captured by the camera 130 and / or a reconstructed surface 174 based on the images 140 captured by the camera 130.

[0034] As mentioned above, in each of the examples disclosed herein, the inspection system 100 includes at least three light sources 150 mounted at different locations on the support frame 102. FIG. 9 shows an example of a light source 150 clamped to one of the beams 104 of the support frame 102. As described in more detail below, the light sources 150 are activated one at a time (via the processor 170) to temporarily illuminate at least a common area 312 (FIGS. 4 and 7) of the surface 310 from different angles while the camera 130 captures an image 140 of the common area 312. The common area 312 can be described as the portion of the surface 310 that appears in all images 140 captured by the different cameras 130. In the examples of FIG. 4 and FIG. 7, the common area 312 is surrounded by the support frame 102.

[0035] In the examples of Figures 4-10, the inspection system 100 includes exactly three light sources 150 as described above. However, in other examples, the inspection system 100 may include four or more light sources 150. For example, the inspection systems 100 of Figures 12 and 13 each have a total of nine light sources 150 mounted at different locations on the support frame 102. In each of the examples of Figures 4-12, the light sources 150 are arranged at equal angular intervals from one another when the inspection system 100 is viewed from above. However, in other examples not shown, the light sources 150 may be arranged at non-equidistant angular intervals.

[0036] The light sources 150 may be omnidirectional light sources (not shown), or the light sources 150 may be directional light sources 154 (FIG. 7) each configured to project a majority of its light along an illumination direction 156 (FIG. 7) as shown in FIGS. 4 and 7. The illumination direction 156 of each directional light source 154 is generally oriented toward a common area 312 of the surface 310 under inspection. In FIG. 7, the directional light sources 154 are mounted to the support frame 102 and / or oriented such that the illumination direction 156 of each directional light source 154 forms an incidence angle 158 of 15-75 degrees with the common area 312. To maximize the shadow effect on the surface 310 under inspection, the incidence angle 158 of the illumination direction 156 of each directional light source 154 is preferably 15-45 degrees. In the illustrated example, the light sources 150 are linearly polarized spotlights configured to emit polarized light to reduce specular reflections from the surface 310 under inspection, thereby improving the accuracy of the reconstructed surface 174.

[0037] As mentioned above, the inspection system 100 includes at least two cameras 130. Figure 8 shows an example of one of the cameras 130 clamped onto one of the beams 104. In Figure 8, the cameras 130 are mounted at different locations on the support frame 102 to capture images 140 of the surface 310 from different viewpoints. As mentioned above, the multiple images 140 taken from different viewpoints correct for low frequency geometric distortions in the reconstructed surface 174. Additionally, the multiple cameras 130 mounted at different locations allow for a relatively large field of view, thereby reducing the total amount of time required to inspect a large surface 310.

[0038] Although a minimum of two cameras 130 is required, the inspection system 100 can include three or more cameras 130. For example, each of the inspection systems 100 of Figures 4-12 includes exactly three cameras 130. In another example, the inspection system 100 of Figure 13 has exactly four cameras 130, including three cameras 130 mounted on the tilted beam 104 of the support frame 102 and one centrally located overhead camera 130 (e.g., a panoramic camera) mounted on the top end 108 of the support frame 102. Regardless of the number of cameras 130 in the inspection system 100, each camera 130 has a direction of view 136 (Figure 7), and all of the cameras 130 are mounted on the support frame 102 such that their respective directions of view 136 are generally directed toward a common area 312 of the surface 310. In some examples, at least some of the cameras 130 may be mounted such that when the inspection system 100 is placed on or in contact with the surface 310 under inspection, the viewing directions 136 of at least some of the cameras 130 are locally non-normal to the surface 310, thereby enhancing the ability to capture in the image 140 shadows generated by discontinuities in the surface 310. In the illustrated example, the cameras 130 are equiangularly spaced from one another when viewed from above the inspection system 100. However, in other examples not shown, the cameras 130 may be non-equidistantly spaced.

[0039] In some examples, the inspection system 100 includes reference targets 120 each mounted at a different location on the support frame 102. The reference targets 120 are used for scaling verification of the image 140 and / or to calculate the camera position. In examples where the inspection system 100 includes the reference targets 120, each camera 130 is configured to capture an image 140 including the reference targets 120 in addition to capturing a common area 312 of the surface 310 under inspection. In order to determine the position of the camera 130, the common area 312 must encompass a sufficient number of the reference targets 120. More specifically, the cameras 130 of the inspection system 100 are configured and oriented such that there are a minimum of three common reference targets 120 visible in each camera view and at least one other camera view that includes those same three reference targets 120. For example, the inspection system 100 has exactly two cameras 130, including a first camera 131 and a second camera 132, where each image 140 captured by the first camera 131 includes at least three reference targets 120 and each image 140 captured by the second camera 132 also includes at least the same three reference targets 120. Increasing the number of cameras 130 in the inspection system 100 increases the accuracy of the reconstructed surface 174, which in turn allows for increased accuracy with which the processor 170 can detect surface features such as defects 200 in the surface 310. As can be appreciated, the inspection system 100 can have more than two cameras 130 which correspondingly increases the accuracy of the reconstructed surface 174.

[0040] 13-14 show reference targets 120 mounted on at least two of the beams 104 interconnecting the frame feet 114 at the bottom end 112 of the support frame 102. The two reference targets 120 on each beam 104 are spaced apart by a known scaling distance 119 that allows such beam 104 to function as a scale bar 118. The scaling distance 119 between the reference targets 120 on each scale bar 118 is measured separately by a high-precision measuring device (not shown). The support frame 102 of the inspection system 100 preferably includes at least one scale bar 118 oriented substantially parallel (e.g., within 30 degrees) to the x-axis of the local coordinate system 442 and at least one scale bar 118 oriented substantially parallel (e.g., within 30 degrees) to the y-axis of the local coordinate system 442 to facilitate scaling verification of the image 140 in the x and y directions, respectively. The support frame 102 of FIG. 13 has a scale bar 118 located at the bottom end 112, but any one or more of the beams 104 of the support frame 102 may include a reference target 120 that enables the beam 104 to function as the scale bar 118.

[0041] The processor 170 uses photogrammetry techniques for scaling validation of the images 140 based on the scaling distances 119 between the reference targets 120 on the scale bar at 18. Additionally, as described above, the processor 170 uses photogrammetry techniques to calculate the camera position relative to the global coordinate system 440 (FIG. 2) of the structure 300 and / or calculate the position and orientation of the inspection system 100 (e.g., local coordinate system 442-FIG. 2). The scaling validation of the images 140 and / or the calculation of the camera position is performed as a separate step prior to generating the reconstructed surface 174. During inspection of the surface 310, the scaling validation and / or the calculation of the camera position can be performed in the background after all the images 140 have been captured.

[0042] With brief reference to FIGS. 15-16, a reference target 120 is shown having a circular shape. However, the reference target 120 may be provided in any of a variety of shapes, such as a square or other shapes. FIG. 16 illustrates an example of a coded target 122. Each coded target 122 of the inspection system 100 has a unique pattern or code (e.g., a coded ring pattern) that is recognized by the processor 170 when processing the image 140 to calibrate the camera 130 and / or determine the camera's position and orientation. In this regard, the coded targets 122 contribute to the generation of a highly accurate reconstructed surface 174, enabling accurate detection and characterization of defects 200 in the surface 310.

[0043] In some examples, the inspection system 100 may include one or more temperature sensors 124 and / or 126 for measuring the temperature of the support frame 102 and / or the temperature of the ambient air. For example, FIG. 14 shows an example of a support frame 102 having a temperature sensor 124 attached to one of the beams 104 and configured to measure the ambient air temperature. Also shown is a thermocouple 126 attached to one of the beams 104 for measuring the temperature of the support frame 102. The measurements from the temperature sensor 124 and the thermocouple 126 are received by a processor 170, which uses the temperature measurements to compensate for thermal expansion or contraction of the support frame 102 during a calibration process and / or when processing the images 140 for surface reconstruction.

[0044] The reference target 120 may be made of a relatively thin, flexible material, such as a vinyl sheet having an adhesive on one side for adhering the reference target 120 to the frame feet 114. However, the reference target 120 may be made of any material and may be located at any one of a variety of alternative locations on the support frame 102, and is not limited to being mounted to the frame feet 114.

[0045] FIG. 13 shows a common area 312 having an enlarged size that encompasses all the reference targets 120 mounted on the support frame 102. As described above, the processor 170 is configured to analyze the image 140 including the reference targets 120 and determine a scaling factor for the image 140 based on the reference targets 120. Alternatively or additionally, the processor 170 is configured to use the reference targets 120 to determine the three-dimensional locations of the cameras 130 relative to each other and relative to the surface 310. Furthermore, the processor 170 can use the reference targets 120 and the recognized structural features 180 (FIG. 20) to determine the position and orientation of the local reference coordinate system 442 of the inspection system 100 (FIGS. 2-3) relative to the global reference coordinate system 440 of the structure 300 (FIGS. 2-3).

[0046] 17-18, an example of an inspection system 100 is shown having a background blocker 160 for blocking background light (not shown) to avoid adverse effects on the quality of the image 140 produced by the camera 130 that may affect the accuracy of the reconstructed surface 174. Background light may be described as light from an external and / or distant source and may include sunlight, factory shop lights, or any other external source located outside the support frame 102. The background blocker 160 is preferably configured to prevent most, if not all, of the background light from entering the interior of the support frame 102 and / or impinging on the common area 312 of the surface 310 under inspection. The background blocker 160 may be formed of either a rigid or flexible material and may be removably attached or mounted to the support frame 102 using any of a variety of techniques, such as mechanical fasteners (not shown), Velcro™ (not shown), adhesive, or any other attachment method. For example, the background blocker 160 may be a fabric, such as a black polyester fabric, that is attached (eg, glued) to the beams 104 and / or connecting elements 106 of the support frame 102 .

[0047] As an alternative to or in addition to the background blocker 160, the inspection system 100 may include a lens filter 134 attached to the lens of each camera 130, as shown in FIG. 19. The lens filter 134 may be configured to filter out one or more wavelengths of background light, thereby avoiding the adverse effect of the background light on the quality of the image 140. In some examples, the light source 150 may be configured to emit light of at least one wavelength different from the wavelengths filtered out by the lens filter 134, such that the common area 312 is sufficiently illuminated to enable the camera 130 to capture a high quality image 140.

[0048] 20, a block diagram of an example of an inspection system 100 is shown, including at least three light sources 150 (e.g., a first light source 151, a second light source 152, a third light source 153, etc.), at least two cameras 130 (e.g., a first camera 131, a second camera 132, etc.), a processor 170, and a memory 172. The processor 170 and the memory 172 may be part of a computing device that controls the light sources 150 and the cameras 130. For example, the processor 170 and the memory 172 may be integrated into a desktop computer (not shown), a laptop computer 212 (e.g., FIG. 2), a tablet 214 (e.g., FIG. 5), or any other type of computing device. The GUI 210 described above may also be integrated into a computing device that includes the processor 170 and the memory 172.

[0049] The processor 170 is configured to control and sequence the light source 150 in synchronization with the camera 130 to capture multiple images 140 of the surface 310 under inspection. The processor 170 receives the images 140, which may be stored in the memory 172. In the illustrated example, the images 140 include a first image 141, a second image 142, and a third image 143 captured by the first camera 131, and a first image 141, a second image 142, and a third image 143 captured by the second camera 132. Alternatively or additionally, the memory 172 may store multiple common structural features 180 that make up the structure 300 under inspection. The common structural features 180 may include fasteners 182 ( FIG. 22 ), section joints 184 ( FIG. 1 ), window surrounds 188 ( FIG. 2 ), and any one of a variety of other types of structural features 180 that are common components of the structure 300. As described in more detail below, the processor 170 is configured to recognize common structural features 180 in the image 140 of the surface 310 under inspection, and to mask such structural features 180 that may otherwise adversely affect the accuracy of the reconstructed surface 174.

[0050] The processor 170 is also configured to analyze the reconstructed surface 174 for structural features 180, defects 200, and / or texture 206. For example, the processor 170 may analyze for surface defects 200, such as scratches 202 (e.g., FIG. 21), gouges 204 (e.g., FIG. 23), and other types of defects 200, using known defect detection techniques such as robust surface fitting or normal vector tracking. Advantageously, the inspection system 100 enables detection of small surface defects 200 and imperfections that cannot be detected using conventional inspection devices, as described above. The processor 170 may analyze the detected defects 200 and provide characterizations (e.g., measurements), such as size (e.g., defect depth 0.010 inches) and type (e.g., scratch, gouge, etc.) of the defects.

[0051] The processor 170 can also analyze the reconstructed surface 174 for texture 206 (e.g., FIG. 24) using known surface fitting methods or by generating a profile line (not shown) directly on the reconstructed surface 174 followed by measuring a highest point to lowest point plot along the profile line. By analyzing the texture 206 of the surface 310, the inspection system 100 advantageously provides a means to better understand the characteristics (e.g., average surface roughness) of the base surface and any surface coating (e.g., paint) that is subsequently applied to the base surface. Understanding the characteristics of the base surface and / or surface coating can help determine the types of defects 200 that can be found when inspecting the surface 310. For example, defects 200 that have feature sizes smaller than the texture or surface roughness may be undetectable.

[0052] 20, as described above, the processor 170 is configured to control and sequence the light sources 150 in synchronization with the cameras 130, as described above. More specifically, while the inspection system 100 is held stationary in position over the surface, the processor 170 is configured to temporarily activate the light sources 150 one at a time, and each time a light source 150 is activated, cause all of the cameras 130 to capture multiple images 140 of each of the common area 312 of the surface 310. This cycle is repeated until each light source 150 has been activated.

[0053] For example, in an inspection system 100 having exactly three light sources 150 and exactly two cameras 130, the processor 170 activates (i.e., turns on) the first light source 151, commands the first camera 131 and the second camera 132 to capture an image 140 of the common area 312 while the first light source 151 is activated, and then deactivates (i.e., turns off) the first light source 151. After capturing the image 140 with the first camera 131 and the second camera 132 with only the first light source 151 activated, the processor 170 then activates the second light source 152 and commands the first camera 131 and the second camera 132 to capture an image 140 of the common area 312 with only the second light source 152 activated, after which the second light source 152 is deactivated. This cycle is repeated for the third light source 153. The amount of time that each light source 150 is activated (e.g., emitting light) can be relatively short, such as less than a second. Depending on the total number of cameras 130, the total amount of time to cycle through all of the light sources 150 while capturing an image 140 can also be relatively short, such as on the order of seconds.

[0054] The processor 170 receives the images 140 that have been combined and processed by the cameras 130 and indexed by the light sources 150. For example, in an inspection system 100 having exactly two cameras 130 (e.g., a first camera 131 and a second camera 132) and exactly three light sources 150 (e.g., a first light source 151, a second light source 152, and a third light source 153), the first camera 131 combines its first, second, and third images 141, 142, 143, and the second camera 132 combines its first, second, and third images 141, 142, 143. The processor 170 then generates a digital surface based on each camera view. For example, the processor 170 uses a photometric algorithm to generate a first digital surface (not shown) based on the first, second, and third images 141, 142, 143 captured by the first camera 131, and then generates a second digital surface (not shown) based on the first, second, and third images 141, 142, 143 captured by the second camera 132. The processor 170 then aligns the first and second digital surfaces using the camera positions, and then generates a best-fit surface (i.e., the reconstructed surface 174) after removing outliers and noise. The above-described process is one example for processing multiple images 140 with multiple viewpoints to generate the reconstructed surface 174. Any one of a variety of other optimization methods and techniques can be implemented to perform multi-view surface reconstruction to result in the reconstructed surface 174.

[0055] Advantageously, in the inspection system 100 of the present disclosure, by using multiple cameras 130 to capture images 140 of the surface 310 from different viewpoints, the inspection system 100 is able to reduce low frequency shape distortions caused by non-uniform reflectance of light across the surface 310. Photometric stereo algorithms assume that the light source is far away and therefore the light intensity across the surface is uniform. However, in many practical applications, the light source is not very far away. For example, if a light source is illuminating the surface of a flat plate (not shown), the light intensity across the surface will be non-uniform even if the surface is perfectly flat. Advantageously, in the inspection system 100 of the present disclosure, by having two or more viewing directions 136 (i.e., multiple cameras 130), the non-uniform reflectance is averaged out to minimize the effect of shape distortion.

[0056] Alternatively or additionally, non-uniform reflectance can be calibrated to minimize the effect of geometric distortion of the reconstructed surface 174. In this regard, the inspection system (e.g., camera) undergoes a calibration process prior to its use in which the camera 130 captures an image of a known, authenticated artifact (not shown) to ensure that the inspection system 100 (e.g., camera 130) is operating properly. Examples of authenticated artifacts include gauge blocks (e.g., prisms of known dimensions), flat plates etched with a grid of known dimensions, calibration targets, and other known artifacts used for camera calibration. As part of the calibration process, adjustments of the camera's internal and external parameters are made in software operated by the processor 170. The calibration process is performed via a GUI and can be repeated periodically during the life of the inspection system 100 to maintain or improve the accuracy and reliability of the surface reconstruction.

[0057] Advantageously, by minimizing geometric distortion by the multiple cameras 130 of the inspection system 100, variations in surface curvature and material are readily highlighted, revealing raised and / or recessed defects 200, such as scratches 202, gouges 204, and tool marks. Additionally, the texture 206 of the surface 310, as well as localized misalignments, anomalies, and / or cracks in surface coatings (e.g., paint), are readily highlighted.

[0058] FIG. 21 shows an example of a display screen 216 of a GUI 210 displaying images 140 captured by an inspection system 100 having exactly three light sources 150 and exactly two cameras 130. The display screen 216 shows three images 140 captured by a first camera 131 from its unique viewing direction 136 under three different lighting conditions, each generated by the three light sources 150. In addition, the display screen 216 shows three images 140 captured by a second camera 132 from its unique viewing direction 136 under the same three lighting conditions. Each image 140 includes a surface defect 200 in the form of a scratch 202. Due to the difference in the lighting direction 156 under which each image 140 is captured, there is a variation in the extent to which the scratch 202 appears in the images 140. For example, only a portion of the scratch 202 may appear in one image 140, while another portion of the same scratch 202 may appear in one or more of the other images 140. The processor 170 combines the images 140 so that the reconstructed surface 174 represents with a high level of accuracy the topography of the common areas 312 of the surface 310. In addition, the processor 170 detects small surface discontinuities, such as scratches 202, with high accuracy. The right side of the display screen 216 in Figure 21 shows the reconstructed surface 174 that fully includes each of the scratches 202.

[0059] As described above, the processor 170 is configured to analyze the reconstructed surface 174 for structural features 180, defects 200, and / or texture 206. As described above, the processor 170 first recognizes any common structural features 180 appearing in the image 140 of the surface 310 under inspection. For example, the processor 170 recognizes the common structural features 180 by comparing the image 140 to the structural features 180 stored in the memory 172. If the image 140 includes common structural features 180, the processor 170 masks such structural features 180 that may otherwise adversely affect the reliability and accuracy of the reconstruction process of the surface 310 due to reflections, shadows, and / or other effects generated by the structural features 180.

[0060] To illustrate the masking of common structural features 180, FIG. 22 shows an example of an inspection system 100 configured to be manually held in place against a surface 310 of a structure 300 having a common structural feature 180 in the form of a fastener 182 that secures a skin panel 186 to an underlying structural component (not shown). FIG. 23 shows a display screen 216 displaying multiple images 140 of the inspected surface 310 of FIG. 22. Each image 140 includes a defect 200 in the surface 310 in the form of a gouge 204 in one of the skin panels 186. To the right of the display screen 216 is a reconstructed surface 174 generated from the images 140. The processor 170 has masked out the common structural features 180 (i.e., the fastener 182), thereby improving the accuracy of the reconstructed surface 174 and also having the effect of highlighting the gouge 204 in the skin panel 186. The gouges 204 are detected using an appropriate defect detection technique, such as robust surface fitting or normal vector tracking, as described below, or another defect detection technique. The processor 170 may also plot and / or highlight the location of the defects 200 in the surface 310.

[0061] FIG. 24 shows an example of a display screen 216 of a GUI 210 displaying images 140 captured by an inspection system 100 having exactly three light sources 150 and exactly two cameras 130. Similar to the arrangements of FIGS. 21 and 23 described above, the display screen 216 of FIG. 24 shows three images 140 captured by a first camera 131 from its unique viewing direction 136 under three different lighting conditions from the three light sources 150, and three images 140 captured by a second camera 132 from its unique viewing direction 136 under the same three lighting conditions. Due to the differences in the lighting directions 156, there is variation in the extent to which the texture 206 appears in the images 140. The processor 170 combines the images 140 such that the reconstructed surface 174 accurately represents the overall texture 206 of the common area 312 of the surface 310. As described above, the processor 170 can also analyze the reconstructed surface 174 by using known surface fitting methods or by generating a profile line (not shown) on the reconstructed surface and measuring the distance from the highest point to the lowest point at different points along the profile line, thereby characterizing the texture 206 (e.g., in terms of surface roughness).

[0062] 25, there is shown a flowchart of operations involved in a method 500 for capturing a topography of a surface 310 under inspection. As described in more detail below, the topography is represented by a reconstructed surface 174 (e.g., a three-dimensional digital model). The method 500 is performed using the inspection system 100 described above, which includes a rigid support frame 102, at least three light sources 150, at least two cameras 130, and a processor 170.

[0063] Method 500 includes calibrating inspection system 100 prior to first use. For example, method 500 includes calibrating camera 130 using an authenticated artifact (not shown) to determine parameters (e.g., intrinsic and / or extrinsic) of camera 130 and verify that inspection system 100 is operating properly. Examples of authenticated artifacts include gauge blocks, flat plates etched with grids, calibration targets, and other known artifacts. Using known techniques, the camera parameters are determined and adjustments are made to algorithms to account for the camera parameters.

[0064] The calibration process also includes performing a scaling verification of the images 140 and / or calculating the three-dimensional position of the cameras 130 using photogrammetry techniques. With reference to FIGS. 13-14 , the method 500 includes mounting a reference target 120 on the bottom end 112 of the support frame and within the field of view of the cameras. After capturing the images 140, each including at least three common reference targets 120, the calibration process includes analyzing the images 140 using the processor 170 to determine a scaling factor of the images 140 and / or the three-dimensional location of the cameras 130 relative to each other based on the reference targets 120. Additionally, the calibration process may include calculating an orientation of the local coordinate system 442 ( FIG. 2 ) of the inspection system 100 relative to the global coordinate system 440 ( FIG. 2 ) of the structure 300 using the reference targets 120 and the recognized structural features 180 using photogrammetry techniques. In some examples, the method 500 may also include steps of measuring the temperature of the ambient air and / or the support frame 102 before and / or during inspection, and compensating for thermal expansion or contraction of the support frame 102 during a calibration process and / or when processing the images 140 for surface reconstruction and / or defect detection.

[0065] Step 502 of the method 500 includes positioning the support frame 102 over the surface 310 to be inspected. As discussed above, the light sources 150 are mounted at different locations on the support frame 102. The cameras 130 each have a direction of view 136 and are mounted at different locations on the support frame 102 such that the direction of view 136 of each camera 130 generally faces a common area 312 of the surface 310, as shown in the example of FIG. 4. In some examples, the method 500 includes mounting the light sources 150 and / or cameras 130 on the support frame 102 such that the light sources 150 and / or cameras 130 are equiangularly spaced from one another when the inspection system 100 is viewed from above, as shown in the example of FIG. 10. However, as discussed above, the light sources 150 and / or cameras 130 can be mounted at any location on the support frame 102 and are not limited to being equiangularly spaced from one another.

[0066] In some examples, positioning 502 the support frame 102 (i.e., the inspection system 100) on the surface 310 includes positioning the support frame 102 at one or more different locations on the surface 310 via a movable platform 400 to which the support frame 102 is attached. For example, FIGS. 2-3 show the support frame 102 attached to the end of a robotic arm 404 of a robotic device 402. As described above, the robotic device 402 is movable along a robotic device track 406 to position the inspection system 100 at any location along the length of the structure 300 under inspection. In other examples not shown, positioning 502 the support frame 102 on the surface 310 can be performed using any of a variety of alternative movable platform configurations, such as an overhead gantry (not shown). In further examples, positioning 502 the support frame 102 on the surface 310 can include manually holding the support frame 102 in a fixed position on the surface 310 under inspection, as shown in the above-described example of FIG. 5.

[0067] 7, the method 500 includes orienting the light sources 150 such that the illumination direction 156 of each light source 150 forms an incidence angle 158 of 15-75 degrees with the common area 312. Preferably, the illumination direction 156 forms an incidence angle 158 of 15-45 degrees, thereby maximizing shadow effects on the surface 310 under inspection. The method 500 may also include orienting or adjusting the camera orientation such that the viewing direction 136 of each camera 130 is locally non-perpendicular to the surface 310 under inspection, thereby enhancing the ability of each camera 130 to capture shadows generated by irregularities or discontinuities in the surface 310.

[0068] The method 500 may further include changing the size and / or shape of the support frame 102 by changing the configuration of the beams 104 interconnected by the connecting elements 106, thereby accommodating surfaces 310 of different sizes and / or shapes. In this regard, the size and / or shape of the support frame 102 may be changed by inserting the beam ends into different openings 116 in the connecting elements 106 and / or by changing the length of the beams 104. FIGS. 7, 12, and 13 show three different versions of a wide variety of configurations of the support frame 102 that may be achieved by changing the length of the beams 104 and / or by changing the way the beams 104 are interconnected via the connecting elements 106.

[0069] Step 504 of method 500 includes temporarily activating the light sources 150 one at a time and having all of the cameras 130 capture multiple images 140 of each of the common areas 312 each time a light source 150 is activated. For example, in an inspection system 100 having exactly three light sources 150 and exactly two cameras 130, the process of performing step 504 includes activating only the first light source 151 and capturing an image 140 of the common area 312 using the first and second cameras 131, 132, activating only the second light source 152 and again capturing an image of the common area 312 using the first and second cameras 131, 132, and then activating only the third light source 153 and again capturing an image of the common area 312 using the first and second cameras 131, 132. As described above, the images 140 are transmitted to the processor 170 for processing to generate a reconstructed surface 174 of the surface 310 under inspection.

[0070] In some examples, the method 500 includes using a background blocker 160 that can be attached to the support frame 102 to at least partially block background light from impinging on the common area 312 of the surface 310. For example, as shown in FIGS. 17-18 described above, the method 500 can include attaching a light blocking fabric (e.g., black polyester fabric) to the beams 104 and / or the connecting elements 106 of the support frame 102. Alternatively, the background blocker 160 can be a rigid, one-piece shell-like cover (not shown) formed of a lightweight metallic material (e.g., aluminum) and / or a lightweight polymeric material (e.g., carbon fiber composite) that can be removably attached to the support frame 102. Instead of or in addition to using the background blocker 160 to block background light, the method 500 can include using a lens filter 134 attached to the lens of each camera 130 to filter out one or more wavelengths of background light. When using such a lens filter 134 , the method 500 may include emitting light from the light source 150 at at least one wavelength different from the wavelengths filtered out by the lens filter 134 .

[0071] Step 506 of method 500 includes processing images 140 to result in a reconstructed surface 174 that represents the topography of common area 312. As described above, processor 170 uses a photometric stereo algorithm to form reconstructed surface 174 based on images 140. In one example of processing images, step 506 includes receiving images 140 at processor 170, which combines and processes images 140 by camera 130 and indexes images 140 by light source 150. For example, in the inspection system 100 described above having exactly two cameras 130 (e.g., a first camera 131 and a second camera 132) and exactly three light sources 150 (e.g., a first light source 151, a second light source 152, and a third light source 153), step 506 includes combining the first, second and third images 141, 142, 143 of the first camera 131 and combining the first, second and third images 141, 142, 143 of the second camera 132.

[0072] Next, step 506 includes using a photometric algorithm to generate a first digital surface (not shown) based on the first, second, and third images 141, 142, 143 captured by the first camera 131, and to generate a second digital surface (not shown) based on the first, second, and third images 141, 142, 143 captured by the second camera 132. Step 506 then includes aligning the first and second digital surfaces using the camera positions, and generating a best-fit surface (i.e., the reconstructed surface 174) after removing outliers and noise. Step 506 of processing the images 140 and generating the reconstructed surface 174 can be performed using any one of a variety of alternative optimization techniques for multi-view surface reconstruction to result in the reconstructed surface 174, and is not limited to the process described above. As described above, by combining multiple images 140 taken from different viewpoints, the non-uniform reflectance of light across the surface 310 under inspection is averaged out, which would otherwise cause shape distortions in the reconstructed surface 174.

[0073] 26, a method 600 for analyzing a reconstructed surface 174 for structural features 180, defects 200, and / or texture 206 is shown. Step 602 of the method 600 involves recognizing common structural features 180 in the images 140 of the surface 310 under inspection and masking the structural features 180, if present. As discussed above, such structural features 180 may adversely affect the reliability and accuracy of the reconstructed surface 174. FIG. 23 shows a display screen 216 of the GUI 210 displaying images 140 of the surface 310 under inspection. Each of the images 140 has a plurality of structural features 180 in the form of fasteners 182. However, the fasteners 182 have been masked and are therefore not present in the reconstructed surface 174 shown on the right side of FIG. 23.

[0074] Step 604 of method 600 includes detecting defects 200 using a suitable defect detection technique. In one example, step 604 includes detecting defects 200 via robust surface fitting, which first includes importing the data (e.g., reconstructed surface 174 and corresponding image 140) into an algorithm, generating a robust polynomial surface fit of the data, and identifying the most deviant points of the polynomial surface fit. Points above a predetermined threshold are removed to obtain a new robust polynomial surface fit. Step 604 then includes generating a heat map (not shown) of the data based on the new robust polynomial surface fit, and identifying defects 200 based on the deviant points in the heat map. In another example, step 604 includes detecting defects 200 via normal vector tracking, which includes extracting the surface normal at each vertex of the reconstructed surface, calculating the average angle between each surface normal and its neighbors, identifying points that have an angular variation greater than a predetermined maximum value, and identifying defects 200 based on the points with the greatest angular variation between the surface normal and their neighbors.

[0075] Advantageously, the above-noted defect detection method 600 maintains and / or generates a relationship between two-dimensional data (i.e., image 140) and three-dimensional data (i.e., reconstructed surface 174) that facilitates the defect detection process. Advantageously, method 600 uses two-dimensional data (e.g., image 140) to locate structural features in a manner that is less computationally intensive than using three-dimensional data (i.e., reconstructed surface 174). Additionally, method 600 performs defect detection and measurement using the three-dimensional data (i.e., reconstructed surface 174).

[0076] Regardless of the defect detection technique used, the method 600 includes a step 606 of plotting and / or highlighting the locations of the defects 200 in the reconstructed surface 174. In this regard, step 606 includes classifying the defect type, which may be performed using machine learning or any other suitable defect classification technique. Additionally, step 606 may include quantifying parameters of each defect 200 by analyzing the reconstructed surface 174 once the locations of the defects 200 have been determined using the defect detection algorithm. For example, step 606 may include classifying the defects as scratches 202 of FIG. 21 and quantifying scratch parameters with respect to a maximum depth and length of each scratch 202 (e.g., maximum depth 0.005 inches, length 2.3 inches, etc.). 23, step 606 may include classifying defect 200 as gouge 204 in FIG. 23 and quantifying the gouge parameters in terms of its maximum width and depth, and its length (e.g., maximum width 0.050 inches, maximum depth 0.020 inches, length 3 inches, etc.). Although not shown, step 606 may further include annotating the reconstructed surface 174 with the defect type (e.g., gouge, scratch, etc.), the defect parameters (e.g., maximum width, depth and length), and the x, y, z location of the defect relative to the original in the global coordinate system.

[0077] In addition to detecting defects 200 in the reconstructed surface 174, the method 500 may also include analyzing the reconstructed surface 174 for texture 206 (e.g., FIG. 24). For example, the method 500 may use known surface fitting methods or generate profile lines (not shown) on the reconstructed surface 174 to analyze the texture 206 of the surface 310. The texture 206 may be characterized based on the distance between the highest point and the adjacent lowest point in a plot of the profile lines. Analysis of the texture 206 of the surface 310 may provide a means to better understand the characteristics (e.g., average surface roughness) of the base surface (not shown) and any surface coating (e.g., paint - not shown) that is subsequently applied to the base surface, and may help determine the size of the defects 200 that are detectable when inspecting the surface 310.

[0078] In some examples, the method 500 includes displaying the image 140 and / or the reconstructed surface 174 on a graphic user interface (GUI 210). For example, FIG. 2 shows the GUI 210 of a laptop computer 212 displaying the image 140 captured by the camera 130 in real time while the robotic device 402 holds the inspection system 100 in place over the surface 310 under inspection. In another example, FIG. 5 shows a technician holding the inspection system 100 in place over the surface 174 while viewing the image 140 and the reconstructed surface 310 in real time on the GUI 210 attached to the support frame 102. FIG. 21 shows the display screen 216 of FIG. 5 displaying the image 140 and the reconstructed surface 174, showing a defect 200 in the form of a scratch 202. FIG. 23 is an example of a display screen 216 displaying the image 140 and the reconstructed surface 174 including a defect 200 (e.g., gouge 204) in one of the skin panels 186 under inspection of FIG. 22.

[0079] Advantageously, combining multiple images 140 taken from different perspectives reduces geometric distortion of the reconstructed surface 174, thereby enabling a high level of accuracy in detecting small defects 200, such as scratches 202 (e.g., FIG. 21), gouges 204 (e.g., FIG. 23), and other types of discontinuities. Furthermore, the high accuracy of the reconstructed surface 174 allows for analysis of small surface features, such as texture 206, to allow for the determination of the type and / or size of defects 200 that may be detected. In addition to analyzing the surface for small defects, the inspection system 100 also provides the ability to detect relatively large defects 200 (e.g., dents) in the surface 310.

[0080] Many modifications and other versions and examples of the present disclosure will occur to those skilled in the art having the benefit of the teachings presented in the foregoing description and the associated drawings. The versions and examples described herein are intended to be illustrative and not limiting or exhaustive. Although specific terms have been employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation. In addition to those enumerated herein, functionally equivalent methods and apparatuses within the scope of the present disclosure are possible from the foregoing description. Such modifications and variations are intended to be included within the scope of the appended claims. The present disclosure is to be limited only by the terms of the appended claims, along with the full scope of equivalents to which such claims are entitled. [Explanation of symbols]

[0081] 100 Inspection Systems 102 Rigid support frame 104 Inclined Beam 106 Connection Elements 108 Top 110 Handle 112 Bottom end 114 Frame foot 116 Window Opening 118 Scale bar 119 Scaling Distance 120 Reference Target 122 Coded Target 124 Temperature Sensor 126 Thermocouple 130 Overhead Camera 131 First Camera 132 Second Camera 134 Lens Filter 136 View Direction 140 images 141 First Image 142 Second Image 143 Third Image 150 light source 151 First Light Source 152 Second Light Source 153 The Third Light Source 154 Directional light source 156 Lighting Direction 158 Angle of incidence 160 Background Shading 170 Processors 172 Memory 174 Surface 180 Structural shape 182 Fasteners 184 Section Joint 186 Skin Panel 188 Window Surroundings 200 Surface defects, concave defects 202 Scratch 204 Gouge 206 Textures 212 Laptop Computer 214 tablets 216 Display screen 300 Structure 302 Aircraft 304 Torso 306 Torso Section 308 Assembly Fixtures 310 Inspected surface, outer surface 312 Common Area 314 Wings 316 caudal face 318 Passenger window 400 Moving Platform 402 Robot Device 404 Robot Arm 406 Robot Device Truck 440 Global Reference Frame 442 Local Reference Frame

Claims

1. 1. An inspection system (100) for capturing a topography of a surface (310), comprising: a rigid support frame (102) configured to be positioned over the surface (310); at least three light sources (150) configured to be mounted at different locations on the support frame (102); at least two cameras (130), each having a line of sight (136), configured to be mounted at different locations on the support frame (102) such that the line of sight (136) of each camera (130) generally points toward a common area (312) of the surface (310); A processor (170), temporarily activating the light sources (150) one at a time and causing all of the cameras (130) to capture a plurality of images (140) of each of the common areas (312) each time a light source (150) is activated; and combining the images (140) to obtain a reconstructed surface (174) representative of the topography of the common area (312); A processor (170) configured to An inspection system (100) comprising:

2. 2. The inspection system (100) of claim 1, wherein the processor (170) is configured to analyze the reconstructed surface (174) for structural features (180), defects (200), and / or texture (206).

3. The inspection system (100) of claim 1, further comprising a graphical user interface (GUI (210)) configured to display the reconstructed surface (174).

4. 4. The inspection system (100) of claim 3, wherein the GUI (210) is mountable on the support frame (102) to enable viewing of the reconstructed surface (174) while the support frame (102) is held in place on the surface (310).

5. 2. The inspection system (100) of claim 1, wherein at least one of the light sources (150) is a directional light source (154) mountable to the support frame (102) such that the illumination direction (156) forms an angle of incidence (158) of between 15 and 75 degrees with the common area (312).

6. The inspection system (100) of claim 1, wherein the light sources (150) and / or the cameras (130) are positioned at equal angular intervals from one another when viewed from above the inspection system (100).

7. 2. The inspection system (100) of claim 1, wherein the support frame (102) is comprised of beams (104) interconnected by connecting elements (106) configured to allow the size and / or shape of the support frame (102) to be altered to accommodate surfaces (310) of different sizes and / or shapes.

8. one or more reference targets (120) attached to a bottom end (112) of the support frame (102); each of the cameras (130) is oriented to include the reference target (120) in the image (140); 2. The inspection system (100) of claim 1, wherein the processor (170) is configured to analyze the image (140) including the reference target (120) and use the reference target (120) to determine a scaling factor for the image (140) and a three-dimensional location of the cameras (130) relative to one another.

9. a background block (160) mountable to the support frame (102) and configured to at least partially block background light from impinging on the common area (312) of the surface (310); a lens filter (134) attachable to each of the cameras (130) and configured to filter out one or more wavelengths of the background light, the light source (150) being configured to emit light of at least one wavelength different from the wavelengths filtered out by the lens filter (134); The inspection system (100) of claim 1, further comprising at least one of:

10. 1. An inspection system (100) for capturing a topography of a surface (310), comprising: A rigid support frame (102); a movable platform (400) configured to position the support frame (102) at one or more different locations on the surface (310); at least three light sources (150) configured to be mounted at different locations on the support frame (102); at least two cameras (130), each having a line of sight (136), configured to be mounted at different locations on the support frame (102) such that the line of sight (136) of each camera (130) generally points toward a common area (312) of the surface (310); A processor (170), temporarily activating the light sources (150) one at a time and causing all of the cameras (130) to capture a plurality of images (140) of each of the common areas (312) each time a light source (150) is activated; combining the images (140) to correct for geometric distortions and generate a reconstructed surface (174) representative of the topography of the common area (312); A processor (170) configured to: An inspection system (100) comprising:

11. 1. A method for capturing a topography of a surface (310), comprising: positioning a rigid support frame (102) on the surface (310), the support frame (102) having at least three light sources (150) mounted at different locations on the support frame (102) and at least two cameras (130), each having a direction of view (136), the at least two cameras (130) mounted at different locations on the support frame (102) such that the direction of view (136) of each camera (130) generally points toward a common area (312) of the surface (310); temporarily activating the light sources (150) one at a time and causing all of the cameras (130) to capture a plurality of images (140) of each of the common areas (312) each time a light source (150) is activated; combining the images (140) to obtain a reconstructed surface (174) representative of the topography of the common area (312); A method comprising:

12. The method of claim 11 , further comprising analyzing the reconstructed surface (174) for structural features (180), defects (200), and / or texture (206).

13. The method of claim 12, further comprising displaying the reconstructed surface (174) on a graphic user interface (GUI (210)).

14. 12. The method of claim 11, wherein displaying the reconstructed surface (174) on the GUI (210) comprises viewing the reconstructed surface (174) on the GUI (210) mounted on the support frame (102).

15. 12. The method of claim 11, further comprising orienting at least one of the light sources (150) such that an illumination direction (156) of the light source (150) forms an angle of incidence (158) of 15 to 75 degrees with the common area (312).

16. 12. The method of claim 11, comprising mounting the light sources (150) and / or the cameras (130) to the support frame (102) such that the light sources (150) and / or the cameras (130) are equiangularly spaced from one another when viewed from above the support frame (102).

17. 12. The method of claim 11, comprising changing the size and / or shape of the support frame (102) by changing an arrangement of a plurality of beams (104) interconnected by a plurality of connecting elements (106), thereby accommodating a surface (310) of different size and / or shape.

18. mounting one or more reference targets (120) adjacent a bottom end (112) of the support frame (102); capturing an image (140) of the reference target (120) using the camera (130); analyzing the image (140) including the reference target (120) using a processor (170) to determine a scaling factor for the image (140) and a three-dimensional location of the cameras (130) relative to one another based on the reference target (120); The method of claim 11 further comprising:

19. at least partially blocking background light from impinging on the common area (312) of the surface (310) using a background block (160) attached to the support frame (102); filtering out one or more wavelengths of the background light using a lens filter (134) attached to each of the cameras (130) and emitting light from the light source (150) at at least one wavelength different from the wavelengths filtered out by the lens filter (134); The method of claim 11 , further comprising at least one of:

20. Positioning the support frame (102) on the surface (310) comprises: The method of claim 11, comprising positioning the support frame (102) at one or more different locations on the surface (310) via a movable platform (400) to which the support frame (102) is attached.

Images