Inspection system and method

Abstract

The invention relates to an inspection system and method. An inspection system (100) for capturing a topography of a surface (310) includes a support frame (102) configured to be positioned above the surface (310), at least three light sources (150) configured to be mounted at different orientations on the support frame (102), and at least two cameras (130), each camera has a viewing direction (136) and is configured to be mounted at a different orientation on the support frame (102) in such a manner that the viewing direction (136) of each camera (130) is generally directed towards a common area (312) of the surface (310). The inspection system (100) further comprises a processor (170) configured to temporarily activate one light source (150) at a time and to cause all cameras (130) to capture a respective plurality of images (140) of the common area (312) at each activation of the light source (150) and to combine the images (140) in a manner that produces a reconstructed surface (174) representing a topography of the common area (312).

Description

Technical Field

[0001] The present disclosure relates generally to inspection systems and, more particularly, to systems and methods for inspecting surface defects. Background Art

[0002] Detecting and measuring surface defects on production parts can be challenging. Part of the challenge in detecting surface defects is their small size, and some of them can only be detected under specific lighting conditions or through tactile methods such as running a fingernail over the surface. Detecting surface defects early in the production process is important because it prevents faulty parts from moving down the production line and potentially reaching the end customer. Additionally, early detection of surface defects allows for rework or replacement of defective parts and adjustments to the manufacturing process to prevent future surface defects.

[0003] One traditional method of inspecting surfaces involves the use of structured light scanners. While structured light scanners are effective at detecting large surface defects such as dents, they lack the accuracy required to detect small surface defects such as scratches, gouges, and tool marks due to the inherent low-frequency noise (e.g., irregularities) in the surface. Laser profilometers are another device traditionally used to inspect surfaces. While laser profilometers are generally effective for their intended purpose, they have a relatively small field of view. In addition, laser profilometers suffer from reflection issues that affect their ability to inspect certain types of surfaces.

[0004] It can be seen that there is a need in the art for a system for inspecting surfaces that is capable of detecting small defects on a variety of different types of surfaces and that has a relatively large field of view. Summary of the invention

[0005] The present invention solves the above-mentioned needs related to inspecting a surface and provides an inspection system for capturing the topography of a surface. The inspection system includes a rigid support frame configured to be positioned above the surface, at least three light sources configured to be mounted on the support frame at different orientations, and at least two cameras, each camera having a viewing direction and configured to be mounted on the support frame at different orientations in a manner such that the viewing direction of each camera is generally directed toward a common area of ​​the surface. The inspection system also includes a processor configured to temporarily activate one light source at a time and cause all cameras to capture a corresponding plurality of images of the common area each time the light source is activated, and to combine the images in a manner to produce a reconstructed surface representing the topography of the common area.

[0006] In another example, an inspection system includes a rigid support frame and a movable platform configured to position the support frame at one or more different locations on a surface. The inspection system also includes at least three light sources configured to be mounted at different locations on the support frame, and at least two cameras, each camera having a viewing direction and configured to be mounted at different locations on the support frame in a manner such that the viewing direction of each camera generally points to a common area of ​​the surface. In addition, the inspection system also includes a processor configured to temporarily activate one light source at a time and cause all cameras to capture corresponding multiple images of the common area each time the light source is activated, and to combine the images in a manner that corrects shape distortion, thereby obtaining a reconstructed surface representing the topography of the common area.

[0007] A method of capturing the topography of a surface is also disclosed. The method includes positioning a rigid support frame above the surface. The support frame has at least three light sources mounted at different locations on the support frame, and at least two cameras, each camera having a viewing direction and mounted at different locations on the support frame in a manner such that the viewing direction of each camera is generally directed toward a common area of ​​the surface. The method also includes temporarily activating one light source at a time, and causing all cameras to capture a corresponding plurality of images of the common area each time a light source is activated. Furthermore, the method includes combining the images in a manner that produces a reconstructed surface representing the topography of the common area.

[0008] The features, functions, and advantages that have been discussed can be achieved independently in various versions of the present disclosure or may be combined in yet other versions, further details of which can be seen with reference to the following description and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The present disclosure may be better understood by reference to the following detailed description taken in conjunction with the accompanying drawings, which show preferred and exemplary versions, but are not necessarily drawn to scale. The drawings are examples and are not meant to be limiting of the specification or claims.

[0010] Figure 1 An example of an aircraft is shown;

[0011] Figure 2 Shows Figure 1 An example of a barrel section of a fuselage of an aircraft in FIG. 1 is provided, and a robotic device supporting an inspection system of the present disclosure for inspecting a surface of a structure such as a barrel section is described;

[0012] Figure 3 During the inspection by the inspection system Figure 2 an end view of a barrel section;

[0013] Figure 4is an enlarged view of an example of an inspection system positioned against a surface of a barrel section and showing inspection system components including a rigid support frame, a plurality of light sources supported on the support frame, and a plurality of cameras supported on the support frame, the cameras being configured to capture a plurality of images of a common area of ​​the surface during sequential activation of the light sources;

[0014] Figure 5 An example of an inspection system is shown that is configured to be manually held in place against a surface being inspected and has a graphical user interface (GUI, such as a tablet computer) mounted to a support frame for displaying a reconstructed surface generated by the inspection system by combining multiple images captured by a camera;

[0015] Figure 6 yes Figure 5 A side view of an inspection system positioned against a surface of a barrel section;

[0016] Figure 7 is a side view of an inspection system showing a camera having a viewing direction forming an angle of incidence with a surface being inspected, and further illustrating a light source having an illumination direction forming an angle of incidence with the surface;

[0017] Figure 8 An example of a camera mounted on one of the beams of a support frame is shown;

[0018] Fig. 9 An example of a light source mounted on a beam of a support frame is shown;

[0019] Fig.10 yes Figure 7 A top view of an inspection system having three light sources and a total of three cameras mounted on a support frame;

[0020] Fig.11 is an exploded view of a portion of a support frame showing a plurality of beams separated from a connecting element;

[0021] Fig.12 shows an example of an inspection system wherein vertically oriented beams are omitted from the support frame, and further illustrates a total of nine light sources and a total of three cameras mounted on the support frame;

[0022] Fig.13 shows an example of an inspection system wherein the diagonal beam is omitted from the support frame, and further illustrates a reference target mounted on the bottom end of the support frame;

[0023] Fig.14 is an enlarged view of a portion of the support frame showing the reference target mounted on one of the beams;

[0024] Fig.15 yes Fig.14 A magnified image of a reference target;

[0025] Fig.16 An example of a reference target configured as an encoding target is shown;

[0026] Fig.17 is a partially exploded view of an example of an inspection system having a background light shield;

[0027] Fig.18 An inspection system showing a background light shield being assembled to a support frame;

[0028] Fig.19 An example of a camera having a lens filter configured to filter out one or more wavelengths of background light is shown;

[0029] Fig. 20 is a block diagram of an example of an inspection system having a processor for controlling operation of a camera and a light source and generating a reconstructed surface based on an image captured by the camera;

[0030] Fig.21 An example of a display screen showing a GUI that displays a reconstructed surface of an inspected surface based on an image captured by a camera and that displays defects in the reconstructed surface in the form of scratches;

[0031] Fig. 22 An example of an inspection system positioned against a structural surface having a common structural feature in the form of a fastener is shown;

[0032] Fig.23 shows an example of a display screen showing a reconstructed surface, where common structural features in the image have been masked out, and where defects in the reconstructed surface are shown in the form of gouges;

[0033] Fig.24 An example of a display screen showing a reconstructed surface showing the texture of the inspected surface is shown;

[0034] Fig.25 is a flow chart of operations involved in a method of capturing the topography of a surface under inspection; and

[0035] Fig.26 is a flow chart of operations included in a method of detecting defects in a reconstructed surface generated by the inspection system of the present disclosure.

[0036] The figures shown in this disclosure represent various aspects of the presented versions, and only the differences will be discussed in detail. DETAILED DESCRIPTION

[0037] The disclosed versions will now be described more fully below with reference to the accompanying drawings, in which some, but not all, of the disclosed versions are shown. In fact, several different versions may be provided, and should not be construed as limited to the versions described herein. Rather, these versions are provided to make the disclosure thorough and fully convey the scope of the disclosure to those skilled in the art.

[0038] This specification includes references to "a version" or "a version". Instances of the phrase "a version" or "a version" do not necessarily refer to the same version. Similarly, this specification includes references to "an example" or "an example". Instances of the phrase "an example" or "an example" do not necessarily refer to the same example. The particular features, structures, or characteristics may be combined in any suitable manner consistent with the present disclosure.

[0039] As used herein, "comprising" is an open term and does not exclude additional structures or steps as described in the claims.

[0040] As used herein, "configured to" means that various parts or components may be described or claimed as being "configured to" perform one or more tasks. In this context, "configured to" is used to characterize a structure by indicating that the part or component includes a structure that performs those one or more tasks during operation. Thus, even if the specified parts or components are not currently operating (e.g., not turned on), they can be said to be configured to perform tasks.

[0041] As used herein, an element or step referenced in the singular and preceded by the word "a" or "an" should be understood to not necessarily exclude a plurality of elements or steps. As used herein, the term "and / or" includes any and all combinations of one or more associated listed items. As used herein, the term "combinations thereof" includes combinations having at least one associated listed item, wherein the combination may also include other similar unlisted items.

[0042] As used herein, when used with a list of items, the phrase "at least one of" means that different combinations of one or more of the listed items may be used, and only one of each item in the list may be required. In other words, "at least one of" refers to any combination of items and quantities of items that can be used in the list, but not all items in the list are required. The item may be a specific object, thing, or category.

[0043] In the present disclosure, the following phrases may be used interchangeably: "surface of a structure", "surface of a barrel section", "surface of a skin panel", "surface to be inspected", "surface to be inspected", "inspected surface", and "surface".

[0044] Referring now to the accompanying drawings which illustrate various examples of the present disclosure, Figure 1 Shown in FIG. 3 is an aircraft 302 that serves as an example of a structure 300 in which the inspection system 100 of the present disclosure may be used. Figure 2 ) and Method 500( Fig.25 ) is used to capture the topography of the surface 310 (i.e., the surface to be inspected) and inspect the defects 200 of the surface 310 ( Fig.21 , Fig.23 and Fig.24 ). Aircraft 302 includes a fuselage 304, a pair of wings 314, and an empennage, the empennage including tail surfaces 316, such as horizontal elevators and vertical stabilizers. In the example shown, fuselage 304 is constructed by separately manufacturing barrel sections 306, which are then joined together at corresponding plurality of section interfaces / joints 184, such as Figure 1 shown.

[0045] refer to Figure 2-4 , the inspection system 100 includes a rigid support frame 102 (e.g. Figure 4 ), at least three light sources 150 ( Figure 4 ), at least two cameras 130 ( Figure 4 ) and processor 170( Figure 2 and Fig. 20 As described in more detail below, processor 170 sequentially activates light source 150 in synchronization with camera 130 to capture multiple images 140 of surface 310 from different viewing angles (e.g., Fig.21 Processor 170 performs surface reconstruction based on image 140 using photometric stereo techniques to form a reconstructed surface 174 (eg, Figure 21-3 In addition, the processor 170 analyzes the reconstructed surface 174 for flaws and defects 200, as described in more detail below.

[0046] Advantageously, multiple images 140 taken from different viewing angles allow for a relatively large field of view of the inspected surface 310. In addition, multiple images 140 reduce low-frequency shape distortions in the reconstructed surface 174 and allow for a high level of accuracy in detecting small surface features, such as texture 206 ( Fig.24 ) and defects, including scratches 202 (e.g. Fig.21 ), chisel marks 204 (e.g. Fig.23), tool marks (not shown), and other types of discontinuities that are otherwise only detectable under certain lighting conditions or by touching the surface 310 with a finger or fingernail. Advantageously, the inspection system 100 is a relatively small, lightweight, low-cost device that operates in a manner that allows for rapid detection of surface defects and flaws that are not detectable using conventional inspection equipment.

[0047] Figure 2-3 An example of an inspection system 100 of the present disclosure is shown positioned against a barrel section 306 in order to capture surface topography and / or inspect an outer surface 310 of the barrel section 306. In the example shown, the barrel section 306 has a generally cylindrical shape with rows of passenger windows 318 on opposite sides of the barrel section 306. Each window 318 has a window surround 188 that provides local strength reinforcement around the window 116 in the barrel section 306. In some examples, the barrel section 306 is of metal construction and the outer surface 310 is defined by the skin panel 186. However, in other examples, the barrel section 306 may be at least partially formed of a composite material.

[0048] Figure 3 The inspection system 100 is shown implemented on a barrel section 306 (e.g., structure 300) statically supported on an assembly fixture 308 within a manufacturing facility. In other examples, not shown, the inspection system 100 may be implemented to capture surface topography and / or inspect a surface 310 of the structure 300 in a field operation. For example, the inspection system 100 may be implemented to capture surface topography and / or inspect a surface 310 of a fuselage 304, a wing 314, or other component of an in-service aircraft 302 during one of the regularly scheduled inspections at a maintenance facility. Although described in the context of a barrel section 306 of a fuselage 304, the inspection system 100 and method 500 of the present disclosure may be implemented to inspect any of a variety of different types of structures of any size, shape, configuration, and material composition, without limitation. Furthermore, in this regard, the inspection system 100 and method 500 of the present disclosure may be implemented for inspecting both planar and non-planar surfaces, including contoured surfaces and complex curved surfaces.

[0049] refer to Figure 2-14, several examples of inspection systems 100 including the support frame 102 described above are shown. The support frame 102 has a top end 108 and a bottom end 112. The top end 108 of the support frame 102 is coupled to a movable platform 400. The bottom end 112 of the support frame 102 is configured to be positioned in close proximity to the surface 310 being inspected. The movable platform 400 is configured to move the inspection system 100 to any one or more positions along the surface 310 being inspected, and to hold the inspection system 200 stationary in each position during activation of the light source 150 and the camera 130 as described below.

[0050] exist Figure 2-3 In the example of , the movable platform 400 is a robotic device 402 having a base that is movable along a robotic device track 406 to allow the inspection system 100 to be positioned at any position along the length of the barrel section 306 or other structure 300. Although shown as a robotic device 402, the movable platform 400 can be provided in any of a variety of alternative configurations. For example, the movable platform 400 can be an overhead gantry (not shown) or any other type of movable platform 400 capable of positioning the inspection system 100 relative to the structure 300. In some examples, the movable platform 400 can sequentially position the inspection system 100 at different positions along the structure 300 and generate a reconstructed surface 174 ( Fig.21 In some examples, processor 170 ( Fig. 20 ) The reconstructed surfaces 174 can be stitched together to form a three-dimensional representation of a majority of the surface 310 of the structure 300.

[0051] exist Figure 2 In the example of FIG. 1 , the inspection system 100 includes a graphical user interface (GUI 210) in the form of a laptop computer 212 that includes a processor 170 that is communicatively coupled to a light source 150 and a camera 130. The laptop computer 212 has a display screen 216 that is configured to display a reconstructed surface 174 of an inspected surface 310. As described below, Fig.21 An example of a display screen 216 of a GUI 210 is shown, which displays an image 140 captured by an inspection system 100 that happens to include two cameras 130 (e.g., a first camera 131 and a second camera 132) and three light sources 150 (e.g., a first light source 151, a second light source 152, and a third light source 153), and also displays a reconstructed surface 174. Also shown in the reconstructed surface 174 are surface flaws or defects 200 detected by the processor 170. The image 140 and / or the reconstructed surface 174 may be displayed in real time, or may also be displayed shortly (e.g., within a few seconds) after the image 140 is captured and processed.

[0052] exist Figure 4-14 In the embodiment, the support frame 102 is configured as a truss structure, which includes a plurality of elongated beams 104 and a plurality of connecting elements 106. In the example shown, the beams 104 are lightweight tubes formed of a high-strength, high-stiffness material, such as a polymer reinforced fiber matrix material (e.g., a carbon fiber composite material). However, the beams 104 can be provided in a configuration other than a tube and can be formed of any suitable material, including any metallic material (e.g., aluminum) and / or any non-metallic material, such as a non-fiber polymer material.

[0053] The connecting elements 106 are configured to interconnect the beams 104. To this end, each connecting element 106 has a plurality of openings 116, each of which is sized to tightly receive an end of a beam 104, such as Fig.11 In the example shown, the connecting element 106 is generally spherical and formed of a high-strength polymer material. However, the connecting element 106 can be provided in any shape or size and can be formed of any type of material (e.g., composite material, metallic material) or any combination of materials.

[0054] The connecting elements 106 at the bottom ends 112 of the support frames 102 are referred to as frame feet 114. The frame feet 114 are interconnected by the beams 104. In the example shown, the frame feet 114 are hemispherical in shape and have a generally flat bottom for placement against the surface 310 being inspected, such as Figure 7 However, in other examples, the frame foot 114 can have the same size, shape, and configuration as the remaining connection elements 106 of the support frame 102. In some examples, the support frame 102 can include a force sensor (not shown) located on the frame foot 114 to provide force feedback to the movable platform 400 (e.g., the robotic device 402) to limit the amount of force applied by the support frame 102 to the surface 310, thereby avoiding damage to the support frame 102 or the surface 310 being inspected.

[0055] The interconnection of the beams 104 via the connection elements 106 results in a high level of rigidity in the support frame 102. For example, the support frame 102 is configured to restrict relative motion between any two points (e.g., any two connection elements 106) on the support frame 102. Advantageously, providing the support frame 102 as a rigid structure allows the inspection system 100 to generate the reconstructed surface 174 with a high level of accuracy, which allows for a high level of precision when inspecting small features in the inspected surface 310.

[0056] exist Fig.10 In the example of FIG. 1 , the truss structure of the support frame 102 is arranged in a dome-like shape having a hexagonal cross-sectional shape when viewed from a top-down direction. However, the plurality of openings 116 ( Fig.11 ) allows for a high level of customization of the size and shape of the support frame 102 to accommodate surfaces of various sizes (e.g., different radii of curvature) and / or shapes (e.g., planar, cylindrical, conical, hyperbolic, etc.). For example, Fig.12 An example of a support frame 102 is shown, wherein the Figure 7 The vertically oriented beam 104 reduces the overall height of the support frame 102, thereby allowing the camera 130 and light source 150 to be larger than Figure 7 The support frame 102 allows for positioning closer to the surface 310 being inspected. In contrast, Fig.13 An example of a support frame 102 is shown, wherein the Figure 7 The inclined beam 104 and the vertically oriented beam 104 are relatively long, which increases the overall height of the support frame 101, thereby allowing the position of the camera 130 and the light source 150 to be higher than Figure 7 The support frame 102 allows the camera 130 to be positioned farther from the surface 310. Positioning the camera farther from the surface 310 allows the camera 130 to have a larger field of view, thereby allowing the inspection system 100 to inspect a larger portion of the structure 300.

[0057] Although the support frame 102 is shown as a truss structure in the figure, the support frame 102 can be provided in any of a variety of alternative configurations capable of supporting multiple cameras 130 and multiple light sources 150. For example, the support frame 102 can be provided as a rigid single-piece shell-like structure (not shown) preferably formed of a lightweight material. Such a shell-like support frame can have a dome-like shape, and the cameras 130 and light sources 150 can be attached to the inside of the shell-like support frame. In one example, the shape of the shell-like support frame can be similar to Figure 15-16 The background light shield 160 is shown. However, the shell-like support frame can have any of a variety of different shapes, including a circular shape, and is not limited to Figure 15-16 Advantageously, the shell-like support frame can perform the dual functions of supporting the camera 130 and the light source 150 and blocking background light.

[0058] Figure 5-6 An example of an inspection system 100 is shown that, due to its light weight, can be manually held in place against a surface 310 being inspected. In the example shown, the support frame 102 includes a pair of handles 110 extending from opposite sides of the top end 108 of the support frame 102, thereby allowing a technician to grasp the handles 110 with both hands and hold the support frame 102 in place near or against the surface 310. As described in more detail below, Figure 5-6Examples include a GUI 210 in the form of a tablet computer 214. The tablet computer 214 can be mounted to the handle 110 and / or the top end 108 of the support frame 102, and is configured to display (e.g., in real time) the image 140 captured by the camera 130 and / or the reconstructed surface 174 based on the image 140 captured by the camera 130.

[0059] As described above, in each of the examples disclosed herein, the inspection system 100 includes at least three light sources 150 mounted at different locations on the support frame 102 . Fig. 9 An example of light sources 150 is shown clamped to one of the beams 104 of the support frame 102. As described in more detail below, the light sources 150 are activated one at a time (via the processor 170) to temporarily illuminate at least a common area 312 ( Figure 4 and Figure 7 ), while the cameras 130 capture images 140 of a common area 312. The common area 312 may be described as the portion of the surface 310 that appears in each image 140 captured by the different cameras 130. Figure 4 and Figure 7 In the example of FIG. 3 , the common area 312 is defined by the support frame 102 .

[0060] exist Figure 4-10 In the example described above, the inspection system 100 includes exactly three light sources 150. However, in other examples, the inspection system 100 may include more than three light sources 150. For example, Fig.12 and Fig.13 Each of the inspection systems 100 has a total of nine light sources 150 mounted at different orientations on the support frame 102. Figure 4-12 In each example of FIG. 1 , the light sources 150 are equiangularly spaced apart from each other when the inspection system 100 is viewed from a top-down perspective. However, in other examples not shown, the light sources 150 may be non-equiangularly spaced apart.

[0061] Light source 150 may be an omnidirectional light source (not shown), or light source 150 may be a directional light source 154 ( Figure 7 ), each directional light source is configured to illuminate along a direction 156 ( Figure 7 ) casts the main part of its light, such as Figure 4 and Figure 7 The illumination direction 156 of each directional light source 154 is generally directed toward a common area 312 of the surface 310 being inspected. Figure 7In the example shown, the directional light sources 154 are mounted to the support frame 102 and / or oriented in a manner such that the illumination direction 156 of each directional light source 154 forms an incident angle 158 of 15-75 degrees with the common area 312. In order to maximize the shadow effect on the inspected surface 310, the incident angle 158 of the illumination direction 156 of each directional light source 154 is preferably 15-45 degrees. In the example shown, the light source 150 is a linear polarized spotlight that is configured to emit polarized light to reduce specular reflections of the inspected surface 310, thereby improving the accuracy of the reconstructed surface 174.

[0062] As described above, the inspection system 100 includes at least two cameras 130 . Figure 8 An example of one of the cameras 130 clamped to one of the beams 104 is shown. Figure 8 , the cameras 130 are mounted at different locations on the support frame 102 to capture images 140 of the surface 310 from different viewing angles. As described above, the multiple images 140 taken from different viewing angles correct for low-frequency shape distortions in the reconstructed surface 174. In addition, the multiple cameras 130 mounted at different locations allow for a relatively large field of view, which reduces the total amount of time required to inspect a large surface 310.

[0063] Although at least two cameras 130 are required, the inspection system 100 may include more than two cameras 130. For example, Figure 4-12 Each inspection system 100 in includes exactly three cameras 130. In another example, Fig.13 The inspection system 100 has exactly four cameras 130, including three cameras 130 mounted on the inclined beam 104 of the support frame 102 and one centrally located overhead camera 130 (e.g., a panoramic camera) mounted at the top end 108 of the support frame 102. Regardless of the number of cameras 130 in the inspection system 100, each camera 130 has a viewing direction 136 ( Figure 7 ), and all of the cameras 130 are mounted on the support frame 102 in a manner such that their respective viewing directions 136 are generally directed toward a common area 312 of the surface 310. In some examples, at least some of the cameras 130 may be mounted such that when the inspection system 100 is placed above or against the inspected surface 310, the viewing directions 136 of at least some of the cameras 130 are locally non-perpendicular to the surface 310, which may enhance the ability to capture shadows created by discontinuities in the surface 310 in the image 140. In the example shown in the figures, the cameras 130 are equiangularly spaced apart from one another when the inspection system 100 is viewed from a top-down perspective. However, in other examples not shown, the cameras 130 may be non-equiangularly spaced apart.

[0064] In some examples, the inspection system 100 includes reference targets 120 mounted at different locations on the support frame 102, respectively. The reference targets 120 are used for zoom verification of the images 140 and / or for calculating the camera position. For examples of the inspection system 100 including the reference targets 120, in addition to capturing the common area 312 of the surface 310 being inspected, each camera 130 is also configured to capture an image 140 containing the reference targets 120. In order to determine the position of the camera 130, the common area 312 must contain a sufficient number of reference targets 120. More specifically, the cameras 130 of the inspection system 100 are configured and oriented so that at least three common reference targets 120 are visible in each camera view, plus at least one additional camera view containing the three same reference targets 120. For example, in an inspection system 100 having exactly two cameras 130 including a first camera 131 and a second camera 132, each image 140 captured by the first camera 131 contains at least three reference targets 120, and each image 140 captured by the second camera 132 also contains at least the three same reference targets 120. Increasing the number of cameras 130 of the inspection system 100 will increase the accuracy of the reconstructed surface 174, which in turn will allow the processor 170 to increase the accuracy with which it can detect surface features, such as defects 200 in the surface 310. As can be appreciated, the inspection system 100 can have three or more cameras 130, which will correspondingly increase the accuracy of the reconstructed surface 174.

[0065] Figure 13-14 Reference targets 120 are shown mounted on at least two beams 104 that are interconnected with frame feet 114 at the bottom end 112 of the support frame 102. The two reference targets 120 on each beam 104 are spaced apart by a known scale distance 119, which allows the beams 104 to be used as scales 118. The scale distance 119 between the reference targets 120 on each scale 118 is individually measured by a high accuracy measuring instrument (not shown). The support frame 102 of the inspection system 100 preferably includes at least one scale 118 oriented substantially parallel (e.g., within 30 degrees) to the x-axis of the local coordinate system 442 and at least one scale 118 oriented substantially parallel (e.g., within 30°) to the y-axis of the local coordinate system 442 to facilitate scale verification of the image 140 in the x and y directions, respectively. Although Fig.13 The support frame 102 in FIG. 1 has a scale 118 located at the bottom end 112 , but any one or more beams 104 of the support frame 102 may include reference targets 120 that allow the beam 104 to be used as a scale 118 .

[0066] When 18, the processor 170 uses photogrammetry to perform zoom verification on the image 140 based on the zoom distance 119 between the reference targets 120 on the scale. In addition, as described above, the processor 170 uses photogrammetry to calculate the camera position and / or calculate the inspection system 100 (e.g., the local coordinate system 442- Figure 2 ) relative to the global coordinate system 440 of the structure 300 ( Figure 2 )'s position and orientation. The scaling verification of the images 140 and / or the calculation of the camera position is performed as a separate step before generating the reconstructed surface 174. The scaling verification and / or the calculation of the camera position can be performed in the background after all the images 140 have been captured during the inspection of the surface 310.

[0067] Brief reference Figure 15-16 , the reference target 120 is shown as having a circular shape. However, the reference target 120 may be configured in any of a variety of shapes, such as a square or other shapes. Fig.16 An example of a coded target 122 is shown. Each coded target 122 of the inspection system 100 has a unique pattern or code (e.g., a coded ring pattern) that is recognized by the processor 170 when processing the image 140 to calibrate the camera 130 and / or determine the camera position and orientation. In this regard, the coded target 122 facilitates the generation of a highly accurate reconstructed surface 174 and allows for accurate detection and characterization of defects 200 in the surface 310.

[0068] In some examples, the inspection system 100 may include one or more temperature sensors 124 and / or thermocouples 126 for measuring the temperature of the support frame 102 and / or the temperature of the ambient air. Fig.14 An example of a support frame 102 is shown having a temperature sensor 124 mounted on one of the beams 104 and configured to measure the temperature of the ambient air. Also shown is a thermocouple 126 mounted on one of the beams 104 for measuring the temperature of the support frame 102. Measurements from the temperature sensor 124 and the thermocouple 126 are received by a processor 170, which uses the temperature measurements to compensate for thermal expansion or contraction of the support frame 102 during calibration and / or when processing the images 140 for surface reconstruction.

[0069] Reference target 120 may be made of a relatively thin, flexible material, such as a sheet of vinyl, with an adhesive on one side for adhering reference target 120 to frame foot 114. However, reference target 120 may be made of any material and may be located in any of a variety of alternative orientations on support frame 102, and is not limited to being mounted on frame foot 114.

[0070] Fig.13A common area 312 is shown with an exaggerated size, which includes all reference targets 120 mounted on support frame 102. As described above, processor 170 is configured to analyze image 140 including reference targets 120 and determine a scaling factor for image 140 based on reference targets 120. Alternatively or additionally, processor 170 is configured to use reference targets 120 to determine the three-dimensional orientation of cameras 130 relative to each other and relative to surface 310. In addition, processor 170 may use reference targets 120 and identified structural features 180 ( Fig. 20 ) to determine the inspection system 100 ( Figure 2-3 )'s local reference coordinate system 442 relative to the structure 300 ( Figure 2-3 )'s position and orientation in the global reference coordinate system 440.

[0071] refer to Figure 17-18 , shows an example of an inspection system 100 having a background light shield 160 for blocking background light (not shown) to avoid any adverse effects on the quality of the image 140 generated by the camera 130 (which may affect the accuracy of the reconstructed surface 174). Background light can be described as light from an external and / or distant light source, and can include sunlight, factory shop lights, or any other external light source located outside the support frame 102. The background light shield 160 is preferably configured to prevent most, if not all, of the background light from entering the interior of the support frame 102 and / or impinging on the common area 312 of the inspected surface 310. The background light shield 160 can be formed of a rigid or flexible material and can be removably attached to or mounted above the support frame 102 using any of a variety of techniques, such as mechanical fasteners (not shown), Velcro fasteners, or other fasteners. TM (not shown), adhesive, or any other attachment method. For example, background shade 160 can be a fabric, such as black polyester fabric, attached (eg, adhered) to beams 104 and / or connecting elements 106 of support frame 102.

[0072] As an alternative or in addition to the background mask 160, the inspection system 100 may include a lens filter 134 mounted on the lens of each camera 130, such as Fig.19 Lens filter 134 may be configured to filter out one or more wavelengths of background light, thereby preventing the background light from having any adverse effect on the quality of image 140. In some examples, light source 150 may be configured to emit at least one wavelength of light that is different from the wavelengths filtered out by lens filter 134, so that common area 312 is sufficiently illuminated in a manner that allows camera 130 to capture high-quality image 140.

[0073] refer to Fig. 20, a block diagram of an example of an inspection system 100 is shown, which includes at least three light sources 150 (e.g., a first light source 151, a second light source 152, a third light source 153, etc.), at least two cameras 130 (e.g., a first camera 131, a second camera 132, etc.), a processor 170, and a memory 172. The processor 170 and the memory 172 may be part of a computing device that controls the light sources 150 and the cameras 130. For example, the processor 170 and the memory 172 may be integrated into a desktop computer (not shown), a laptop computer 212 (e.g., Figure 2 ), tablet computer 214 (e.g. Figure 5 ) or any other type of computing device. The above-mentioned GUI 210 can also be integrated into a computing device including a processor 170 and a memory 172.

[0074] The processor 170 is configured to control and sequence the light source 150 in synchronization with the camera 130 to capture a plurality of images 140 of the inspected surface 310. The processor 170 receives the images 140, which may be stored in the memory 172. In the example shown, the images 140 include a first image 141, a second image 142, and a third image 143 captured by the first camera 131, and a first image 141, a second image 142, and a third image 143 captured by the second camera 132. Alternatively or additionally, the memory 172 may store a plurality of common structural features 180 that constitute the inspected structure 300. The common structural features 180 may include fasteners 182 ( Fig. 22 )、Segment interface 184( Figure 1 )、Window surround 188( Figure 2 ) and any of a variety of other types of structural features 180 that are common components of the structure 300. As described in more detail below, the processor 170 is configured to identify common structural features 180 in the image 140 of the inspected surface 310 and mask out these structural features 180. Otherwise, these structural features can negatively affect the accuracy of the reconstructed surface 174.

[0075] The processor 170 is also configured to analyze the structural features 180, defects 200, and / or textures 206 of the reconstructed surface 174. For example, the processor 170 may use known defect detection techniques, such as robust surface fitting or normal vector tracking, to analyze the surface defects 200, such as scratches 202 (e.g., Fig.21 As shown), chisel marks 204 (as Fig.23) and other types of defects 200. Advantageously, as described above, the inspection system 100 allows for the detection of small surface defects 200 and flaws that cannot be detected using conventional inspection equipment. The processor 170 can analyze the detected defects 200 and provide characteristics (e.g., measurements), such as size (e.g., defect depth of 0.010 inches) and defect type (e.g., scratches, gouges, etc.).

[0076] Processor 170 may also analyze texture 206 (eg, surface texture 206 of reconstructed surface 174) using known surface fitting methods or by generating contour lines (not shown) directly on reconstructed surface 174 and then measuring peaks and valleys along the contour lines. Fig.24 By analyzing the texture 206 of the surface 310, the inspection system 100 advantageously provides a method for better understanding the characteristics (e.g., average surface roughness) of the substrate surface and any surface coatings (e.g., paint) subsequently applied to the substrate surface. Understanding the characteristics of the substrate surface and / or the surface coatings can help determine the types of defects 200 that may be found when inspecting the surface 310. For example, defects 200 with a feature size smaller than the texture or surface roughness may not be detected.

[0077] Still reference Fig. 20 As described above, the processor 170 is configured to control and sequence the light sources 150 in synchronization with the cameras 130. More specifically, while the inspection system 100 remains stationary in position above the surface, the processor 170 is configured to temporarily activate one light source 150 at a time and cause all cameras 130 to capture a corresponding plurality of images 140 of a common area 312 of the surface 310 each time a light source 150 is activated. This cycle is repeated until each light source 150 is activated.

[0078] For example, in an inspection system 100 having exactly three light sources 150 and exactly two cameras 130, the processor 170 activates (i.e., turns on) the first light source 151, commands the first camera 131 and the second camera 132 to capture the image 140 of the common area 312 while the first light source 151 is activated, and then deactivates (i.e., turns off) the first light source 151. After capturing the image 140 with the first camera 131 and the second camera 132 with only the first light source 151 activated, the processor 170 then activates the second light source 152 and commands the first camera 131 and the second camera 32 to capture the image 140 of the common area 312 with only the second light source 152 activated, and then deactivates the second light source 152. The cycle is repeated for the third light source 153. The amount of time that each light source 150 is activated (e.g., emitting light) can be relatively short, such as less than one second. Depending on the total number of cameras 130, the total amount of time to cycle through all the light sources 150 while capturing the image 140 can also be relatively short, such as a few seconds.

[0079] The processor 170 receives the images 140, which are combined and processed by the cameras 130 and indexed by the light sources 150. For example, in an inspection system 100 having exactly two cameras 130 (e.g., a first camera 131 and a second camera 132) and exactly three light sources 150 (e.g., a first light source 151, a second light source 152, and a third light source 153), the first camera 131 combines its first image 141, the second image 142, and the third image 143, and the second camera 130 combines its first image 141, the second image 142, and the third image 143. The processor 170 then generates a digital surface based on each camera view. For example, the processor 170 uses a photometric algorithm to generate a first digital surface (not shown) based on the first image 141, the second image 142, and the third image 143 captured by the first camera 131, and then generates a second digital surface (not shown) based on the first image 141, the second image 142, and the third image 143 captured by the second camera 132. The processor 170 then uses the camera positions to align the first digital surface and the second digital surface, and then generates a best fit surface (i.e., a reconstructed surface 174) after removing outliers and noise. The above process is an example for processing multiple images 140 with multiple viewing angles to generate the reconstructed surface 174. Any of a variety of other optimization methods and approaches can be implemented to perform multi-view surface reconstruction to obtain the reconstructed surface 174.

[0080] Advantageously, in the inspection system 100 of the present disclosure, using multiple cameras 130 to capture images 140 of the surface 310 from different viewing angles allows the inspection system 100 to reduce low-frequency shape distortions that occur due to uneven reflections of light on the surface 310. Photometric stereo algorithms assume that the light source is far away, so the light intensity across the surface is uniform. However, in many practical applications, the light source is not too far away. For example, if the light source is illuminating the surface of a flat plate (not shown), the light intensity on the surface will be uneven even if the surface is completely flat. Advantageously, in the inspection system 100 of the present disclosure, by having two or more viewing directions 136 (i.e., multiple cameras 130), the non-uniform reflectivity is averaged to minimize the effects of shape distortion.

[0081] Alternatively or additionally, the non-uniform reflectivity may be calibrated to minimize the effects of shape distortion in the reconstructed surface 174. In this regard, prior to use, the inspection system (e.g., camera) undergoes a calibration process in which the camera 130 captures images of known and certified artifacts (not shown) to ensure that the inspection system 100 (e.g., camera 130) is functioning properly. Examples of certified artifacts include gauge blocks (e.g., prisms of known size), flat plates etched with grids of known size, calibration targets, and other known artifacts used for camera calibration. As part of the calibration process, intrinsic and extrinsic parameters of the camera are adjusted in software operated by the processor 170. The calibration process is performed through the GUI and may be repeated periodically during the life of the inspection system 100 to maintain or improve the accuracy and reliability of the surface reconstruction.

[0082] Advantageously, by minimizing shape deformation due to the plurality of cameras 130 of the inspection system 100, variations in surface curvature and material are easily highlighted, thereby exposing raised and / or recessed defects 200, such as scratches 202, gouges 204, and tool marks. Additionally, any local deviations, anomalies, and / or defects in the texture 206 of the surface 310 and the surface coating (e.g., paint) are easily highlighted.

[0083] Fig.21 An example of a display screen 216 of a GUI 210 is shown, showing images 140 captured by an inspection system 100 having exactly three light sources 150 and exactly two cameras 130. The display screen 216 shows three images 140 captured by a first camera 131 from its only viewing direction 136 under three different lighting conditions respectively produced by the three light sources 150. In addition, the display screen 216 shows three images 140 captured by a second camera 132 from its only viewing direction 136 under the same three lighting conditions. Each image 140 contains a surface defect 200 in the form of a scratch 202. Due to the differences in the lighting direction 156 in which each image 140 was captured, there are variations in the extent to which some of the scratches 202 appear in the images 140. For example, only a portion of a scratch 202 may appear in one image 140, while another portion of the same scratch 202 may appear in one or more other images 140. The processor 170 combines the images 140 in a manner such that the reconstructed surface 174 represents the topography of a common region 312 of the surface 310 with a high level of accuracy. Additionally, processor 170 detects small surface discontinuities, such as scratches 202, with a high level of accuracy. Fig.21 The right side of the display screen 216 shows the entire reconstructed surface 174 including each scratch 202.

[0084] As described above, the processor 170 is configured to analyze the structural features 180, defects 200, and / or textures 206 of the reconstructed surface 174. As described above, the processor 170 initially identifies any common structural features 180 that appear in the image 140 of the inspected surface 310. For example, the processor 170 identifies the common structural features 180 by comparing the image 140 with the structural features 180 stored in the memory 172. If the image 140 contains common structural features 180, the processor 170 masks these structural features 180. Otherwise, the structural features 180 may negatively impact the reliability and accuracy of the reconstruction process of the surface 310 due to reflections, shadows, and / or other effects caused by the structural features 180.

[0085] To illustrate the masking of the common structural feature 180, Fig. 22 An example of an inspection system 100 is shown that is configured to be manually held in place against a surface 310 of a structure 300 having a common structural feature 180 in the form of fasteners 182 securing a skin panel 186 to an underlying structural component (not shown). Fig.23 Display screen 216 is shown, which shows Fig. 22 216. A plurality of images 140 of a surface 310 under inspection are shown. Each image 140 contains a defect 200 in the surface 310 in the form of a gouge 204 in a skin panel 186. On the right side of the display screen 216 is a reconstructed surface 174 generated from the images 140. The processor 170 masks common structural features 180 (i.e., fasteners 182), which improves the accuracy of the reconstructed surface 174 and also has the effect of highlighting the gouge 204 in the skin panel 186. The gouge 204 is detected using a suitable defect detection technique (robust surface fitting or normal tracking as described below) or another defect detection technique. The processor 170 may also plot and / or highlight the location of the defect 200 in the surface 310.

[0086] Fig.24 An example of a display screen 216 of the GUI 210 is shown, the display screen 216 showing an image 140 captured by the inspection system 100 having exactly three light sources 150 and exactly two cameras 130. Fig.21 and Fig.23 Similar to the arrangement in Fig.24The display screen 216 in illustrative embodiment shows three images 140 captured by the first camera 131 from its only viewing direction 136 under three different lighting conditions from three light sources 150, and three images 140 captured by the second camera 132 from its only viewing direction 136 under the same three lighting conditions. Due to the differences in the lighting directions 156, there are variations in the extent to which the texture 206 appears in the images 140. The processor 170 combines the images 140 in a manner such that the reconstructed surface 174 accurately represents the texture 206 throughout the common region 312 of the surface 310. As described above, the processor 170 may also characterize the texture 206 (e.g., in terms of surface roughness) using known surface fitting methods or by analyzing the reconstructed surface 174 by generating contour lines (not shown) on the reconstructed surface and measuring the peak-to-valley distances at different points along the contour lines.

[0087] refer to Fig.25 , a flow chart of operations included in a method 500 for capturing the topography of an inspected surface 310 is shown. As described in more detail below, the topography is represented by a reconstructed surface 174 (e.g., a three-dimensional digital model). The method 500 is performed using the inspection system 100 described above, which includes a rigid support frame 102, at least three light sources 150, at least two cameras 130, and a processor 170.

[0088] The method 500 includes calibrating the inspection system 100 prior to its first use. For example, the method 500 includes calibrating the camera 130 using a certified workpiece (not shown) to determine the parameters (e.g., intrinsic and / or extrinsic) of the camera 130 and to confirm that the inspection system 100 is functioning properly. Examples of certified workpieces include gauge blocks, flat plates etched with grids, calibration targets, and other known workpieces. Using known techniques, the camera parameters are determined and the algorithm is adjusted to account for the camera parameters.

[0089] The calibration process may also include using photogrammetry techniques to perform scale verification of the image 140 and / or calculate the three-dimensional position of the camera 130. Figure 13-14 , the method 500 includes mounting the reference target 120 on the bottom end 112 of the support frame and within the field of view of the camera. After capturing the images 140, each containing at least three common reference targets 120, the calibration process includes analyzing the images 140 using the processor 170 to determine the zoom factor of the images 140 and / or the three-dimensional orientation of the cameras 130 relative to each other based on the reference targets 120. In addition, the calibration process may include using photogrammetry techniques to calculate the local coordinate system 442 ( Figure 2 ) relative to the global coordinate system 440 of the structure 300 ( Figure 2In some examples, method 500 may also include measuring the temperature of ambient air and / or support frame 102 before and / or during inspection, and compensating for thermal expansion or contraction of support frame 102 during calibration and / or when processing image 140 for surface reconstruction and / or defect detection.

[0090] Step 502 of method 500 includes positioning support frame 102 above surface 310 to be inspected. As described above, light source 150 is mounted at different locations on support frame 102. Each camera 130 has a viewing direction 136 and is also mounted at different locations on support frame 102 in a manner such that viewing direction 136 of each camera 130 is generally directed toward common area 312 of surface 310, such as Figure 4 In some examples, method 500 includes mounting light source 150 and / or camera 130 on support frame 102 in a manner such that when inspection system 100 is viewed from a top-down perspective, light source 150 and / or camera 130 are equiangularly spaced from each other, such as Fig.10 However, as described above, the light source 150 and / or the camera 130 may be mounted at any orientation on the support frame 102 and are not limited to being equiangularly spaced from each other.

[0091] In some examples, step 502 of positioning support frame 102 (ie, inspection system 100) above surface 310 includes positioning support frame 102 at one or more different locations on surface 310 via movable platform 400 to which support frame 102 is attached. For example, Figure 2-3 The support frame 102 is shown attached to the end of a robotic arm 404 of the robotic device 402. As described above, the robotic device 402 is movable along the robotic device track 406 for positioning the inspection system 100 at any orientation along the length of the structure 300 being inspected. In other examples not shown, the step 502 of positioning the support frame 102 above the surface 310 may be performed using any of a variety of alternative movable platform configurations, such as an overhead gantry (not shown). In further examples, the step 502 of positioning the support frame 102 above the surface 310 may include manually holding the support frame 102 in position above the surface 310, such as by moving the support frame 102 downwardly from the surface 310. Figure 5 as shown in the above example.

[0092] Brief reference Figure 7In an example of the method 500, the method 500 includes orienting the light sources 150 in a manner such that the illumination direction 156 of each light source 150 forms an incident angle 158 of 15-75 degrees with the common area 312. Preferably, the illumination direction 156 forms an incident angle 158 between 15-45 degrees, thereby maximizing the shadow effect on the inspected surface 310. The method 500 may also include orienting the cameras 130 or adjusting the camera orientation so that the viewing direction 136 of each camera 130 is locally non-perpendicular to the inspected surface 310, thereby enhancing the ability of each camera 130 to capture shadows generated by irregularities or discontinuities in the surface 310.

[0093] The method 500 may also include changing the size and / or shape of the support frame 102 by changing the configuration of the beams 104 interconnected by the connecting elements 106 to accommodate the differently sized and / or shaped surfaces 310. In this regard, the size and / or shape of the support frame 102 may be changed by inserting the beam ends into different openings 116 in the connecting elements 106 and / or by changing the length of the beams 104. Figure 7 , Fig.12 and Fig.13 Three different versions of various configurations of the support frame 102 are shown, which may be achieved by varying the length of the beams 104 and / or varying the manner in which the beams 104 are interconnected by the connecting elements 106 .

[0094] Step 504 of method 500 includes temporarily activating one light source 150 at a time and causing all cameras 130 to capture a corresponding plurality of images 140 of common area 312 each time a light source 150 is activated. For example, in an inspection system 100 having exactly three light sources 150 and exactly two cameras 130, the process of performing step 504 includes activating only the first light source 151 and capturing images 140 of common area 312 using first camera 131 and second camera 132, activating only the second light source 152 and capturing images of common area 312 again using first camera 131 and second camera 132, and then activating only the third light source 153 and capturing images of common area 312 again using first camera 131 and second camera 132. As described above, images 140 are sent to processor 170 for processing to generate a reconstructed surface 174 of inspected surface 310.

[0095] In some examples, method 500 includes using background light shield 160 that can be mounted to support frame 102 to at least partially block background light from shining onto common area 312 of surface 310. For example, as described above Figure 17-18As shown, the method 500 may include attaching a blackout fabric (e.g., a black polyester fabric) to the beams 104 and / or the connecting elements 106 of the support frame 102. Alternatively, the background light shield 160 may be a rigid, one-piece, shell-like cover (not shown) formed of a lightweight metal material (e.g., aluminum) and / or a lightweight polymer material (e.g., a carbon fiber composite material) that is removably attached to the support frame 102. As an alternative or in addition to using the background light shield 160 to block background light, the method 500 may include using a lens filter 134 mounted on the lens of each camera 130 to filter out one or more wavelengths of background light. When using such a lens filter 134, the method 500 may include emitting at least one wavelength of light from the light source 150 that is different from the wavelength filtered out by the lens filter 134.

[0096] Step 506 of method 500 includes processing image 140 in a manner that produces a reconstructed surface 174 that represents the topography of common area 312. As described above, processor 170 uses a photometric stereo algorithm to form reconstructed surface 174 based on image 140. In one example of processing the image, step 506 includes receiving image 140 at processor 170, processor 170 combining and processing image 140 through camera 130, and indexing image 140 through light source 150. For example, in the above-mentioned inspection system 100 having exactly two cameras 130 (e.g., first camera 131 and second camera 132) and exactly three light sources 150 (e.g., first light source 151, second light source 152, and third light source 153), step 506 includes combining the first image 141, the second image 142, and the third image 143 of the first camera 131, and combining the first image 141, the second image 142, and the third image 143 of the second camera 132.

[0097] Step 506 then includes using a photometric algorithm to generate a first digital surface (not shown) based on the first image 141, the second image 142, and the third image 143 captured by the first camera 131, and generating a second digital surface (not shown) based on the first image 141, the second image 142, and the third image 143 captured by the second camera 132. Step 506 then includes using the camera positions to align the first and second digital surfaces and generate a best fit surface (i.e., the reconstructed surface 174) after removing outliers and noise. Step 506 of processing the images 140 and generating the reconstructed surface 174 can be performed using any of a variety of alternative optimization methods for multi-view surface reconstruction to produce the reconstructed surface 174, and is not limited to the above process. As described above, combining multiple images 140 taken from different viewing angles averages the uneven reflectivity of light on the inspected surface 310, which would otherwise result in shape distortions in the reconstructed surface 174.

[0098] refer to Fig.25 , a method 600 of analyzing structural features 180, defects 200, and / or textures 206 of a reconstructed surface 174 is shown. Step 602 of the method 600 includes identifying common structural features 180 in the image 140 of the inspected surface 310, if present, thereby obscuring the structural features 180. As described above, such structural features 180 can negatively impact the reliability and accuracy of the reconstructed surface 174. Fig.23 Display screen 216 of GUI 210 showing images 140 of inspected surface 310 is shown. Each image 140 has a plurality of structural features 180 in the form of fasteners 182. However, fasteners 182 have been obscured and are therefore not visible in the image. Fig.23 On the reconstructed surface 174 shown on the right.

[0099] Step 604 of method 600 includes detecting defects 200 using a suitable defect detection technique. In one example, step 604 includes detecting defects 200 by robust surface fitting, which initially involves importing data (e.g., reconstructed surface 174 and corresponding image 140) into an algorithm, generating a robust polynomial surface fit of the data, and identifying the most deviating points of the polynomial surface fit. Points exceeding a predetermined threshold are removed to produce a new robust polynomial surface fit. Step 604 then includes generating a heat map (not shown) of the data based on the new robust polynomial surface fit, and identifying defects 200 based on the deviating points in the heat map. In another example, step 604 includes detecting defects 200 by normal tracking, which involves extracting surface normals at each vertex of the reconstructed surface, calculating the average angle between each surface normal and its neighboring surface normals, identifying points where the angle changes by more than a predetermined maximum value, and identifying defects 200 based on the points where the angle changes by the surface normal and its neighboring surface normals are the largest.

[0100] Advantageously, the above-described defect detection method 600 maintains and / or generates a relationship between two-dimensional data (i.e., image 140) and three-dimensional data (i.e., reconstructed surface 174), which facilitates the defect detection process. Advantageously, the method 600 uses two-dimensional data (e.g., image 140) to locate structural features, which is less computationally intensive than using three-dimensional data (i.e., reconstructed surface 174). In addition, the method 600 uses three-dimensional data (i.e., reconstructed surface 174) to perform defect detection and measurement.

[0101] Regardless of the defect detection technique used, method 600 includes step 606 of mapping and / or highlighting the locations of defects 200 in reconstructed surface 174. In this regard, step 606 includes classifying the defect type, which may be performed using machine learning or any other suitable defect classification technique. Additionally, step 606 may include quantifying parameters of each defect 200 by analyzing reconstructed surface 174 once the locations of defects 200 are determined using the defect detection algorithm. For example, step 606 may include classifying the defects as Fig.21 The scratches 202 in the image are identified and the scratch parameters are quantified according to the maximum depth and length of each scratch 202 (e.g., the maximum depth is 0.005 inches, the length is 2.3 inches, etc.). Fig.23 In step 606, the defect 200 may be classified as Fig.23 204 in the image and quantify the gouge parameters according to its maximum width and depth and its length (e.g., maximum width is 0.050 inches, maximum depth is 0.020 inches, length is 3 inches, etc.). Although not shown, step 606 can also include annotating the reconstructed surface 174 with the defect type (e.g., gouge, scratch, etc.), defect parameters (e.g., maximum width, depth, and length), and the x, y, z location of the defect relative to the origin of the global coordinate system.

[0102] In addition to detecting defects 200 in the reconstructed surface 174, the method 500 may also include analyzing the texture 206 of the reconstructed surface 174 (e.g., Fig.24 ). For example, method 500 can use known surface fitting methods or generate contour lines (not shown) on reconstructed surface 174 to analyze texture 206 of surface 310. Texture 206 can be characterized based on the distance between peaks and adjacent valleys in the contour line plot. Analysis of texture 206 of surface 310 provides a method to better understand the characteristics (e.g., average surface roughness) of a substrate surface (not shown) and any surface coatings (e.g., paint—not shown) subsequently applied to the substrate surface, and can help determine the size of defects 200 that may be detected when inspecting surface 310.

[0103] In some examples, method 500 includes displaying image 140 and / or reconstructed surface 174 on a graphical user interface (GUI 210). For example, Figure 2 The GUI 210 of the laptop computer 212 is shown displaying the image 140 captured by the camera 130 in real time while the robotic device 402 holds the inspection system 100 in position over the surface 310 being inspected. In another example, Figure 5A technician is shown holding the inspection system 100 in position above the surface 310 while viewing the image 140 and the reconstructed surface 174 in real time on the GUI 210 mounted on the support frame 102 . Fig.19 Shows Figure 5 Display screen 216 , which displays image 140 and reconstructed surface 174 , shows defect 200 in the form of scratch 202 . Fig.21 is an example of a display screen 216 displaying an image 140 and a reconstructed surface 174 including Fig. 22 A defect 200 (eg, a gouge 204) is shown in one of the skin panels 186 being inspected.

[0104] Advantageously, combining multiple images 140 taken from different viewing angles can reduce shape distortions in the reconstructed surface 174, and this allows for high accuracy in detecting small defects 200, such as scratches 202 (e.g., Fig.21 ), chisel marks 204 (e.g. Fig.23 ) and other types of discontinuities. In addition, the highly accurate reconstructed surface 174 allows for analysis of small surface features, such as texture 206, thereby enabling determination of the type and / or size of defects 200 that may be detected. In addition to analyzing a surface for small defects, the inspection system 100 also provides the ability to detect relatively larger defects 200 (e.g., dents) in the surface 310.

[0105] With the benefit of the teachings presented in the foregoing description and the associated drawings, those skilled in the art to which the present disclosure belongs will think of many modifications and other versions and examples of the present disclosure. The versions and examples described herein are intended to be illustrative and are not intended to be limiting or exhaustive. Although specific terms are used herein, they are used only in a general and descriptive sense and not for the purpose of limitation. In addition to those listed herein, functionally equivalent methods and devices within the scope of the present disclosure are possible according to the above description. Such modifications and variations are intended to fall within the scope of the appended claims. The present disclosure is limited only by the terms of the appended claims and the full scope of equivalents to which these claims are entitled.

Claims

1. An inspection system (100) for capturing the topography of a surface (310), comprising: a rigid support frame (102) configured to be positioned above the surface (310); at least three light sources (150) configured to be mounted at different positions on the support frame (102); at least two cameras (130), each camera having a viewing direction (136) and configured to be mounted at different locations on the support frame (102) in a manner such that the viewing direction (136) of each camera (130) is generally directed toward a common area (312) of the surface (310); A processor (170) configured to: temporarily activating one light source (150) at a time and causing all of the cameras (130) to capture a corresponding plurality of images (140) of the common area (312) each time the light source (150) is activated; and The images (140) are combined in a manner that produces a reconstructed surface (174) representing the topography of the common region (312).

2. The inspection system (100) according to claim 1, wherein: The processor (170) is configured to analyze the reconstructed surface (174) for structural features (180), defects (200) and / or texture (206).

3. The inspection system (100) according to claim 1, further comprising: A graphical user interface (GUI (210)) configured to display the reconstructed surface (174).

4. The inspection system (100) according to claim 3, wherein: The GUI (210) is mountable on the support frame (102) in a manner that allows viewing of the reconstructed surface (174) when the support frame (102) is held in position above the surface (310).

5. The inspection system (100) according to claim 1, wherein: At least one of the light sources (150) is a directional light source (154) having an illumination direction (156) and is mountable to the support frame (102) in such a manner that the illumination direction (156) forms an incident angle (158) of 15-75 degrees with the common area (312).

6. The inspection system (100) of claim 1, wherein: When viewing the inspection system (100) from a top-down perspective, the light sources (150) and / or the cameras (130) are equiangularly spaced from each other.

7. The inspection system (100) of claim 1, wherein: The support frame (102) is composed of beams (104) interconnected by connecting elements (106), the beams being configured to allow the size and / or shape of the support frame (102) to be changed to accommodate surfaces (310) of different sizes and / or shapes.

8. The inspection system (100) according to claim 1, further comprising: one or more reference targets (120) mounted on a bottom end (112) of the support frame (102); Each of the cameras (130) is oriented in a manner to include the reference target (120) in the image (140); and The processor (170) is configured to analyze the image (140) including the reference target (120) and determine a scaling factor of the image (140) and a three-dimensional orientation of the cameras (130) relative to each other using the reference target (120).

9. The inspection system (100) of claim 1, further comprising at least one of the following: a background light shield (160) mountable to the support frame (102) and configured to at least partially block background light from impinging on the common area (312) of the surface (310); A lens filter (134) is mountable on each of the cameras (130) and configured to filter out one or more wavelengths of the background light, and the light source (150) is configured to emit light of at least one wavelength different from the wavelength filtered out by the lens filter (134).

10. An inspection system (100) for capturing the topography of a surface (310), comprising: a rigid support frame (102); a movable platform (400) configured to position the support frame (102) at one or more different locations on the surface (310); at least three light sources (150) configured to be mounted at different positions on the support frame (102); at least two cameras (130), each having a viewing direction (136) and configured to be mounted at different locations on the support frame (102) in such a manner that the viewing direction (136) of each camera (130) is generally directed toward a common area (312) of the surface (310); A processor (170) configured to: temporarily activating one light source (150) at a time and causing all of the cameras (130) to capture a corresponding plurality of images (140) of the common area (312) each time the light source (150) is activated; and The images (140) are combined in a manner that corrects shape distortions and produces a reconstructed surface (174) representing the topography of the common region (312).

11. A method of capturing the topography of a surface (310), comprising: Positioning a rigid support frame (102) above the surface (310), the support frame (102) having at least three light sources (150) mounted at different locations on the support frame (102), and at least two cameras (130), each camera having a viewing direction (136) and mounted at different locations on the support frame (102) in a manner such that the viewing direction (136) of each camera (130) is generally directed toward a common area (312) of the surface (310); temporarily activating one light source (150) at a time and causing all of the cameras (130) to capture a corresponding plurality of images (140) of the common area (312) each time a light source (150) is activated; and The images (140) are combined in a manner that produces a reconstructed surface (174) representing the topography of the common region (312).

12. The method according to claim 11, further comprising: The reconstructed surface (174) is analyzed for structural features (180), defects (200) and / or texture (206).

13. The method according to claim 12, further comprising: The reconstructed surface (174) is displayed on a graphical user interface (GUI (210)).

14. The method of claim 11, wherein displaying the reconstructed surface (174) on the GUI (210) comprises: The reconstructed surface (174) is viewed on the GUI (210) mounted on the support frame (102).

15. The method according to claim 11, further comprising: At least one of the light sources (150) is oriented in such a way that an illumination direction (156) of the light source (150) forms an incident angle (158) of 15-75 degrees with the common area (312).

16. The method of claim 11, wherein: The light sources (150) and / or the cameras (130) are mounted to the support frame (102) in such a manner that the light sources (150) and / or the cameras (130) are equiangularly spaced apart from one another when the support frame (102) is viewed from a top-down perspective.

17. The method of claim 11, wherein: The size and / or shape of the support frame (102) is varied by changing the arrangement of a plurality of beams (104) interconnected by a plurality of connecting elements (106) to accommodate surfaces (310) of varying sizes and / or shapes.

18. The method according to claim 11, further comprising: Mounting one or more reference targets (120) near a bottom end (112) of the support frame (102); capturing an image (140) of the reference target (120) using the camera (130); and The image (140) including the reference target (120) is analyzed using a processor (170) and a scaling factor of the image (140) and a three-dimensional orientation of the cameras (130) relative to each other are determined based on the reference target (120).

19. The method of claim 11, further comprising at least one of the following: at least partially blocking background light from striking the common area (312) of the surface (310) using a background light shield (160) mounted to the support frame (102); One or more wavelengths of the background light are filtered out using a lens filter (134) mounted on each of the cameras (130), and at least one wavelength of light is emitted from the light source (150), the at least one wavelength being different from the wavelength filtered out by the lens filter (134).

20. The method of claim 11, wherein positioning the support frame (102) above the surface (310) comprises: The support frame (102) is positioned at one or more different locations on the surface (310) by a movable platform (400) to which the support frame (102) is attached.

Images