System and method for inspecting surfaces

Abstract

We provide a surface inspection system for inspecting inspection surfaces. [Solution] A surface inspection system 100 for inspecting an inspection surface 306 includes a three-dimensional (3D) scanner 104 configured to scan the inspection surface 306 and acquire a point cloud of multiple points representing at least a local portion 308. The local portion 308 includes a non-defective region and a suspected region 158 that is at least partially surrounded by the non-defective region and potentially contains one or more defects. The surface inspection system 100 includes a processor 172 that establishes a reference plane based on multiple points in the non-defective region and excluding multiple points in the suspected region, the reference plane extending over the suspected region 158. The processor 172 identifies one or more characteristics of the shape of the point cloud relative to the reference plane within the suspected region 158.

Description

Technical Field

[0001]

[0001] This disclosure relates generally to inspection systems, and more particularly to systems and methods for inspecting surfaces.

Background Art

[0002]

[0002] In the commercial aircraft industry, inspection is an essential part of the manufacturing process to ensure the quality and safety of aircraft. For example, at various stages of airframe manufacturing, the outer skin is inspected to see if it conforms to surface contour requirements. One inspection method involves using straightedges and feeler gauges to measure bulges and depressions in the outer skin. Bulges and depressions in the outer skin can affect the aesthetics, aerodynamics, and / or structural integrity of the fuselage. Although generally effective, the straightedge method is a time-consuming process and has low reproducibility.

[0003]

[0003] A number of other inspection methods use scanners and analysis packages to measure and analyze three-dimensional surface data. However, with such methods, there is a possibility that the measured values may be biased and / or the inspection data may be misinterpreted according to surface contour requirements. In addition, existing surface inspection methods may delay the detection of surface non-conformities until the latter half of the manufacturing cycle after the fuselage is coated with a gloss finish that enhances visual detection. Unfortunately, detecting non-conformities in the latter half of the manufacturing cycle increases the cost and time required for rework and results in a delay in product delivery to customers.

[0004]

[0004] Thus, there is a need in the art for a system and method for inspecting surface contours in a timely manner with a high degree of reproducibility and accuracy and for properly interpreting inspection data according to surface contour requirements.

Summary of the Invention

[0005]

[0005] The aforementioned needs relating to surface inspection are addressed by the present disclosure. The present disclosure provides a surface inspection system for inspecting a local portion of an inspection surface of a structure. The surface inspection system includes a three-dimensional (3D) scanner configured to scan the inspection surface and acquire a point cloud of a plurality of points representing at least a local portion. The local portion includes a non-defective area and a suspected area at least partially surrounded by the non-defective area and potentially containing one or more defects. The surface inspection system further includes a processor configured to establish a reference plane based on a plurality of points in the non-defective area and excluding a plurality of points in the suspected area. The reference plane extends over the suspected area. The processor is also configured to identify one or more characteristics of the shape of the point cloud relative to the reference plane in the suspected area.

[0006]

[0006] Alternatively, a surface inspection system having a scan template, a 3D scanner, and a processor is also disclosed. The scan template is positionable relative to the inspection surface and defines the size and shape of a local portion of the inspection surface. The local portion includes a non-defective area and a suspected area that is at least partially surrounded by the non-defective area and potentially has one or more defects. The 3D scanner is configured to scan the local portion and acquire a point cloud of multiple points. The processor is configured to establish a reference plane based on multiple points in the non-defective area and excluding multiple points in the suspected area. The processor is also configured to identify one or more characteristics associated with the shape of the point cloud relative to the reference plane in the suspected area. The multiple characteristics include the position and height of the highest point, the position and depth of the lowest point, and the position and orientation of the slope gradient of the cross-sectional shape that passes through at least one of the highest and lowest points and is oriented parallel to the principal direction of the surface shape requirements of the local portion.

[0007]

[0007] In addition, a method for inspecting an inspection surface of a structure is disclosed. The method includes using a 3D scanner to scan the inspection surface and obtaining a point cloud of a plurality of points representing at least a local portion of the inspection surface. The local portion includes a non-defective area and a suspected area that is at least partially surrounded by the non-defective area and potentially contains one or more defects. The method also includes using a processor to establish a reference plane based on a plurality of points in the non-defective area and excluding a plurality of points in the suspected area. The reference plane extends over the suspected area. In addition, the method includes using a processor to identify one or more characteristics associated with the shape of the point cloud relative to the reference plane in the suspected area.

[0008]

[0008] The above features, functions, and advantages may be realized individually in various versions of this disclosure, or in combination in several other versions, which may be further detailed by referring to the following description and drawings.

[0009]

[0009] This disclosure can be better understood by referring to the following detailed description in relation to the accompanying drawings, which show preferred illustrative versions but are not necessarily drawn to scale. The drawings are illustrative and are not intended to limit the scope of this description or the claims. [Brief explanation of the drawing]

[0010] [Figure 1]

[0010] An example of an aircraft having a fuselage composed of barrel sections from end to end is shown. [Figure 2]

[0011] Figure 1 shows one of the barrel sections and illustrates an embodiment of a surface inspection system of the present disclosure having a three-dimensional (3D) scanner configured to scan an inspection surface of the fuselage and acquire a point cloud of multiple points representing at least a local portion of the inspection surface; further illustrates a computing device having a processor for processing the point cloud in a manner that establishes a reference plane and for identifying one or more characteristics relating to the shape of the point cloud relative to the reference plane. [Figure 3]

[0012] Figure 2 is an enlarged view of a portion of the torso, showing an embodiment of a template placement aid having a approximate center for centering a scan template over a suspicious area of ​​the inspection surface that potentially contains one or more defects. [Figure 4]

[0013] This shows a template placement aid installed on the inspection surface. [Figure 5]

[0014] This shows one example of a scan template placed on an inspection surface via a template placement aid. [Figure 6]

[0015] This shows the template placement aid, removed from the inspection surface and separated from the scan template. [Figure 7]

[0016] An embodiment of a 3D scanner configured as a cross-laser line scanner is shown, which is held by an operator to scan a scan template and a portion of a torso enclosed by the scan template. [Figure 8]

[0017] This document shows one embodiment of a 3D scanner configured as a structured optical scanner supported on a tripod. [Figure 9]

[0018] This is a plan view of an embodiment of a scan template having directional arrows for aligning the scan template in the main direction of the surface outline requirements of the fuselage, and also shows three alignment targets at each of the three corners of the template opening for defining the boundary (e.g., size and shape) of the template opening for defining the orientation of the point cloud relative to the fuselage. [Figure 10]

[0019] Figure 7 or Figure 8 is a perspective view of one embodiment of the point cloud of an inspection surface generated by a 3D scanner. [Figure 11]

[0020] Figure 10 shows an example of a display screen of a computing device showing a polygonized mesh representing the point cloud, and further shows an example of a table listing the positions of the scan template relative to the torso. [Figure 12]

[0021] It is an enlarged view of the table of the scan template position data in FIG. 7. [Figure 13]

[0022] It is a perspective view of the polygonized mesh in FIG. 11 showing a local part of the inspection surface defined by the scan template, and is shown separated from the remaining part of the polygonized mesh. [Figure 14]

[0023] It is a plan view of the polygonized mesh of the point cloud of a local part in FIG. 13. [Figure 15]

[0024] It is a perspective view of a local part of the polygonized mesh in FIG. 14 separated into two regions including a non-defective region and a suspicious region, and the suspicious region is surrounded by the non-defective region and potentially contains a plurality of points representing defects within the suspicious region of the inspection surface. [Figure 16]

[0025] It is a plan view of the polygonized mesh of the non-defective region in FIG. 15. [Figure 17]

[0026] It is a perspective view of the polygonized mesh of the non-defective region in FIG. 16. [Figure 18]

[0027] It is a perspective view of an orderly lattice of a plurality of points generated as a best fit for the arrangement of a plurality of points within a local part of the point cloud, and shows the positions of some surface discontinuities (e.g., fastener pits - shown as circles) within the inspection surface. [Figure 19]

[0028] It is a plan view of a local part of the point cloud excluding a plurality of points representing a suspicious region and excluding a plurality of points representing surface discontinuities (e.g., fastener pits) identified through the orderly lattice in FIG. 18. [Figure 20]

[0029] It is an enlarged view of a part of the point cloud in FIG. 19 showing an example excluding a plurality of points of one of the surface discontinuities shown in FIG. 19. [Figure 21]

[0030] It is a perspective view of the point cloud in FIG. 19. [Figure 22]

[0031] Figure 19 shows a reference plane generated as a polynomial fit of the point cloud, which is a perspective view of a reference plane that is continuous across the questionable region and across surface discontinuities. [Figure 23]

[0032] This is a plan view of the reference plane in Figure 22. [Figure 24]

[0033] This is a perspective view of the reference plane and the point cloud of a suspected region separated from the reference plane. [Figure 25]

[0034] This shows a point cloud of suspicious regions superimposed on the reference plane. [Figure 26]

[0035] An example of a heatmap of a suspicious region surrounded by a point cloud of non-defective regions is shown, where the heatmap (e.g., a difference map) graphically displays the contour lines of the surface of the suspicious region relative to a reference plane, and also shows the position and height of the highest point and the position and depth of the lowest point within the suspicious region. [Figure 27]

[0036] This example shows a heatmap of a suspicious region indicating the direction and magnitude of the steepest slope. [Figure 28]

[0037] An example of a cross-sectional profile is shown, along with the length (L) and depth (D) dimensions used to calculate the rate of change of the cross-sectional profile. [Figure 29]

[0038] This example shows a heatmap of the highest point, lowest point, and suspected area of ​​the steepest slope at three different lengths of incline. [Figure 30]

[0039] This example shows a heatmap that graphically displays the contour lines of the entire surface of a localized portion, including both suspected and non-defective areas. [Figure 31]

[0040] Figure 30 shows a heatmap illustrating the positions of cross-sectional profiles that pass through the highest and lowest points, and are aligned in the longitudinal and circumferential directions of the body. [Figure 32]

[0041] This is a plot of cross-sectional profiles of a suspected region passing through the highest point, aligned along the longitudinal direction of the torso (e.g., the x-axis). [Figure 33]

[0042] This is a plot of cross-sectional profiles of a suspicious region passing through the highest point, aligned along the circumferential direction (y-axis) of the torso. [Figure 34]

[0043] This is a plot of cross-sectional profiles of a suspected region passing through the lowest point, aligned along the longitudinal direction of the body (e.g., the x-axis). [Figure 35]

[0044] This is a plot of cross-sectional profiles of a suspected region passing through the lowest point, aligned along the circumferential direction (y-axis) of the torso. [Figure 36]

[0045] A heatmap of surface contours for the suspicious area is shown, along with a legend for the deviations, including the cross-hatching and corresponding thresholds specified from the surface outline requirements of the inspected surface. [Figure 37]

[0046] This is a flowchart of the steps involved in a method for inspecting the surface of a structure. [Modes for carrying out the invention]

[0011]

[0047] The figures presented in this disclosure represent various aspects of the presented version, and only the differences will be described in detail.

[0012]

[0048] Next, the various versions disclosed will be described more fully with reference to the accompanying drawings. These drawings do not show all of the various embodiments of this disclosure. In fact, several different versions may be provided, and these should not be construed as limiting to the versions described herein. Rather, these versions are provided so that this disclosure is comprehensive and so that the scope of this disclosure is fully conveyed to those skilled in the art.

[0013]

[0049] This specification includes references to “one example,” “an example,” and “some examples.” The examples in “one example,” “an example,” and “some examples” do not necessarily refer to the same example. Certain features, structures, or properties may be combined in any suitable manner consistent with this disclosure.

[0014]

[0050] As used herein, “comprising” is an open-ended term, and as used in the claims, it does not exclude additional structures or steps.

[0015]

[0051] As used herein, “configured to” means that various parts or components may be described or claimed to be “configured to” perform one or more tasks. In such contexts, “configured to” is used to imply a structure by indicating that the parts or components include a structure that performs one or more of those tasks during operation. Thus, it can be said that a part or component is configured to perform its task even when a particular part or component is not currently operating (e.g., it is not powered on).

[0016]

[0052] When used herein, a singular element or step following the word “a” or “an” should be understood not to necessarily exclude multiple such elements or steps. The expression “and / or” as used herein includes any and all combinations of one or more items from the relatedly listed items. Also, when used herein, “combination of ~” includes a combination having at least one of the relatedly listed items. In this case, the combination may further include other items not listed.

[0017]

[0053] When used herein, the expression “at least one of the listed items” means that one or more different combinations of the listed items may be used, and only one of each listed item may be required. In other words, “at least one of the listed items” means that any combination and any number of items from the listed items may be used, and not all of the listed items are required. An item may be a specific object, thing, or category.

[0018]

[0054] Referring now to the drawings illustrating various embodiments of the present disclosure, Figure 1 shows an aircraft 400. The aircraft 400 serves as an embodiment of a structure 300, in which a surface inspection system 100 and method of the present disclosure (Figure 37) is used to inspect the surface of the aircraft 400. The aircraft 400 includes a fuselage 402, a pair of wings 412, and a tail section 414 including a tail surface (such as a horizontal elevator and vertical fins). In the illustrated embodiment, the fuselage 402 consists of barrel sections 404 which are manufactured separately and then joined end to end by a plurality of section joints 406, as shown in Figures 1 and 2.

[0019]

[0055] Figure 2 shows one of the barrel sections 404 during an external surface inspection using the surface inspection system 100 of the present disclosure. In several embodiments shown, the barrel section 404 is constructed of metal, and its external surface is defined by a metal casing 302, which is coupled via fasteners 314 to lower structural members 304 such as longitudinal members 408 (Figure 7) and circumferential frames 410 (Figure 7). Although described in the context of the casing 302 of the fuselage 402, the surface inspection system 100 and method 500 may be implemented to inspect the surface of any of various different types of structures and is not limited to aircraft 400. For example, the surface inspection system 100 and method 500 may be implemented to inspect any large external surface such as ship hulls, wind turbine airfoils, power generation turbine blades, large-area antennas, sheet metal-covered buildings or roofs (e.g., hail damage inspection), and any of various different types of surfaces. In addition, the surface inspection system 100 may be used to inspect the surface of a structure made of any type of material, including metallic materials (e.g., aluminum) and / or non-metallic materials (e.g., carbon fiber composite materials).

[0020]

[0056] The surface inspection system 100 is configured to perform deviation analysis by measuring and characterizing a local portion 308 of the inspection surface 306 of the structure 300 (i.e., the surface being inspected) to determine whether a suspicious area 158 of a local portion 308 satisfies specified surface shape requirements (e.g., aerodynamic requirements) for the inspection surface 306. The surface inspection system 100 is configured to inspect the inspection surface 306 for depressions 210 (Figure 26), bulges 206 (Figure 26), bumps, ripples, and other surface characteristics that may potentially violate the surface shape requirements. The surface inspection system 100 can measure and characterize the inspection surface 306 of any shape and size, whether the inspection surface 306 is planar, simple, or complex. Advantageously, the surface inspection system 100 collects a sufficient amount of data to enable accurate evaluation of the inspection surface 306, in addition to providing context for root cause analysis of potential defects.

[0021]

[0057] As shown in Figure 2, the inspection system includes a data acquisition system 102 and a data analysis system 170. The data acquisition system 102 includes a three-dimensional (3D) scanner 104 configured to scan the inspection surface 306 and generate 3D data for surface reconstruction. In this regard, the 3D scanner 104 acquires a point cloud 150 (Figure 10) of a plurality of points 152 (Figure 10) representing at least a local portion 308 (Figure 7) of the inspection surface 306 (Figure 7). In addition to the 3D scanner 104, the data acquisition system 102 includes a scan template 120 (i.e., a field of view template) that can be positioned relative to the inspection surface 306. The scan template 120 has a frame-like template body 126 (Figure 7) having a template opening 130 (Figure 7) that defines the size and shape of the local portion 308 of the inspection surface 306. The local portion 308 preferably surrounds a suspected area 158 that potentially contains one or more defects. As described below, the scan template 120 helps to cut out an area of ​​the point cloud 150 surrounding a local portion 308. In addition, the scan template 120 allows the 3D scanner 120 to localize itself to the structure 300 (e.g., aircraft 400) (i.e., data localization).

[0022]

[0058] The data analysis system 170 includes a processor 172 (shown in Figure 2) that is communicatively coupled to the 3D scanner 104. In the embodiments of Figures 1 and 2, the processor 172 is contained within a computing device 174, such as a laptop computer 176, which may be positioned near the 3D scanner 104. The processor 172 processes the 3D data in a manner that enables proper interpretation of the surface properties of the inspection surface 306, including identifying any surface defects within a suspicious area 158. In this disclosure, surface defects are interchangeably referred to as defects, nonconformities, anomalies, irregularities, and out-of-limit conditions. The processor 172 is configured to establish a reference plane 164 (Figures 22 and 23) based on the point cloud 150, as will be described in more detail below, and to identify one or more properties of the shape of the point cloud 150 relative to the reference plane 164 (e.g., surface defects).

[0023]

[0059] In the surface inspection system 100 of this disclosure, a data acquisition system 102 (Figure 2) is directly linked to a data analysis system 170 (Figure 2). Thus, when a 3D scanner 104 acquires a point cloud 150 (Figure 10) of the inspection surface 306 of the structure 300, the processor 172 autonomously processes the point cloud 150 data through a framework that enables appropriate interpretation, taking into account specified surface shape requirements for the inspection surface 306, as described below. The surface inspection system 100 also includes a data reporting system in which the processor 172 autonomously generates a report 200 (Figures 26-36) containing inspection results, including positional data of the inspection surface 306. The inspection results are autonomously stored in a data library (not shown) to monitor the location of defects (e.g., dents 210 and bulges 206) identified using the surface inspection system 100, and are continuously incorporated into a digital thread (e.g., a digital twin of the structure – not shown). Advantageously, the digital thread can be reviewed throughout the product lifecycle. Root cause analysis, processing, and repair actions are also stored in a digital thread for quick access, facilitating automated root cause identification of defects (e.g., via machine learning) in future versions of Structure 300.

[0024]

[0060] In Figure 2, the 3D scanner 104 is a gripping scanner operated by a human (i.e., an operator), and the scan template 120 is a physical template 122 attached to the inspection surface 306 (e.g., barrel section 404). To begin the process of inspecting the inspection surface 306, the operator visually identifies a suspicious area 158 in a localized area of ​​the inspection surface 306 that potentially contains one or more defects. For example, the operator may visually observe a depression 210 and / or bulge 206 within the inspection surface 306 that potentially violates the specified surface shape requirements for the inspection surface 306. In the illustrated embodiment, the suspicious area 158 has a roughly 8 x 8 inch square shape. However, the suspicious area 158 can be any of several other shapes and is not limited to orthogonal shapes. Instead of visually identifying the suspicious area 158, the suspicious area 158 on the inspection surface 306 can be identified using a 3D scanner 104 and a probabilistic threshold or a surface change rate threshold. Identifying the suspicious area 158 using thresholds typically results in an irregular shape (not shown) that defines the boundary of the suspicious area 158.

[0025]

[0061] In the embodiment shown in Figure 2, after identifying the suspicious area 158, the scan template 120 is attached to the inspection surface 306 to initiate the process of evaluating the surface properties of the suspicious area 158 and determining whether the initially observed depressions 210 and / or bulges 206 actually violate the surface shape requirements. The scan template 120 is attached to the inspection surface 306 in such a way that the template opening 130 surrounds the suspicious area 158. In this regard, the template opening 130 of the scan template 120 is larger than the suspicious area 158, providing a context for measuring the surface properties within the suspicious area 158, enabling proper interpretation of the surface properties and allowing inspection approval to be made.

[0026]

[0062] Referring to Figures 3 to 6, an embodiment of a template placement aid 132 for positioning a physical template 122 (i.e., a scan template 120) on the inspection surface 306 at the location of a suspected area 158 is shown. In the illustrated embodiment, the template placement aid 132 has an orthogonal shape complementary to the orthogonal shape of the template opening 130 in the physical template 122. The template placement aid 132 includes a hole approximately at its center 160 for centering the template placement aid 132 on the suspected area 158. This hole is identified by X. In the illustrated embodiment, the corners of the template placement aid 132 include an opening 134 for attaching the template placement aid 132 to the inspection surface 306 via adhesive tape 136, as shown in Figures 4 to 5. However, any of several other mechanisms may be implemented to detachably attach the template placement aid 132 to the inspection surface 306.

[0027]

[0063] Figures 5 and 6 show a physical template 122 attached to the inspection surface 306 via a template placement aid 132. The physical template 122 can be detachably held in place via adhesive tape (not shown) on the edges of the physical template 122, or via several other means such as double-sided tape. Once the physical template 122 is positioned to align with the template placement aid 132 and fixed to the inspection surface 306, the template placement aid 132 can be removed as shown in Figure 6, exposing a local portion 308 of the inspection surface 306 surrounded by the template body 126.

[0028]

[0064] Referring to Figure 7, an embodiment of a physical template 122 fixed to the inspection surface 306 is shown, defining a localized portion 308 surrounded by the template body 126. As described above, the physical template 122 is configured to be detachably attached to the inspection surface 306 before scanning via the 3D scanner 104. The physical template 122 may be formed of a flexible material that can conform to the shape of the inspection surface 306. For example, the physical template 122 may be a sheet of a non-metallic material (e.g., cardboard, plastic, paper, etc.) or a sheet of a metallic material (e.g., aluminum, etc.).

[0029]

[0065] In the illustrated embodiment, the template opening 130 is orthogonal in shape (e.g., rectangular, square), but alternative shapes such as round (circular, oval, etc.) may be provided. The template opening 130 defines the shape and size of a local portion 308. The local portion 308 consists of two regions: a suspected region 158 and a non-defective region 154 surrounding the suspected region 158. The non-defective region 154 is larger than the suspected region 158 to allow for proper evaluation of the inspection surface 306. In one embodiment, the area of ​​the non-defective region 154 is 1.5 to 20 times larger than the area of ​​the suspected region 158. However, in several other embodiments, the non-defective region 154 may be larger than 20 times larger than the suspected region 158.

[0030]

[0066] As described above, the non-defective region 154 is used to provide context to the suspected region 158. In this regard, the non-defective region 154 is not necessarily nominal in the following respect: namely, the non-defective region 154 of the as-built structure 300 may not precisely correspond to the as-designed configuration of the structure 300 for various reasons. For example, gravity or temperature changes on the as-built structure 300 are typically not represented in the digital version of the structure 300 (e.g., CAD model). However, in this disclosure, the non-defective region 154 of the as-built structure 300 is assumed to be generally free of significant surface defects and is therefore used to generate a reference plane 164 (Figures 22-23) on which multiple points 152 (Figures 24-25) within the suspected region 158 are compared.

[0031]

[0067] In some embodiments, the surface inspection system 100 may be used to define a suspicious area 158 as an alternative to visually identifying the suspicious area 158. For example, once a scan template 120 is attached to the inspection surface 306, a 3D scanner 104 may scan the inspection surface 306 and generate a point cloud 150. The processor 172 receives the point cloud 150 data from the 3D scanner 104 and designates the central portion 156 of the point cloud 150 as the suspicious area 158. For example, the processor 172 may designate a smaller area of ​​the point cloud 150 as the suspicious area 158, which is presumed to contain multiple points 152 representing potential defects. The presumption that the central portion 156 of the point cloud 150 contains potential defects may be based on historical data, such as inspection reports from previously manufactured versions of the same structure 300. Alternatively, or further, the method of designating the central portion 156 of the point cloud 150 as a suspicious area 158 is used when it is difficult to visually detect defects, such as when the inspection surface 306 has low reflectivity or is covered with a low-reflectivity coating such as a primer or anti-corrosion coating.

[0032]

[0068] In several other embodiments, the processor 172 may designate an area of ​​the point cloud 150 containing points 152 that deviate from a deviation threshold (not shown) relative to the remaining points 152 in the point cloud 150 as a suspicious region 158. For example, as described above, the processor 172 may perform calculations on the point cloud 150 based on a stochastic threshold or a surface change rate threshold and designate a portion of the point cloud 150 containing points 152 that deviate from the stochastic threshold or surface change rate threshold as a suspicious region 158. In several such embodiments, the shape of the point cloud 150 is typically defined by the outline (not shown) of the inspection surface 306.

[0033]

[0069] In addition to defining the size of the non-defective area 154 based on the suspected area 158, the non-defective area 154 may also be sized and / or shaped to encompass an area of ​​the structure 300 including at least one substructure member 304 (Figure 7). Sizing the non-defective area in this manner may provide insight into whether such a substructure member 304 contributes to the presence or formation of a surface defect potentially contained within the suspected area 158. In some embodiments, the substructure member 304 may be an internal frame supporting the outer panel 302. For example, the non-defective area 154 may include an area of ​​the structure 300 supported by at least one substructure member 304. More preferably, the non-defective area 154 encompasses substructure members 304 located on both sides of the suspected area 158, or the closest substructure members 304 on the two sides of the suspected area 158 (these are not necessarily on both sides of the suspected area 158).

[0034]

[0070] In one embodiment of the fuselage 402 in Figure 7, the sublayer structural members 304 are longitudinal members 408 or circumferential frames 410. Several other embodiments of the sublayer structural members 304 include, but are not limited to, the lower ribs (not shown) and / or spars (not shown) of a main wing (Figure 1), or flight control surfaces such as ailerons, flaps, horizontal stabilizers, elevators, vertical fins, and rudders. In some embodiments, the size of the non-defective area 154 may be based on the distance from the suspected area 158 to one or more sublayer structural members 304 (which may be contributing to the defect) within the non-defective area 154. By sizing the template opening 130 to encompass one or more sublayer structural members 304, it may be easier to identify the effect of surface defects on one or more properties of the structure 300, such as strength properties, aerodynamic properties, or several other properties.

[0035]

[0071] In Figure 7, the 3D scanner 104 is a cross-laser scanner 106. This scanner 106 is grasped by an operator (e.g., a human inspector) and operated in a manner for scanning a physical template 122 and a localized portion 308 of the inspection surface 306 surrounded by the physical template 122. In several of the illustrated embodiments, the cross-laser scanner 106 is the T-Scanhawk 2™ scanner, commercially available from the Zeiss Group in Oberkochen, Germany. Although illustrated as a grasped scanner, the 3D scanner 104 disclosed herein may be supported by a fixed platform (e.g., a tripod 110 – Figure 8) or a movable platform (not shown). For example, the 3D scanner 104 may be supported by a robot (not shown) or gantry system (not shown) configured to move the 3D scanner 104 while scanning the inspection surface 306 for data acquisition.

[0036]

[0072] Figure 8 shows one embodiment of a 3D scanner 104 configured as a structured optical scanner 108 mounted on a tripod 110. One embodiment of the structured optical scanner 108 is the ATOS, which is commercially available from the Zeiss Group in Oberkochen, Germany. TMIt is a scanner. However, the 3D scanner 104 may be provided in any of a variety of configurations that can digitize the inspection surface 306, and is not limited to a laser scanner. For example, the 3D scanner 104 may be provided in a configuration that uses LiDAR or photogrammetry to generate a point cloud 150 of the inspection surface 306.

[0037]

[0073] In Figure 8, the template body 126 of the physical template 122 includes a unique constellation of retroreflective targets 128, which may be required for data localization when using a particular type of 3D scanner 104. The retroreflective targets 128 allow for the alignment and merging of multiple scans of the same area but taken from different orientations of the 3D scanner 104 when scanning the inspection surface 306.

[0038]

[0074] In Figures 7-9, the scan template 120 includes three alignment targets 142, each positioned at three of the corners of the template opening 130. The alignment targets 142 outline the boundaries (e.g., size and shape) of the template opening 130 (i.e., the local portion 308). In addition, the alignment targets 142 orient the scan (e.g., the point cloud 150) in a way that is easier to observe. For example, the three alignment targets 142 enable the processor 172 to display the point cloud 150 aligned in the x and y directions on a display screen (on the display screen 178 of the computing device 174 - Figures 7-8), as shown in Figure 11 and 178.

[0039]

[0075] In Figures 7-9, the scan template 120 includes an alignment feature 144 in the form of a directional arrow 146 for aligning the scan template 120 with a principal direction 148 associated with the surface outline requirements of the inspection surface 306. The principal direction 148 refers to the main or primary direction, along which the surface outline requirements are specified. For example, in the fuselage 402 of Figures 1-2, the principal direction 148 coincides with the centerline of the fuselage 402 and is the general direction of airflow over the aircraft 400 in flight. The directional arrow 146 facilitates the manual placement (via the operator) of the scan template 120 onto the inspection surface 306 of the fuselage 402, where the directional arrow 146 points forward. However, the alignment feature 144 may be provided in any of a variety of alternative configurations and is not limited to a directional arrow 146. Furthermore, the alignment feature 144 may be oriented in any of a variety of orientations and is not limited to being oriented parallel to the principal axis of the structure 300.

[0040]

[0076] While the above-described embodiments of the scan template 120 refer to a physical template 122, the scan template 120 may alternatively be provided as a projected template 124 (Figure 7) projected onto the inspection surface 306 by a light projector (not shown). The projected template 124 may have the same size, shape, and configuration as the physical template 122. For example, the projected template 124 may include projected elements (not shown), such as the outer perimeter outline and the outline of the template opening 130, which are projected onto the inspection surface 306 by the light projector. Furthermore, the projected elements may include one or more alignment features 144 (e.g., directional arrows 146) and / or one or more alignment targets 142. The above-described projected elements may be incorporated into the point cloud 150 of the inspection surface 306 while being scanned by the 3D scanner 104 to enable post-processing (e.g., cutting) of multiple parts of the point cloud 150 located outside the local portion 308.

[0041]

[0077] As an alternative to a physical template 122 or a projected template 124 applied to the inspection surface 306, the scan template 120 may be a virtual template (not shown) incorporated into the software of the processor 172. For example, after the 3D scanner 104 scans the inspection surface 306 and generates a point cloud 150, the processor 172 may preprocess certain portions of the point cloud 150. In this regard, the processor 172 may perform a rough review of the inspection surface 306 to locate areas (i.e., suspicious areas 158) where curvature (e.g., depressions, bulges) exceeds a predetermined threshold (e.g., a stochastic threshold, a rate of change threshold) as described above. If a suspicious area 158 is identified during the rough review process, the processor 172 centers the suspicious area 158 within a larger local portion 308 surrounding the suspicious area 158, and then the processor 172 may cut out, remove, or ignore multiple portions of the point cloud 150 located outside the local portion 308 before performing the local surface fitting and deviation analysis described below. Advantageously, the virtual template adapts to the structure 300 under inspection, and in that sense, the size, shape, and / or position of the local portion 308 of the inspection surface 306 are easily defined and adjusted via the software on which the processor 172 operates.

[0042]

[0078] Referring to Figure 10, one embodiment of a point cloud 150 of an inspection surface 306 is shown, such as that generated by a 3D scanner 104, including the cross-laser scanner 106 in Figure 7 or the structured optical scanner 108 in Figure 8. Figure 10 shows the surface of the scan template 120, including the outline of the template opening 130, the outer periphery of the template body 126, and the alignment targets 142 at the three corners of the template opening 130. In addition, several points 152 in the point cloud 150 capture features of the inspection surface 306 not covered by the scan template 120, including the fastener 314.

[0043]

[0079] Referring to Figure 11, an embodiment of the display screen 178 of a computing device 174 is shown, displaying a polygonized mesh 166 representing the point cloud 150 of Figure 10. When the processor 172 receives the point cloud 150 data from the 3D scanner 104 (Figure 7), the processor 172 autonomously converts the point cloud 150 into a polygonized mesh 166. The polygonized mesh 166 consists of multiple edges (not shown) that interconnect multiple points 152 of the point cloud 150 to form multiple polygon elements such as triangular or quadrilateral elements. In this disclosure, the terms polygonized mesh 166 and point cloud 150 are interchangeable.

[0044]

[0080] Figure 12 shows a table of template location data 138 from Figure 11, which identifies the location of the scan template 120 on the structure 300 for data localization of 3D scan data on the structure 300. In the illustrated embodiment, the template location data 138 includes terminology typical of aircraft production, including line number (a unique aircraft identifier on the production line), station number (the longitudinal position of the aircraft), butt line (the transverse position), and waterline (the longitudinal position). However, any of several alternative terms may be used for the template location data 138 (i.e., data localization) of the scan template 120 on the structure 300, and is not limited to the above-mentioned terms used in aircraft production. In another embodiment, the scan template 120 data may be provided in x, y, z coordinates relative to a known reference origin 140 (Figure 1) or a hardpoint on the aircraft 400. In addition to the template location data 138, any of various other types of metadata may be included. For example, the operator's name, date, and temperature may be included along with the template location data 138.

[0045]

[0081] In some embodiments of the surface inspection system 100, template position data 138 and other metadata may be manually entered, such as by the operator of the 3D scanner 104. Manual entry relies on the operator's precise knowledge of the placement of the structure 300. As an alternative to manual entry, a template positioning system (not shown) may capture the position of the scan template 120 relative to the structure 300 before, during, or after the inspection surface 306 is scanned by the 3D scanner 104.

[0046]

[0082] In one embodiment, the template positioning system may be provided as an indoor tracking system (not shown). The indoor tracking system tracks the position of the scan template 120 relative to the structure 300 and automatically records template position data 138 when the scan is initiated. In one embodiment, the indoor tracking system may be a local positioning system. In this case, a small transceiver (not shown) is attached to the scan template 120 and communicates with a number of transmitters (not shown) installed at fixed locations in the surrounding environment of the structure 300. In another embodiment, the template positioning system may consist of a network of infrared cameras (not shown) and / or ultrasonic beacons (not shown) attached to the scan template 120 in the surrounding environment to track and record their relative positions to the structure 300. In yet another embodiment, the template positioning system may include one or more laser trackers (not shown) in the surrounding environment and one or more spherical reflectors attached to the scan template 120. When the scan template 120 is placed on the structure 300, the laser trackers emit laser beams. The laser beam is reflected back to the laser tracker, which allows the laser tracker to pinpoint the three-dimensional position of the spherical reflector of the structure 300. In another embodiment, one or more laser trackers or videogrammetry systems can monitor the position of the 3D scanner 104 relative to the structure 300 as the 3D scanner 104 moves around the structure 300 during the scanning process. When the 3D scanner 104 is mounted on an articulated arm (not shown) or a robotic device (not shown), data localization can be achieved via laser tracking of the position of the 3D scanner 104 or via kinematic tracking using the known shape and joint angles of the (one or more) arms.

[0047]

[0083] Regardless of the method used to input the template position data 138 for data localization of the scan data, it is important to know the position of the scan template 120 on the structure 300 (e.g., aircraft 400). This allows surface defects to be located and corrected on the structure 300 if they are discovered during the analysis of the inspection data. Furthermore, the template position data 138 is important because the surface shape requirements for one location on the structure 300 may differ from those for several other locations. In addition, knowledge of the template position data 138 is important for properly recording and / or maintaining the aforementioned digital thread of the structure 300 as it is constructed.

[0048]

[0084] Referring to Figure 13, the polygonized mesh 166 of Figure 11 is shown, and a localized portion 308 of the inspection surface 306 is shown separated from the rest of the polygonized mesh 166. As shown in the drawing, the localized portion 308 is defined by a template opening 130 of the scan template 120. In the illustrated embodiment, a fastener 314 in the outer plate 302 of the inspection surface 306 is shown relative to a reference in the polygonized mesh 166 of the localized portion 308. Figure 14 is a plan view of the polygonized mesh 166 of the localized portion 308 of Figure 13, and Figure 13 shows the template opening 130 that defines the size and shape of the localized portion 308.

[0049]

[0085] Referring to Figure 15, a local portion 308 of the polygonized mesh 166 of Figure 14 is shown, separated into two regions: a non-defective region 154 and a suspected region 158. Figures 16-17 show the polygonized mesh 166 of the non-defective region 154. The suspected region 158 is surrounded by the non-defective region 154 and contains several points 152 that potentially represent multiple defects within the suspected region 158 of the inspection surface 306. As described above, the non-defective region 154 is presumed to be substantially free of significant defects.

[0050]

[0086] Figure 18 shows an embodiment of an orderly grid 162 of multiple points 152 within a local portion 308 that best fits the loosely structured arrangement of multiple points 152 within the point cloud 150 of Figure 17. As will be described later, the orderly grid 162 is used to identify surface discontinuities 310 within the inspection surface 306. Figure 18 shows the locations of fasteners 314 within the outer plate 302 of the inspection surface 306, some of which are identified as surface discontinuities 310 taking the form of fastener pits 316 (shown as circles).

[0051]

[0087] Figure 19 is a plan view of the point cloud 150 of a local portion 308 showing a surface discontinuity 310 (e.g., fastener pit 316) identified via the orderly grid 162 of Figure 18. Figure 20 is a magnified view of the portion of the point cloud 150 of Figure 19 showing one of the surface discontinuities 310. Although not shown in Figures 19-21, the point cloud 150 from which the reference plane 164 (Figure 22) is generated ignores scan noise (e.g., sensor noise) generated during the process of scanning the inspection surface 306 with the 3D scanner 104.

[0052]

[0088] Figures 22-23 show one embodiment of a reference plane 164 based on multiple points 152 of the point cloud 150 shown in Figure 21. As described above, the processor 172 establishes the reference plane 164 based on multiple points 152 representing non-defective regions 154, and excluding multiple points 152 representing suspicious regions 158. In some embodiments, scan noise is ignored when establishing the reference plane 164. In some embodiments, surface discontinuities 310 are ignored when establishing the reference plane 164. The reference plane 164 is a mathematical function (i.e., a non-discrete representation of the surface) that extends across the suspicious regions 158 and the surface discontinuities 310. In other words, the point cloud 152 representing the suspicious regions 158 and the surface discontinuities 310 are not used to establish the reference plane 164. However, as shown in Figures 21 and 22, the reference plane 164 is continuous across the suspicious region 158 and the surface discontinuity 310.

[0053]

[0089] Prior to the establishment of the reference plane 164, the processor 172 is configured to identify a number of points 152 in a point cloud 150 representing surface discontinuities 310 within the inspection surface 306. Surface discontinuities 310 can be described as significant deviations or abrupt changes in the expected shape within the inspection surface 306. Significant deviations may include as-design discontinuities in the inspection surface 306, such as lap joints 312 (Figure 2) and section joints 406 (Figure 2) of the structure 300. A lap joint 312 is where two outer panels 302 overlap each other (e.g., Figure 2), resulting in a sharp step within the inspection surface 306. A section joint 406 is the interface between two end-to-end barrel sections 404 of the fuselage 402, as shown in Figure 2. Significant deviations may also include as-built discontinuities, such as fastener pits 316 (Figures 17-18) within the inspection surface 306. The fastener 314 pit can be described as a relatively deep pocket surrounding the rivet head. As can be understood, the surface discontinuity 310 may include any of various different types of surface features in which there is a significant deviation in the expected shape of the inspection surface 306, and is not limited to overlapping seams 312, section seams 406, or fastener pits 316.

[0054]

[0090] The process of identifying surface discontinuities 310 includes generating the aforementioned orderly grid 162 (Figure 18) of uniformly spaced points 152 by interpolating a loosely structured set of points 152 within a point cloud 150 representing the entire local segment 308, which includes the non-defective region 154 and the suspected region 158. The orderly grid 162 consists of points 152 arranged in uniformly spaced rows and columns with relatively fine spacing, such as 1 / 8 inch intervals. The processor 172 also generates a baseline surface (i.e., an initial reference plane - not shown) as a polynomial surface fit of all points 152 in the raw point cloud 150 data (Figure 17) of the local segment 308, which includes the non-defective region 154 and the suspected region 158. The processor 172 then compares each point of the orderly grid 162 against the baseline surface to identify the highest value for each point of the orderly grid 162 relative to the baseline surface. The highest value at each point is converted to a grayscale pixel intensity value between 0 and 255. The conversion of each point within an orderly grid 162 to grayscale intensity values ​​results in a localized portion 308 image (not shown).

[0055]

[0091] By converting the point cloud 150 of the local portion 308 into an image, the processor 172 can perform any of a variety of well-known image processing techniques to detect surface discontinuities 310. For example, the processor 172 may use OpenCV edge detection (e.g., Canny edge detection) and outline mapping (i.e., outline detection) methods to identify a plurality of points 152 that have the sharpest or most abrupt changes in pixel intensity. Abrupt changes in pixel intensity represent sharp deviations or locations of surface discontinuities 310 within the inspection surface 306. These corresponding points 152 are then ignored when the locations of the plurality of points 152 in the image representing the surface discontinuities 310 are traced back to the corresponding plurality of points 152 in the point cloud 150 of the local portion 308 to establish a reference surface 164.

[0056]

[0092] The process described above, which compares multiple points 152 within an orderly grid 162 to a baseline surface, also facilitates the identification of multiple points 152 within a point cloud 150 that represent scan noise. Scan noise is described as random deviations at some heights among multiple points 152 within the point cloud 150 and may be the result of environmental factors such as lighting, vibration, and reflectivity, as well as / or characteristics of the 3D scanner 104 such as calibration problems and sensor resolution. Advantageously, the exclusion of scan noise and multiple points 152 representing surface discontinuities 310 when establishing the reference plane 164 improves the accuracy with which the reference plane 164 is constructed. Improving the accuracy of the reference plane 164 helps avoid false positives when analyzing areas 158 suspected of having surface defects, as described below.

[0057]

[0093] The processor 172 establishes the reference plane 164 described above by performing a surface fit on multiple points 152 in the point cloud 150 of non-defective regions 154, only after ignoring scan noise and surface discontinuities 310 (i.e., not on the suspicious regions 158). The reference plane 164 can be established using any of various surface fitting techniques, such as polynomial surface fitting. In the above-described embodiment of the fuselage 402 (Figures 1-2), the reference plane 164 can be established using a cubic polynomial surface fitting, which is used with the aircraft surface having cubic behavior depending on the load of the aircraft 400. However, the reference plane 164 of the structure 300 having different order behaviors (e.g., cubic or cubic behavior) can be established using different mathematical functions, such as polynomial fits of different orders (e.g., cubic or cubic polynomial surface fittings). As an alternative to polynomial surface fitting, the reference plane 164 can be established using any of various surface filtering techniques. For example, the reference plane 164 can be established using Gaussian, robust Gaussian, or surface normal Gabor filtering.

[0058]

[0094] The above-described embodiment for generating a reference plane 164 of a local portion 308 ignores the questionable region 158, surface discontinuities 310, and scan noise; however, in several other embodiments not shown, the reference plane 164 of the local portion 308 may be generated by ignoring only the questionable region 158, without ignoring scan noise and / or surface discontinuities 310.

[0059]

[0095] Next, referring to Figures 24 and 25, Figure 24 shows a reference plane 164 and a point cloud 150 of a suspicious region 158 separated from the reference plane 164. Figure 25 shows the point cloud 150 of the suspicious region 158 superimposed on the reference plane 164 and illustrates the operation of the processor 172 in comparing the point cloud 150 of the suspicious region 158 with the reference plane 164 as a means of identifying one or more characteristics of the shape of the point cloud 150 of the suspicious region 158 with respect to a portion of the reference plane 164 within the suspicious region 158. The comparison of the point cloud 150 of the suspicious region 158 with respect to the reference plane 164 can be performed in a similar manner to the process described above of comparing an orderly grid 162 of a local portion 308 with a baseline surface of the local portion 308.

[0060]

[0096] Referring to Figure 26, an embodiment of a difference map, referred to herein as a heatmap 202 of the suspected region 158, is shown, which is generated as a result of comparing the point cloud 150 (i.e., raw point cloud data) of the suspected region 158 with a reference plane 164 within the suspected region 158. In the illustrated embodiment, the heatmap 202 of the suspected region 158 is surrounded by the reference plane 164 of the non-defective region 154. In Figure 26, the heatmap 202 and the reference plane 164 are shown superimposed on a grid to facilitate the identification of the location of surface characteristics identified within the heatmap 202. The heatmap 202 shows contour lines 204 of the surface of the suspected region 158 with respect to the reference plane 164. On the right side of the image is a deviation legend 220, which includes cross-hatching and the corresponding deviation values ​​of the suspected region 158 with respect to the reference plane 164.

[0061]

[0097] The processor 172 is configured to generate a heatmap 202 and identify one or more characteristics of the shape of the suspicious region 158 relative to the reference plane 164. For example, when comparing the point cloud 150 (i.e., raw scan data) of the suspicious region 158 with the reference plane 164, the processor 172 is configured to identify the position and height of the highest point 208 within the suspicious region 158. The highest point 208 is located within a bulge 206 within the suspicious region 158. In the illustrated embodiment, the highest point 208 is +0.0251 linear units above the reference plane 164. Alternatively or further, the processor 172 is configured to identify the position and depth of the lowest point 212 within the suspicious region 158. The lowest point 212 is located within a depression 210 within the suspicious region 158. In the illustrated embodiment, the lowest point 212 is -0.0217 linear units below the reference plane 164.

[0062]

[0098] Referring to Figure 27, an embodiment of heatmap 202 is shown, having the same surface contours 204 as heatmap 202 in Figure 26. Instead of showing the highest point 208 and lowest point 212 as in Figure 26, Figure 27 shows the location and orientation of the steepest slope gradient 214 (i.e., the maximum surface change rate) within the suspicious region 158, as identified by processor 172. In the illustrated embodiment, the steepest slope gradient 214 is expressed as the ratio of length L to depth D, providing a numerical measure of the steepness of the slope.

[0063]

[0099] Conveniently, identifying the steepest slopes facilitates the analysis of the shape of the questionable region 158 regarding its impact on the structure 300. For example, knowledge of the location and magnitude of the surface slopes can be an important characteristic, as the slopes can affect the magnitude of mechanical stress and / or strain in the material of the structure 300 in a static or unloaded state (e.g., no passengers or cargo inside the fuselage 402). Furthermore, knowledge of the location and magnitude of the surface slopes can help determine how the surface shape affects the behavior of airflow over the surface (i.e., aerodynamics). In addition, surface slopes can affect the aesthetics and visual appeal of the surface, which can impact customer confidence in the quality of the delivered product.

[0064] [000100] As shown in Figure 28, depth D is the change in elevation between two positions or points and is measured in a direction parallel to the z-direction of the area of ​​doubt 158 ​​(i.e., "vertical" distance). Length L is the distance between two points measured in a direction parallel to the local x-direction (i.e., "horizontal" distance). As shown in Figures 27, 29, 30, and 36, as an alternative to expressing the steepest slope gradient 214 as an L / D ratio, the steepest slope gradient 214 may be expressed as the ratio of depth D to length (i.e., D / L).

[0065] [000101] In this disclosure, the processor 172 is configured to identify the steepest slope gradient 214 within the suspicious region 158 by calculating the value of the deviation between the raw point cloud 150 data (Figures 24-25) and the reference plane 164 (Figures 24-25). The value of the deviation is converted to local z coordinates and the partial derivatives of the z coordinate with respect to the x and y coordinates of each point are calculated. Using the components in each direction, the processor 172 identifies the steepest slope gradient 214 which has the smallest L / D ratio in this embodiment. A predetermined length L may be specified to improve the efficiency of the algorithm in finding the steepest slope gradient 214 and to limit the influence of scan noise in embodiments where the predetermined length L is relatively small.

[0066] [000102] In the embodiment of Figure 27, the steepest slope 214 has an L / D of 45.11 for a given length of 1.0 linear unit. As can be understood, if different values ​​are given for length L (e.g., L = 0.5 linear units), the steepest slope 214 will occur at different points within the questionable region 158, and will have different orientations and different values ​​for the ratio L / D. Figure 29 shows a heatmap 202 similar to Figures 26-27, and also shows the locations of three steepest slopes 214 for three different given lengths L. More specifically, in addition to showing the above-mentioned steepest slope 214 of L / D of 45.11 for a length L of 1.0 linear unit, Figure 29 also shows the location and orientation of the steepest slope 214 (i.e., L / D of 45.70) for a given length L of 0.5 linear units, and the steepest slope 214 (i.e., L / D of 38.55) for a given length L of 1.5 linear units.

[0067] [000103] Figure 30 shows an example of the overall heatmap 202 of a local portion 308, including a suspected area 158 (Figure 29) and a non-defective area 154 (Figure 29). As described above, the heatmap 202 is a difference map showing the contour lines 204 of the surface. This map graphically shows the deviation of the point cloud 150 (i.e., raw scan data - Figure 25) of the local portion 308 relative to a reference plane 164 (Figure 25) of the local portion 308. The deviation legend 220 on the right side of Figure 30 shows the deviation values ​​and cross-hatching patterns corresponding to the cross-hatching in the heatmap 202 of the local portion 308. As described below, the heatmap 202 over the suspicious area 158 and / or over the entire local portion 308 helps identify the root cause of surface properties (e.g., surface defects) of the inspection surface 306 and further provides a means to understand the impact of surface properties on the structure 300, such as from a performance standpoint (e.g., strength, aerodynamics, etc.) and / or from an aesthetic standpoint (e.g., visual appeal).

[0068] [000104] In some embodiments of the surface inspection system 100, the processor 172 is configured to generate a cross-sectional profile 218 of the suspicious region 158. This profile 218 is normalized with respect to a reference plane 164. For example, Figure 31 is a heatmap 202 of Figure 30, further showing the location of the cross-sectional profile 218 passing through the highest point 208 and lowest point 212 within the suspicious region 158. This profile 218 is shown in Figures 32 to 35. Figure 32 shows a plot of the cross-sectional profile 218 of the suspicious region 158 passing through the highest point 208, which is a plot of the cross-sectional profile 218 aligned along the longitudinal direction (i.e., parallel to the x-axis) of the fuselage 402. Figure 33 shows a cross-sectional profile 218 passing through the highest point 208 of the suspicious region 158, which is a cross-sectional profile 218 aligned along the circumferential direction (i.e., parallel to the y-axis) of the fuselage 402. Figure 34 is a plot of the cross-sectional profile 218 of the suspicious region 158 passing through the lowest point 212, and is a plot of the cross-sectional profile 218 aligned in the longitudinal direction of the fuselage 402. Figure 35 shows the cross-sectional profile 218 passing through the lowest point 212 of the suspicious region 158, which is the cross-sectional profile 218 aligned in the circumferential direction of the fuselage 402.

[0069] [000105] As described above, the scan template 120 may include alignment features 144, such as a directional arrow 146 for aligning the scan template 120 with the principal direction 148 of the specified surface outline requirements relative to the inspection surface 306. In some embodiments, the processor 172 may identify the steepest slope gradient 214 in the cross-sectional profile 218 parallel to the principal direction 148 (Figure 2). This is similar to the cross-sectional profile 218 described above, shown in Figures 32 and 34, which passes through the highest point 208 and lowest point 212 of the suspected area 158, respectively. The cross-sectional profile 218 can facilitate the identification of an influence on the behavior of airflow over the inspection surface 306 or on the strength characteristics of the structure 300. The cross-sectional profile 218 may be generated at any location within the inspection surface 306 and is not limited to passing through the highest point 208 and lowest point 212 of the suspected area 158. In addition, the cross-sectional profile 218 can be oriented in any direction and is not limited to being parallel to the main direction 148 of the surface outline requirements.

[0070] [000106] In some embodiments, the cross-sectional profile 218 may be generated based on the local z-direction difference between the point cloud 150 (i.e., raw scan data) and the reference plane 164. This effectively normalizes the cross-sectional data. Alternatively, the cross-sectional profile 218 may be normalized when generating the point cloud 150, taking into account the out-of-plane curvature and / or out-of-plane inclination of the inspection surface 306 with respect to the pointing direction of the 3D scanner 104, in contrast to the inspection surface 306 which is planar and / or perpendicular (i.e., perpendicular) to the pointing direction of the 3D scanner 104. For example, in the case of the fuselage 402, as shown in Figures 1 and 2, the cross-sectional profile 218 in the circumferential direction may be flattened to take into account the elliptic-circular cross-sectional shape typical of the fuselage 402 of a civil aircraft 400. In tapered areas of the fuselage 402, such as at the front or rear end, the cross-sectional profile 218 in the longitudinal direction may be rotated to take into account the out-of-plane inclination of the inspection surface 306.

[0071] [000107] Referring to Figure 36, an example of a pass / fail heatmap 202 is shown, which includes contour lines 204 of the surface of a suspicious area 158, similar to the example of the heatmap 202 shown in Figure 29. In Figure 36, the deviation legend 220 lists the acceptable thresholds or limits specified by the surface shape requirements of the inspection surface 306. In some embodiments of the surface inspection system 100, the processor 172 is configured to determine whether one or more characteristics of the shape of the point cloud 150 violate one or more limits specified by the surface shape requirements. For example, the processor 172 may determine whether the height of the highest point 208 in the suspicious area 158 exceeds the upper specification limit 222 specified by the surface shape requirements. This may be a fail condition. The processor 172 may also determine whether the depth of the lowest point 212 in the inspection area 158 exceeds the lower specification limit 224 specified by the surface shape requirements. This may also be a fail condition.

[0072] [000108] In another embodiment, if the depth of the lowest point 212 exceeds the lower specification limit 224 and the slope L / D is above a specified value (e.g., L / D is greater than 50), the processor 172 may determine whether the depth exceeds the limit specification limit 226. This may also be a limit pass condition. In this case, if the slope L / D is below a specified value (e.g., L / D is less than 50), it may be a fail condition. As can be understood, the surface shape requirements may include any of the various different acceptable conditions or limits from which the questionable area 158 can be analyzed in order to determine whether the questionable area 158 conforms to the surface shape requirements. For example, the processor 172 may determine whether the questionable area 158 contains a slope that violates the slope specification limit, regardless of height or depth along the slope. This may be useful information when performing a stress analysis of the structure 300.

[0073] [000109] The cross-hatching of the pass / fail heatmap 202 in Figure 36 identifies which parts of the questionable area 158 meet the specification limits of the surface shape requirements (i.e., pass conditions), which parts are at the limit (i.e., limit conditions), and which parts are outside the acceptable range of the surface shape requirements (i.e., fail conditions). If a fail condition exists within the questionable area 158, the processor 172 may notify the operator or inspector via a pop-up message (not shown) on the display screen 178 (Figure 7) of the computing device 174 (Figure 7), or via a text message, email, or any other means to signal and / or notify the operator or inspector of the fail condition. Information regarding the analysis of the questionable area 158, including the heatmap 202, the cross-sectional profile 218, and the list of fail conditions, may be recorded in the data library (not shown) described above for later reference.

[0074] [000110] As described above, the processor 172 functions as a data reporting system for the surface inspection system 100 and autonomously generates a report, including generating a heatmap 202 and inspection results that the processor 172 stores in a data library. Furthermore, the processor 172 may autonomously generate a report 200 (e.g., in PDF format) including the heatmap 202 (e.g., Figures 26, 27, 29, 31, and 36). The heatmap 202 may identify various surface characteristics of the suspected area 158 and / or non-defective area 154, such as identifying the highest point 208, lowest point 212, and / or slope gradient (e.g., the steepest slope gradient 214) within the suspected area 158 (e.g., Figures 26, 27, 29, and 36) of the inspection surface 306 and / or within the entire local portion 308 (e.g., Figure 30). The heatmap 202 of this disclosure uses cross-hatching to specify the amount by which the suspected area 158 (or non-defective area 154) deviates from the reference plane 164, but any of several other types of indicators, such as color coding, may be used within the heatmap 202. A cross-sectional profile 218 (e.g., Figures 32–35) may be included in the report 200 along with the heatmap 202. As stated above, the cross-sectional profile 218 may be obtained at any point within a local area of ​​the inspection surface 306, may be oriented in any orientation, and is not limited to passing through the highest point 208 and lowest point 212 of the suspected area 158, nor is it limited to being oriented parallel to the principal direction 148 of the surface outline requirements.

[0075] [000111] The processor 172 is configured to store all data generated by the surface inspection system 100 in a timestamped folder (not shown) in a data library (not shown). The stored data includes raw point cloud 150, baseline surface, orderly grid 162, reference plane 164, cross-sectional data, and other data. In addition to storing raw data, the inspection results may include pass / fail data indicating the size and location of surface defects (e.g., depressions 210, bulges 206), and root cause analysis and processing or repair actions for surface defects. The raw data and inspection results are stored together with template position data 138 (Figure 12). The template position data 138 is defined for the structure 300 to enable traceability and establish a digital thread (e.g., an as-built CAD model of the structure - not shown). The digital thread allows surface defects (e.g., depressions 210, bulges 206) identified by the surface inspection system 100 to be monitored, reviewed, and addressed at any point throughout the lifecycle of the structure 300. In the context of aircraft maintenance, the information stored in the digital thread ensures that the aircraft is safe and operational throughout its operational life.

[0076] [000112] The surface inspection system 100 may also be configured to optimize any one or more of its operating parameters. Since the data acquisition system 102, the data analysis system 170, and the data reporting system are integrated, any one or more operating parameters may be adapted and optimized to ensure that the surface inspection system 100 performs as desired for any given structure 300. For example, the 3D scanner 104 and / or scan template 120 of the data acquisition system 102 may be optimized for a given size, shape, configuration, and environment of the structure 300 to be inspected. Furthermore, the operating parameters of the data analysis processor 172 may be optimized, including optimizing techniques for detecting surface discontinuities 310, performing root cause analysis, and identifying defect trends. The operating parameters of the data reporting processor 172 may also be optimized with respect to template position data 138 and the types of metrics included in the inspection results.

[0077] [000113] Referring to Figure 37, a flowchart of method 500 for inspecting the inspection surface 306 of structure 300 is shown. One or more of the functions and capabilities of the surface inspection system 100 described above may be performed in one or more of the steps or processes of method 500 described below.

[0078] [000114] Step 502 of Method 500 includes scanning the inspection surface 306 using a three-dimensional (3D) scanner, as shown in Figure 10, to obtain a point cloud 150 of a plurality of points 152 representing at least a local portion 308 of the inspection surface 306. As described above and shown in Figure 15, the local portion 308 includes a non-defective area 154 and a suspected area 158. The suspected area 158 is at least partially surrounded by the non-defective area 154 and potentially contains one or more defects. In the embodiment of Figure 16, the suspected area 158 is entirely surrounded by the non-defective area 154 to provide a context for measuring surface properties, enabling proper interpretation of the inspection and justifying inspection approval.

[0079] [000115] As described above, the process of inspecting the inspection surface 306 may be initiated by an operator or inspector identifying a suspicious area 158 on the inspection surface 306 by visually observing for potential defects (e.g., depressions 210, bulges 206, etc.). Alternatively or further, the location of the suspicious area 158 on the inspection surface 306 may be based on historical data that identifies a specific area of ​​the inspection surface 306 where a defect was detected. The historical data may be derived from the inspector's personal knowledge and / or from inspection reports of previously manufactured versions of the same type of structure 300.

[0080] [000116] Alternatively, the suspicious region 158 can be identified by analyzing the point cloud 150 generated during scanning of the inspection surface 306. In this regard, the processor 172 may analyze the point cloud 150 and, based on calculations by the processor 172, designate a plurality of points 152 that exceed a probabilistic threshold or a surface change rate threshold as the suspicious region 158, as described above. In several other embodiments, the method may include designating the central portion 156 of the point cloud 150 as the suspicious region 158, based on the assumption that the central portion 156 contains a plurality of points 152 representing potential defects in the inspection surface 306.

[0081] [000117] In some embodiments, the method 500 includes positioning a scan template 120 on the inspection surface 306 to define the size and shape of a local portion 308. For example, as described above, the method may include attaching a physical template 122 to the inspection surface 306 before scanning through the 3D scanner 104. The physical template 122 has a template opening 130 of a predetermined size and shape. The physical template 122 may be positioned on the inspection surface 306 with the assistance of a template positioning aid 132, as described above and shown in Figures 3-4. Alternatively, positioning a scan template 120 on the inspection surface 306 may include using a light projector to project a template 124 onto the inspection surface 306, at least during scanning through the 3D scanner 104, as described above. In several other embodiments, the process of positioning the scan template 120 on the inspection surface 306 may include, via software (i.e., a virtual template), scanning the inspection surface 306 and then cropping an area of ​​the point cloud 150 located outside the local portion 308.

[0082] [000118] In any of the embodiments described above, the size and shape of a local portion 308 can be identified by visual observation of the size and shape of a portion of the inspection surface 306 that contains a potential defect. To provide appropriate context for the inspection process, the area of ​​the non-defective area 154 is preferably 1.5 to 20 times larger than the area of ​​the suspected area 158. Alternatively or further, the non-defective area 154 may be sized and shaped to encompass an area of ​​the structure 300 including at least one substructure member 304, such as an internal frame supporting the outer panel 302 (e.g., longitudinal members 408, circumferential frame 410), for the substructure member 304 may contribute to the presence of a surface defect potentially contained within the suspected area 158.

[0083] [000119] Method 500 includes using a template positioning system to capture the position of the scan template 120 relative to the structure 300. In some embodiments, the template positioning system may include manual input of template position data 138, as shown in Figures 11-12. As described above, the template position data 138 defines the position of the scan template 120 relative to the structure 300. In lieu of or in addition to manual input of the template position data 138, the template positioning system may include an indoor tracking system (not shown) that tracks the position of the scan template 120 relative to the structure 300 and can automatically record the template position data 138 when the scan is initiated. As described above, the scan system may include, but is not limited to, any of a variety of configurations, including a local positioning system (e.g., a system like GPS) including a small transceiver attached to the scan template 120. The small transceiver communicates with a number of transmitters installed at fixed locations within the environment of the structure 300. In another embodiment, the tracking system may include a network of infrared cameras and / or ultrasonic beacons for tracking the position of the scan template 120 on the structure 300. In yet another embodiment, the tracking system may include a laser tracker. The laser tracker emits a laser reflected from a small spherical reflector mounted on the scan template 120 for triangulation of its position on the structure 300.

[0084] [000120] The process for generating the point cloud 150 (Figure 10) is shown in Figures 11 and 12 and, as described above, results in a polygonized mesh 166. The polygonized mesh 166 is cut out to remove a local portion 308 of the inspection surface 306 from the rest of the polygonized mesh 166, as shown in Figure 13. As described above, the local portion 308 is defined by a template opening 130 and separated into two regions: a suspected region 158 and a non-defective region 154. The non-defective region 154 surrounds the suspected region 158.

[0085] [000121] Step 504 of Method 500 includes using the processor 172 to establish a reference plane 164 based on a plurality of points 152 in the non-defective region 154 and excluding a plurality of points 152 in the suspected region 158. The plurality of points 152 in the suspected region 158 are excluded when establishing the reference plane 164, as shown in Figure 21, but the reference plane 164 extends over the suspected region 158, as shown in Figure 22. Step 504 of establishing the reference plane 164 may be performed using any of a variety of surface fitting techniques, including, but not limited to, polynomial surface fitting (e.g., cubic polynomial surface fitting) or surface filtering.

[0086] [000122] In some embodiments, method 500 includes identifying and excluding surface discontinuities 310 when establishing a reference plane 164. The process of identifying surface discontinuities 310 includes generating an orderly grid 162 of uniformly spaced points 152 by interpolating the loose or unstructured arrangement of points 152 in the point cloud 150 of a local portion 308, as described above. In addition to generating the orderly grid 162, processor 172 also generates a baseline surface (i.e., an initial reference plane - not shown). The baseline surface is a polynomial surface fit of all points 152 in the raw point cloud 150 data of a local portion 308. Each point in the orderly grid 162 is compared to the baseline surface to establish a height value (i.e., in the local z direction) for each point in the orderly grid 162. Each point within the orderly grid 162 is then converted to a grayscale pixel intensity value ranging from 0 to 255, resulting in the formation of a local portion 308 image.

[0087] [000123] Once formed, the image is subjected to image processing techniques such as edge detection and contour mapping to identify a plurality of points 152 in an orderly grid 162 that have the sharpest changes in pixel intensity and represent the locations of surface discontinuities 310 of the inspection surface 306. The plurality of points 152 in the orderly grid 162 that represent the surface discontinuities 310 are traced back to the corresponding plurality of points 152 in the point cloud 150 and are ignored when establishing a reference plane 164 based on the point cloud 150, as described above.

[0088] [000124] In some embodiments, method 500 includes ignoring a plurality of points 152 representing scan noise from the point cloud 150. As described above, scan noise includes random deviations at some height of a plurality of points 152 in the point cloud 150 as a result of a plurality of factors such as vibration of the 3D scanner 104 and / or surface reflection of the inspection surface 306 during the scanning process. In the process described above to identify surface discontinuities 310 via an orderly grid 162, scan noise is ignored when it is essentially identified and reference noise is established.

[0089] [000125] Step 506 of Method 500 includes using the processor 172 to identify one or more properties associated with the shape of the point cloud 150 relative to the reference plane 164 in the suspicious region 158. As shown in Figures 24-25, the point cloud 150 of the suspicious region 158 is compared with the reference plane 164 and the suspicious region 158. This comparison may be performed via the processor 172 in a manner similar to the process described above for comparing multiple points 152 of an orderly grid 162 of the local portion 308 relative to a baseline surface of the local portion 308. In this regard, step 506 includes identifying the local z-direction deviation of each point in the point cloud 150 of the suspicious region 158 relative to the reference plane 164 in the suspicious region 158. As a result of such a comparison, a difference map or heatmap 202 is obtained showing relative areas of the suspicious region 158 that differ in height or depth from the reference plane 164, as shown in Figure 26.

[0090] [000126] In some embodiments, step 506 of identifying the surface properties of the suspected area 158 relative to the reference plane 164 may include identifying the location and height of the highest point 208 and lowest point 212 within the suspected area 158, as shown in Figures 26 and 29 and described above. Alternatively or further, step 506 of identifying the surface properties of the suspected area 158 may include identifying the location and orientation of the steepest slope gradient 214 within the suspected area 158, as shown in Figures 27 and 29. In the illustrated embodiments, the steepest slope gradient 214 is expressed as the ratio of length L to depth D. To reduce computation time, one of several different values ​​for length L may be predetermined. As an alternative to L / D, the steepest slope gradient 214 may be calculated as the ratio of depth D to length L (i.e., D / L).

[0091] [000127] Method 500 may optionally include generating a cross-sectional profile 218 of the suspected region 158, as shown in Figures 31 to 35. In some embodiments, the cross-sectional profile 218 is generated based on the local z-direction difference between the point cloud 150 and the reference plane 164. In several other embodiments, the cross-sectional profile 218 is normalized to take into account the out-of-plane curvature and / or out-of-plane inclination of the inspection surface 306.

[0092] [000128] In some embodiments, Method 500 includes aligning the scan template 120 to a principal direction 148 of the surface outline requirements of the inspection surface 306, for example, by using one or more alignment features 144 (e.g., directional arrows 146) on the scan template 120. With the scan template 120 aligned to the principal direction 148, Method 500 includes identifying the steepest slope gradient 214 in the cross-sectional profile 218 of the suspected area 158 parallel to the principal direction 148. In the illustrated embodiments, the cross-sectional profile 218 in Figures 32-35 passes through the highest point 208 (Figures 32-33) and lowest point 212 (Figures 34-35) of the suspected area 158 relative to the reference plane 164. However, the method may include generating a cross-sectional profile 218 oriented in any of several different directions, which passes through any of several different locations within the suspected area 158.

[0093] [000129] In some embodiments, method 500 may include using a processor 172 to determine whether one or more characteristics of the shape of the point cloud 150 within the suspicious area 158 violate one or more limitations specified by the surface shape requirements of the inspection surface 306. For example, as shown in Figure 36, the pass / fail heatmap 202 includes cross-hatching to identify which parts of the suspicious area 158 violate the specification limitations of the surface shape requirements (i.e., fail conditions) and which parts of the suspicious area 158 conform to the specification limitations (i.e., pass conditions), as indicated by the deviation legend 220. As stated above, method 500 may include providing a display when a fail condition exists within the suspicious area 158 under inspection. For example, the processor 172 may display a pop-up message on the display screen 178 of the computing device 174, or send a text message or email, to notify the operator of the fail condition.

[0094] [000130] Method 500 may further include using the processor 172 to generate a report 200 which includes the heatmap 202 and / or cross-sectional profile 218 described above within the suspicious area 158 and / or within a local portion 308 encompassing both the suspicious area 158 and the non-defective area 154. As described above, the heatmap 202 graphically displays the shape of the point cloud 150 within the suspicious area 158 and may optionally identify the highest point 208, the lowest point 212, and / or the steepest slope gradient 214. The report 200 may also include an arbitrary cross-sectional profile 218 of the shape of the point cloud 150 passing through any location in the suspicious area 158, such as the highest point 208 and / or the lowest point 212. As described above, the cross-sectional profile 218 may be aligned to one or more principal directions 148 of the surface outline requirements of the local portion 308. Report 200 may indicate whether the highest point 208, lowest point 212, and / or slope gradient of the suspicious area 158 violate the limits specified by the surface outline requirements.

[0095] [000131] A number of modifications and other versions and embodiments of the Disclosure will be conceivable to a person skilled in the art to which the Disclosure belongs, having the merits of the teachings presented in the foregoing description and the accompanying drawings. The versions and embodiments described herein are illustrative and are not intended to be limiting or exhaustive. Certain terms are used herein, but these terms are used in a general and descriptive sense only and are not intended to be limiting. In addition to the methods and apparatus described herein, functionally equivalent methods and apparatus included in the scope of the Disclosure are also achievable from the foregoing description. Such modifications and changes are intended to fall within the appended claims. The Disclosure is limited only by the entire scope of the appended claims and the equivalents to which such claims are granted.

Claims

1. A surface inspection system (100) for inspecting a local portion (308) of an inspection surface (306) of a structure (300), A three-dimensional (3D) scanner (104) configured to scan the inspection surface (306) and acquire a point cloud (150) of a plurality of points (152) representing at least the local portion (308), wherein the local portion (308) includes a non-defective region (154) and a suspected region (158) at least partially surrounded by the non-defective region (154) and potentially containing one or more defects, and The system comprises a processor (172), and the processor (172) is Establishing a reference plane (164) based on the plurality of points (152) within the non-defective area (154) and excluding the plurality of points (152) within the suspected area (158), wherein the reference plane (164) extends over the suspected area (158), and A surface inspection system (100) is configured to perform the task of identifying one or more characteristics of the shape of the point cloud (150) with respect to the reference surface (164) within the suspected area (158).

2. The processor (172) has a relationship with respect to the reference surface (164). The position and height of the highest point (208), The position and depth of the lowest point (212), and The location and orientation of the steepest slope (214), The surface inspection system (100) according to claim 1, configured to identify one or more characteristics of the shape of the point group (150) by identifying at least one of the following.

3. The surface inspection system (100) according to claim 1, wherein the processor (172) is configured to determine whether one or more characteristics of the shape of the point cloud (150) violate one or more restrictions specified by the surface shape requirements of the inspection surface (306).

4. A scan template (120) that can be placed on the inspection surface (306), which defines the size and shape of the local portion (308), Before scanning via the 3D scanner (104), a physical template (122) is configured to be attached to the inspection surface (306). During scanning via the 3D scanner (104), a projected template (124) is projected onto the inspection surface (306), and After scanning via the 3D scanner (104), the processor (172) is configured to cut out an area of ​​the point cloud (150) located outside the local portion (308), which is a virtual template. The surface inspection system (100) according to claim 1, further comprising a scan template (120) configured as one of the surfaces.

5. The scan template (120) includes one or more alignment features (144) configured to facilitate alignment of the scan template (120) with respect to the principal direction (148) of the surface shape requirements of the local portion (308), The surface inspection system (100) according to claim 4, wherein the processor (172) is configured to determine the inclination gradient of the outer cross-section (216) parallel to the principal direction (148), and the outer cross-section (216) passes through one of the highest point (208) and lowest point (212) in the point cloud (150) relative to the reference plane (164).

6. The non-defective region (154) is The non-defective area (154) includes an area of ​​the inspection surface (306) that includes at least one lower structural member (304), and The area of ​​the non-defective region (154) is 1.5 to 20 times larger than the area of ​​the suspected region (158). A surface inspection system (100) according to claim 1, wherein the size is determined according to at least one of ours.

7. The surface inspection system (100) according to claim 1, wherein the processor (172) is configured to exclude the plurality of points (152) representing scan noise from the point cloud (150).

8. The surface inspection system (100) according to claim 1, wherein the processor (172) is configured to exclude the plurality of points (152) representing surface discontinuities (310) within the inspection surface (306) from the point group (150).

9. The surface inspection system (100) according to claim 1, wherein the processor (172) is configured to establish the reference surface (164) using either polynomial surface fitting or surface filtering.

10. The aforementioned processor (172) A graph displays contour lines (204) of the surface of the point cloud (150) within the suspicious region (158), and a heat map (202) showing at least one of the highest point (208), lowest point (212), and steepest slope gradient (214). A cross-sectional profile (218) of the shape of the point group (150) within the suspicious region (158), the cross-sectional profile (218) passing through the highest point (208) and the lowest point (212) and aligned with one or more principal directions (148) of the surface shape requirements of the local portion (308), and An indication of whether at least one of the highest point (208), the lowest point (212), and one or more gradients of the suspected area (158) violates one or more restrictions specified by the surface shape requirements, The surface inspection system (100) according to claim 1, configured to generate a report (200) including at least one of the following.

11. A surface inspection system (100) for inspecting a local portion (308) of an inspection surface (306) of a structure (300), A scan template (120) that can be positioned on the inspection surface (306) and defines the size and shape of the local portion (308) of the inspection surface (306), wherein the local portion (308) includes a non-defective area (154) and a suspected area (158) that is at least partially surrounded by the non-defective area (154) and potentially has one or more defects. A three-dimensional (3D) scanner (104) configured to scan the aforementioned local portion (308) and acquire a point cloud (150) of multiple points (152), and The system comprises a processor (172), and the processor (172) is Establishing a reference plane (164) based on the plurality of points (152) within the non-defective area (154) and excluding the plurality of points (152) within the suspected area (158), wherein the reference plane (164) extends over the suspected area (158), and Identifying one or more characteristics associated with the shape of the point cloud (150) with respect to the reference plane (164) within the suspected region (158), wherein the characteristics are: The position and height of the highest point (208), The position and depth of the lowest point (212), and A surface inspection system (100) is configured to perform the task of identifying one or more characteristics, including the position and orientation of the slope gradient of an outer cross section (216) that passes through at least one of the highest point (208) and the lowest point (212) and is oriented parallel to the principal direction (148) of the surface outline requirements of the local portion (308).

12. A method for inspecting the inspection surface (306) of a structure (300), The method involves scanning the inspection surface (306) using a three-dimensional (3D) scanner (104) and obtaining a point cloud (150) of a plurality of points (152) representing at least a local portion (308) of the inspection surface (306), wherein the local portion (308) includes a non-defective region (154) and a suspected region (158) at least partially surrounded by the non-defective region (154) and potentially containing one or more defects. Establishing a reference plane (164) using a processor (172) based on the plurality of points (152) in the non-defective region (154) and excluding the plurality of points (152) in the suspected region (158), wherein the reference plane (164) extends over the suspected region (158), and A method comprising using the processor (172) to identify one or more characteristics associated with the shape of the point cloud (150) relative to the reference plane (164) within the suspected region (158).

13. Identifying the one or more characteristics associated with the shape of the point group (150) is, with respect to the reference plane (164), Identify the position and height of the highest point (208). Identifying the location and depth of the lowest point (212), and Identify the location and orientation of the steepest slope (214), The method according to claim 12, comprising at least one of the following.

14. The method according to claim 12, further comprising using the processor (172) to determine whether the one or more characteristics of the shape violate one or more limitations specified by the surface shape requirements of the inspection surface (306).

15. Before scanning via the 3D scanner (104), a physical template (122) is attached to the inspection surface (306). During scanning via the 3D scanner (104), a light projector is used to project a template (124) onto the inspection surface (306), and After scanning via the 3D scanner (104), a virtual template is used to cut out an area of ​​the point cloud (150) located outside the local portion (308). The method of claim 12, further comprising positioning a scan template (120) relative to the inspection surface (306) in order to define the size and shape of the local portion (308) by performing one of the following:

16. Using one or more alignment features (144) of the scan template (120), the scan template (120) is aligned in the principal direction (148) of the surface outline requirements of the local portion (308), and The method of claim 15, further comprising identifying the steepest slope gradient (214) in the cross-sectional profile (218) of the point group (150) within the suspected region (158), which is parallel to the principal direction (148) and passes through one of the highest point (208) and lowest point (212) of the suspected region (158) with respect to the reference plane (164).

17. The method according to claim 12, further comprising ignoring the plurality of points (152) representing scan noise from the point cloud (150).

18. The method according to claim 12, further comprising ignoring the plurality of points (152) representing surface discontinuities (310) within the inspection surface (306) from the point group (150).

19. Establishing the aforementioned reference surface (164) The method according to claim 12, comprising establishing the reference surface (164) using either polynomial surface fitting or surface filtering.

20. Using the aforementioned processor (172), The shape of the point cloud (150) within the suspicious region (158) is displayed graphically, and a heat map (202) showing at least one of the highest point (208), lowest point (212), and steepest slope gradient (214) is displayed. A cross-sectional profile (218) having the shape that passes through the highest point (208) and the lowest point (212), and which is aligned with one or more principal directions (148) of the surface shape requirements of the local portion (308), and An indication of whether at least one of the highest point (208), the lowest point (212), and one or more gradients within the suspected area (158) violates the limits specified by the surface shape requirements, The method according to claim 12, further comprising generating a report (200) which includes at least one of the following.

Images