Apparatuses and methods for large area wireless fuselage dent inspection

Abstract

To provide an inspection cart for inspecting large area objects, such as airplane fuselages.SOLUTION: An inspection cart includes a tower supporting a track. The inspection cart also includes a carriage movable along the track. The carriage supports a 3D scanner having at least two cameras and a projector. The inspection cart also includes integrated optical fiducial markers arranged relative to the track so that the optical fiducial markers are in a field of view of the 3D scanner during scanning of an object. In other aspects, an inspection system that includes at least one inspection cart is provided. In further aspects, a method of inspecting a large area object using the inspection cart is provided.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] Aspects of the present disclosure relate to apparatus, systems, and methods for inspecting large area objects, such as aircraft fuselages. [Background technology]

[0002] Traditionally, inspecting aircraft fuselages for defects has been a manual process using flashlights and bump caps, relying on human eyesight and meticulous attention. Therefore, inspecting aircraft fuselages for defects has traditionally been quite time-consuming. Furthermore, because aircraft fuselages are typically inspected during final assembly, when the aircraft fuselages are coated with a green temporary protective coating (TPC), many defects may be overlooked. As a result, some defects may not be discovered until later stages, such as after the aircraft fuselages are coated with a shiny final paint job. Rework to repair such undetected defects at such stages can be costly. In some cases, for example, it may be necessary to remove the cockpit or restroom from the fuselage to remove a dent that was not detected during final assembly. Currently available scanners and vision systems are relatively small, localized, and require too much time to process images, making them of little use in automating the inspection process. Summary of the Invention

[0003]

[0003] In one aspect, the present disclosure provides a method, the method including: translating a 3D scanner relative to a target object, the 3D scanner being disposed in an inspection cart; scanning the target object using the 3D scanner to capture images of at least a slice of the target object as the 3D scanner is translated, wherein optical fiducial markers integrated in the inspection cart are positioned within a field of view of the 3D scanner during scanning; stitching together the captured images of the target object obtained during the scanning to create a slice image, the captured images being stitched according to positional targets provided by the optical fiducial markers; and detecting one or more defects in the target object based on the slice image.

[0004]

[0004] In one aspect, in combination with any exemplary method described above or below, the method includes projecting one or more defect indicators onto the target object by a 3D scanner to indicate each of one or more detected defects in the target object.

[0005]

[0005] In one aspect, in combination with any of the exemplary methods described above or below, at least one defect indicator of one or more defect indicators is projected onto the target object to outline a defect of one or more detected defects in the target object.

[0006]

[0006] In one aspect, in combination with any of the exemplary methods described above or below, the 3D scanner is translated along a linear path oriented at an angle to the vertical.

[0007]

[0007] In one aspect, in combination with any of the exemplary methods described above or below, the optical reference marker includes a first reference marker rail and a second reference marker rail spaced apart from each other and arranged parallel to a linear path.

[0008]

[0008] In one aspect, in combination with any of the exemplary methods described above or below, the position targets are positioned on a first finger extending from a first reference marker rail and a second finger extending from a second reference marker rail, the first fingers being spaced apart from each other along the first reference marker rail and the second fingers being spaced apart from each other along the second reference marker rail.

[0009]

[0009] In one aspect, in combination with any of the exemplary methods described above or below, the method includes masking optical reference markers in the slice images so that the optical reference markers do not appear in the slice images.

[0010]

[0010] In one aspect, in combination with any exemplary method above or below, the slice of the target object is a first slice, and the method further includes: a) moving the inspection cart so that the inspection cart is aligned with a subsequent slice of the target object; b) translating the 3D scanner relative to the target object with the inspection cart aligned with the subsequent slice; c) scanning the target object using the 3D scanner to capture images of the subsequent slice of the target object as the 3D scanner is translated with the inspection cart aligned with the subsequent slice, wherein optical fiducial markers are positioned within the field of view of the 3D scanner during the scanning of the subsequent slice; d) stitching together the captured images of the subsequent slice of the target object to create subsequent slice images, wherein the captured images of the subsequent slices are stitched together according to positional targets provided by the optical fiducial markers; and e) detecting one or more defects on the subsequent slice of the target object based on the subsequent slice images.

[0011]

[0011] In one aspect, in combination with any of the exemplary methods described above or below, the method further includes repeating a) to e) for a predetermined length of the target object and stitching together the slice image of the first slice and each subsequent slice image of the subsequent slices into a combined slice image.

[0012]

[0012] In one aspect, in combination with any of the exemplary methods described above or below, the combined slice image is a first combined slice image associated with a first side of the target object, and the method further includes creating a second combined slice image associated with a second side of the target object.

[0013]

[0013] In one aspect, in combination with any exemplary method described above or below, the method includes creating a digital twin of the target object based at least in part on a first combined slice image and a second combined slice image.

[0014] In one aspect, in combination with any of the exemplary methods described above or below, an inspection cart moves autonomously from one slice of the target object to another.

[0015]

[0015] In one aspect, in combination with any of the exemplary methods described above or below, the target object is a torso and the torso is scanned so that the captured image of the slice extends from the midline of the lower abdomen of the torso to the window line of the torso.

[0016]

[0016] In one aspect, in combination with any of the exemplary methods described above or below, the target object is a torso and the torso is scanned so that the captured image of the slice extends from the midline of the lower abdomen of the torso to the midline of the upper torso.

[0017]

[0017] In another aspect, the present disclosure provides an inspection system. The inspection system includes an inspection cart. The inspection cart includes a tower supporting a track. The inspection cart further includes a carriage movable along the track, the carriage supporting a 3D scanner having at least two cameras and a projector. Further, the inspection cart includes an optical fiducial marker positioned relative to the track such that the optical fiducial marker is within a field of view of the 3D scanner during scanning of a target object.

[0018] In one aspect, in combination with any exemplary system described above or below, the inspection system further includes one or more processors and one or more non-transitory memory devices having stored thereon a program that, when executed by any combination of one or more processors, causes the one or more processors to perform steps including: moving a carriage along a trajectory such that the 3D scanner is moved relative to the target object; causing the 3D scanner to scan the target object as the 3D scanner is moved to capture images of at least a slice of the target object; stitching together the captured images of the slice of the target object to create a slice image, wherein the captured images are stitched together according to positional targets provided by the optical fiducial markers; and detecting one or more defects in the target object based on the slice image.

[0019]

[0019] In one aspect, in combination with any of the exemplary systems described above or below, the process further includes causing a projector of the 3D scanner to project one or more defect indicators onto the target object to indicate each of the one or more detected defects in the target object.

[0020]

[0020] In one aspect, in combination with any of the exemplary systems described above or below, the track is a linear track and the optical reference marker includes a first reference marker rail and a second reference marker rail spaced apart from each other and arranged parallel to the linear track.

[0021]

[0021] In one aspect, in combination with any of the exemplary systems described above or below, the target object is a fuselage and the trajectory is a curved trajectory extending above and below the fuselage.

[0022]

[0022] In yet another aspect, the present disclosure provides an inspection cart, which may include a tower supporting a track, a carriage movable along the track, the carriage supporting a 3D scanner having at least two cameras and a projector, and an optical fiducial marker positioned relative to the track such that the optical fiducial marker is within a field of view of the 3D scanner during scanning of a target object.

[0023]

[0023] So that the above-described features of the present disclosure can be understood in detail, a more detailed description of the present disclosure than that briefly summarized above can be made by reference to several exemplary embodiments, some of which are illustrated in the accompanying drawings. [Brief explanation of the drawings]

[0024] [Figure 1]

[0024] FIG. 1 is a front-to-rear perspective view of an inspection cart according to an exemplary embodiment of the present disclosure. [Figure 2]

[0025] FIG. 2 is a rear-front perspective view of the examination cart of FIG. 1. [Figure 3]

[0026] FIG. 2 is a side view of the inspection cart of FIG. 1. [Figure 4]

[0027] FIG. 1 is a system diagram of an inspection system according to an exemplary aspect of the present disclosure. [Figure 5]

[0028] FIG. 1 is a flow diagram of an exemplary method for inspecting a target object using an inspection system, according to an exemplary aspect of the present disclosure. [Figure 6]

[0029] 1 is a schematic side view of an airplane fuselage illustrating one exemplary manner in which the fuselage may be divided into slices. [Figure 7]

[0030] 1 shows an examination cart positioned relative to a slice on the port side of the fuselage and performing the scan. [Figure 8] 1 shows an examination cart positioned relative to a slice on the port side of the fuselage and performing the scan. [Figure 9]

[0031] 9 shows multiple images of a slice of the torso captured during the scan shown in FIGS. 7 and 8. FIG. [Figure 10]

[0032] 10 shows a combined slice image stitched together from the slice images shown in FIG. 9. [Figure 11]

[0033] Shows defect indicators projected onto the fuselage by the 3D scanner's projector. [Figure 12]

[0034] 1 illustrates a plurality of slice images, each associated with one of the predefined slices of the torso, stitched together into a combined slice image. [Figure 13]

[0035] 1 illustrates a front-to-back view of a fuselage, as well as a first inspection cart and a second inspection cart for scanning and capturing images of each side of the fuselage, according to an exemplary embodiment of the present disclosure. [Figure 14]

[0036] FIG. 1 is a perspective view of an inspection cart having an enclosure tower in a scanner / motivator configuration, according to an exemplary embodiment of the present disclosure. [Figure 15]

[0037] FIG. 10 is a perspective view of another inspection cart having an encircling tower in a roller coaster style conveyor configuration according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0025]

[0038] The present disclosure relates to improved apparatus, systems, and methods for inspecting aircraft fuselages. In particular, the apparatus, systems, and methods disclosed herein facilitate detecting defects in aircraft fuselages, for example, in real time. The apparatus and methods of the present disclosure may be utilized to inspect aircraft fuselages at various stages of their lifecycle, such as during final assembly, delivery, or even after the aircraft has been deployed in the field (e.g., post-hail storm inspection). While the inspection apparatus, systems, and methods are described herein in the context of inspecting aircraft fuselages, aspects of the present disclosure are also applicable to inspecting other large-area objects, such as other areas of airplanes (e.g., wings, cockpits, tails, etc.), ships, spacecraft, trains, industrial pipes, etc.

[0026]

[0039] In one exemplary embodiment, an inspection cart for an inspection system is provided. The inspection cart facilitates inspection of an airplane fuselage, for example, by detecting defects in the skin in real time. The inspection cart is movable and includes a tower supporting a track. The inspection cart also includes a carriage movable along the track. The carriage supports a 3D scanner having at least two cameras and a projector. The inspection cart also includes an integrated optical fiducial positioned relative to the track, such that the optical fiducial is within the field of view of the 3D scanner during scanning of the fuselage. In this regard, the optical fiducial is integrated into the inspection cart. During inspection of the fuselage, the 3D scanner can be translated along the track. As the 3D scanner is translated, the 3D scanner captures images of at least a slice of the fuselage. The integrated optical fiducial is positioned within the field of view of the 3D scanner during scanning. The captured images can be stitched together to create a slice image. The captured images are stitched together according to positional targets provided by the optical fiducial. Based on the slice images, one or more defects in the fuselage may be detected. The detected defects may be presented to the operator while the slice is being scanned or immediately after the slice is scanned. In particular, one or more defect indicators may be projected onto the fuselage to indicate the location of the defects. This allows the operator to immediately recognize the defects and their respective locations on the fuselage. Other warnings or reports identifying the defects are also possible, such as a real-time display of the identified defects on a display, an audible warning, or an automatically generated report. This process may be repeated for other slices of the fuselage, and the slice images may be stitched together to form a combined slice image. In some aspects, multiple inspection carts may be utilized to simultaneously scan different sections of the fuselage.

[0027]

[0040] The inspection carts, inspection systems, and methods disclosed herein may provide certain advantages, benefits, and / or technical effects. For example, manual inspection of fuselages to detect defects (e.g., dents, missing parts, unacceptable errors, etc.) may be eliminated or largely replaced by the devices and methods provided herein. Eliminating or reducing the need for manual inspection can reduce escapes, lower costs, increase repeatability, precision, and accuracy of detected defects, improve quality and safety, reduce inspection time, and eliminate or reduce ergonomic challenges for operators (e.g., head injuries on the underside of the fuselage, repetitive stress (awkward posture), eye strain, and noise, etc.). Furthermore, optical fiducial markers are integrated into the inspection cart, enabling non-contact inspection of the fuselage. Thus, the fuselage does not need to be touched or contacted to locate the optical fiducial markers. Optical fiducial markers have traditionally been placed on the target part or a fixture holding the target part, e.g., with stickers, at known locations. The setup and cleanup of this conventional technique can be time consuming and expensive for large target parts such as an airplane fuselage.

[0028]

[0041] Furthermore, defects can be detected and their locations can be immediately communicated to an operator, for example, by projecting defect indicators onto the aircraft fuselage. The operator can use the indicators to mark the aircraft, for example, with tape, if desired. Furthermore, the carriage supporting the 3D scanner can be driven in a controlled motion with optical fiducial markers at known distances and within the camera's field of view. The 3D scanner can identify its relative position using position targets on the "fingers" and / or fiducial marker rails. This aids in stitching together captured images for slice images of the fuselage. Furthermore, the inspection cart is mobile and can therefore be moved (e.g., manually or autonomously) from one capture position to the next to capture different slices of the fuselage. The slice images can be stitched together to create, for example, a combined slice image of the length of the aircraft fuselage. The slice images and / or combined slice images can be used to create a digital twin of the fuselage. This digital twin can be used as a baseline for comparison purposes with future scans of the fuselage, among other possible uses. Additionally, the slice images and / or combined slice images may be used by a predictive assembly system to make assembly predictions, such as predictions regarding how to address one or more of the detected defects, for example, by adding shims, machining a surface, etc.

[0029]

[0042] Turning now to the drawings, FIGS. 1, 2, and 3 provide various views of an inspection cart 100 according to exemplary embodiments of the present disclosure. For reference, the inspection cart 100 defines a longitudinal direction T, a lateral direction L, and a vertical direction V, which are mutually perpendicular to one another and form an orthogonal system of directions. The inspection cart 100 has a front end 102 and a rear end 104. The front end 102 and the rear end 104 are spaced apart from one another along the longitudinal direction T. The inspection cart 100 also has a first side 106 and a second side 108. The first side 106 is spaced apart from the second side 108 along the lateral direction L.

[0030]

[0043] The inspection cart 100 includes a chassis 110 to which a plurality of casters 112 are attached. The casters 112 allow the inspection cart 100 to be moved. The chassis 110 is generally formed by a plurality of frame members and has a rectangular configuration. The inspection cart 100 also includes a first side mount 114 and a second side mount 116, both of which are connected to and supported by the chassis 110. The first side mount 114 includes a first side rail 118, a first front post 120, and a first intermediate post 122. Similarly, the second side mount 116 includes a second side rail 124, a second front post 126, and a second intermediate post 128. The first side rail 118 and the second side rail 124 are each positioned at an ergonomically convenient height. They can be easily grasped by an adult of average height, and the testing cart 100 can be manually moved, if desired, by pushing or pulling one or both of the first side rail 118 and the second side rail 124.

[0031]

[0044] The inspection cart 100 also includes a tower 130 that structurally supports a translating carriage 132 that supports a 3D scanner 134. The tower 130 includes, among other things, a first tower post 136, a second tower post 138, and a tower cross member 140. The tower cross member 140 extends between and connects the first tower post 136 and the second tower post 138 at their upper ends. The tower 130 also includes a first angle frame 141 and a second angle frame 142. The first angle frame 141 and the second angle frame 142 couple to the tower cross member 140 at the top of the tower 130 and to a chassis cross member 144 at the bottom of the tower 130. The tower 130 also includes a first intermediate tower post 146 and a second intermediate tower post 148. The inspection cart 100 includes a first angled frame 141 and a second angled frame 142. The first side rail 118 and the second side rail 124 connect to the first tower post 136 and the second tower post 138, respectively. A rear handle bar 150 connects to the first tower post 136 and the second tower post 138 and provides an ergonomic bar for a user to manually grip and move the inspection cart 100. The rear handle bar 150 also advantageously extends above the control cabinet 152, for example, to provide protection for the control cabinet 152. The control cabinet 152 may house various power, communication, and computing equipment associated with operating the inspection cart 100.

[0032]

[0045] The tower 130 also includes a track 154 and a drive system 156. The drive system 156 functions to translate or move the carriage 132 along the track 154. The track 154 includes a first side rail 158 and a second side rail 160 coupled to the first angled frame 141 and the second angled frame 142, respectively. The track 154 is a linear track in this example. In this regard, the first side rail 158 and the second side rail 160 are linear. In some example embodiments, the first side rail 158 and the second side rail 160 may include end stops at their upper and / or lower ends, for example, to provide a fail-safe for the carriage and / or the 3D scanner 134 therein. As one example, the end stops may be shock absorbers disposed at the bottom of the first side rail 158 and / or the second side rail 160. As another example, the end stops may be compression springs disposed on the bottom of the first side rail 158 and / or the second side rail 160. As yet another example, the end stops may be elastomeric stop buffers disposed on the top of the first side rail 158 and / or the second side rail 160. An elastomeric stop buffer 162 disposed on the top of the second side rail 160 is shown in FIG.

[0033]

[0046] The drive system 156, in this embodiment, is a belt drive system and includes, among other things, an electric motor 164, a gearbox 166 mechanically coupled to the electric motor 164, and a belt pulley system 168 mechanically coupled to the gearbox 166. Generally, the carriage 132 is slidably received within the track 154, and the drive system 156 can be controlled to translate the carriage 132, and consequently the 3D scanner 134, along the track 154. The carriage 132 is translated along the track 154 between a first position (e.g., a lower position) and a second position (e.g., an upper position). While the carriage 132 is shown in the lower or first position in FIGS. 1 and 3 , the carriage 132 is shown in the upper or second position in FIG. 2 . Thus, the 3D scanner 134 can be translated along a translation path TP ( FIG. 3 ), which in this embodiment is a linear path oriented at an angle with respect to the vertical direction V. In some exemplary embodiments, the translation path TP is oriented at forty-five degrees (45°) relative to the vertical direction V. However, the translation path TP may be oriented at other angles, such as thirty degrees (30°) or sixty degrees (60°), in other exemplary embodiments.

[0034]

[0047] The inspection cart 100 also includes optical fiducial markers that are positioned within the field of view (FOV) of the 3D scanner 134 during scanning of a target object, such as an aircraft fuselage. In this embodiment, the optical fiducial markers include a first optical fiducial marker 170 and a second optical fiducial marker 172. The first optical fiducial marker 170 includes a first fiducial marker rail 174, and the second optical fiducial marker 172 includes a second fiducial marker rail 176. The first fiducial marker rail 174 and the second fiducial marker rail 176 are spaced apart from each other, for example, along the lateral direction L and aligned parallel to the translation path TP, or rather aligned parallel to the translation angle of the 3D scanner 134. Both the first fiducial marker rail 174 and the second fiducial marker rail 176 are spaced apart from the track 154, for example, along a direction perpendicular to the translation path TP. A lower end of the first fiducial marker rail 174 is supported by a first forward support post 178. The first forward support post 178 is connected to the first side mount 114 and extends longitudinally along the vertical direction V. The upper end of the first reference marker rail 174 is connected to the first aft support post 180. The first aft support post 180 is connected to the tower cross member 140 and extends longitudinally along a direction perpendicular to the translation path TP. Similarly, the lower end of the second reference marker rail 176 is supported by the second forward support post 182. The second forward support post 182 is connected to the second side mount 116 and extends longitudinally along the vertical direction V. The upper end of the second reference marker rail 176 is connected to the second aft support post 184. The second aft support post 184 is connected to the tower cross member 140 and extends longitudinally along a direction perpendicular to the translation path TP.

[0035]

[0048] As shown in FIG. 2 , the first optical fiducial marker 170 and the second optical fiducial marker 172 may include position targets 186. Generally, the position targets 186 function to assist the 3D scanner 134 in locating its position as it translates during scanning of a target object and also allow captured images to be stitched together to form a combined image. In some aspects, the position targets 186 may be located on the first fiducial marker rail 174 and the second fiducial marker rail 176. Additionally or alternatively, the position targets 186 may be located on fingers of the first fiducial marker rail 174 and the second fiducial marker rail 176. In some aspects, for example, the position targets 186 may be located on the first finger 188 and the second finger 190. A first finger 188 extends from the first fiducial marker rail 174, and a second finger 190 extends from the second fiducial marker rail 176. The first fingers 188 are spaced apart from one another along the first fiducial marker rail 174, and the second fingers 190 are spaced apart from one another along the second fiducial marker rail 176. The first fingers 188 and the second fingers 190 each extend from their respective first and second fiducial marker rails 174, 176 toward one another along the lateral direction L. In the embodiment shown in FIG. 2 , a position target 186 is disposed on each of the first fingers 188 and is located along the entire length of the first fiducial marker rail 174. Additionally, a position target 186 is disposed on each of the second fingers 190 and is located along the entire length of the second fiducial marker rail 176. Each enlarged view in Figure 2 shows a location target 186 for the feature being described. The location target 186 is positioned to be aimed at or otherwise within the FOV of the 3D scanner 134.

[0036]

[0049] The position targets 186 may be arranged in different patterns along the first and second fiducial marker rails 174, 176 and the first and second fingers 188, 190. The pattern and position of the position targets 186 (e.g., coordinates relative to the 3D scanner 134) may be known to the 3D scanner 134 and / or a computing system associated with the inspection cart 100. The position targets 186 may thereby be used to assist the 3D scanner 134 in locating its position as it translates during scanning of a target object, and also to stitch together captured images. Advantageously, the first and second optical fiducial markers 170, 172 are integrated into the inspection cart 100 with the position targets 186 located within the FOV of the 3D scanner 134. In this way, the 3D scanner 134 can know or locate its position relative to the position targets 186. That is without the need for any targets or markups on the target object itself. This provides non-contact scanning capability.

[0037]

[0050] FIGS. 1, 2, and 3 provide an inspection cart according to exemplary embodiments of the present disclosure. In some alternative embodiments, the inspection cart 100 of FIGS. 1, 2, and 3 may have other configurations. For example, in some embodiments, the inspection cart 100 may be disposed on wheels. The wheels may be locked into tracks, which may extend along at least a portion of a target object, such as the entire longitudinal length of a fuselage. The inspection cart 100 may be moved along the tracks to capture images of various "slices" of the fuselage. In still other embodiments, the inspection cart 100 may be configured to have one or more steering wheels and, optionally, one or more non-steering wheels. In such embodiments, the inspection cart may include a steering device (e.g., a steering wheel, a lever for zero-turn capability, etc.) and may have a motor or engine for propelling the inspection cart 100. In this regard, the inspection cart 100 may be driven to a desired location for scanning. In some further embodiments, the inspection cart 100 may include automatic or autonomous driving features. These allow the cart to be autonomously moved to various positions, for example, along the longitudinal length of the fuselage. In still further embodiments, as will be described in more detail herein, the inspection cart 100 may include an "encirclement tower," which allows, for example, the upper quadrants of the fuselage to be scanned in addition to the lower quadrants.

[0038]

[0051] 4, in some aspects, the inspection cart 100 of FIGS. 1, 2, and 3 may be part of an inspection system 200. The inspection system 200 may include the inspection cart 100 (FIGS. 1-3) and a computing system 210, and in some aspects may also include a data repository 238 and / or a predictive assembly system 240. Other systems may also be communicatively coupled to the inspection system 200, for example, via a communication bus 244.

[0039]

[0052] As shown in FIG. 4, the inspection cart 100 (FIGS. 1-3) includes a 3D scanner 134 that is translatable along a track 154 (FIG. 1). The 3D scanner 134 includes, among other things, at least two cameras and a projector 191. In this example, the cameras include a first camera 192 and a second camera 193, which are positioned on either side of the projector 191. The first camera 192 and the second camera 193 capture images of the target object as the 3D scanner 134 is translated along the track 154. As will be described in more detail herein, during scanning, the projector 191 may project light onto the target object to enhance the images captured by the first camera 192 and the second camera 193. The projector 191 may also be used to project defect indicators onto the target object to indicate each detected defect. The 3D scanner 134 also includes a first laser emitter 194 and a second laser emitter 195, which are spaced apart from each other. The first laser emitter 194 and the second laser emitter 195 can emit lasers onto the target object, for example, to ensure that the focal distance between the first camera 192 and the second camera 193 and the target object is within a predetermined range or specification. For example, when the lasers emitted from the first laser emitter 194 and the second laser emitter 195 meet at a certain point, it can be determined that the focal distance is within a predetermined range or specification. At this point, the first laser emitter 194 and the second laser emitter 195 can be used to determine that the inspection cart 100 has an appropriate “standoff” with respect to the target object.

[0040]

[0053] The 3D scanner 134 may also be communicatively coupled to other systems onboard the inspection cart 100 (FIGS. 1-3), such as components within the control cabinet 152 (FIG. 1) and / or onboard components of the computing system 210, as well as one or more off-board components, such as off-board components of the computing system 210, the data repository 238, the predictive assembly system 240, etc. In some embodiments, the computing system 210 may be located partially onboard and partially off-board the inspection cart 100. In other exemplary embodiments, the computing system 210 may be located entirely onboard the inspection cart 100.

[0041]

[0054] The computing system 210 may include one or more processors 212 and one or more non-transitory memory devices 214. The one or more non-transitory memory devices 214 may store computer-readable instructions 216 or code. The computer-readable instructions 216 or code may be executed by the one or more processors 212 to perform operations such as causing the first camera 192 and the second camera 193 to capture images of the target object as the 3D scanner 134 moves along the trajectory 154, stitching the captured images together, and / or causing the projector 191 to project defect indicators corresponding to detected defects onto the target object. The instructions 216 may be software written in any suitable programming language or may be implemented in hardware. The one or more processors 212 and the one or more non-transitory memory devices 214 may be embodied in one or more computing devices, for example.

[0042]

[0055] The one or more processors 212 may include any suitable processing device, such as a microprocessor, microcontroller, integrated circuit, logic device, or other suitable processing device. The one or more memory devices 214 may include one or more computer-readable media, including, but not limited to, non-transitory computer-readable media, RAM, ROM, hard drives, flash drives, and other memory devices. The one or more memory devices 214 may store information accessible by the one or more processors 212, including instructions 216 that may be executed by the one or more processors 212. The memory device(s) 214 may also store data 218 that may be accessed by the processor(s). For example, the data 218 may include images captured by the 3D scanner 134, known coordinates of the position target 186 ( FIG. 2 ), data regarding the target object, etc. The data 218 may include one or more tables, functions, algorithms, models, equations, etc., according to exemplary aspects of the present disclosure.

[0043]

[0056] The computing system 210 also includes a communication interface 220 used, for example, to communicate with other components of the inspection system 200. The communication interface 220 may include any suitable components for interacting with one or more networks, including, for example, a transmitter, a receiver, a port, a controller, an antenna, or other suitable components.

[0044]

[0057] As further shown in FIG. 4 , in some example embodiments, the instructions 216 may include an image analysis module 222 that includes an image stitching container 224 and a defect detection container 226. The one or more processors 212 of the computing system 210 may receive multiple captured images 228 from the 3D scanner 134 and stitch the captured images together to create slice images that represent “slices” of a large area of ​​the target object. The slice images may be combined or stitched together with other slice images of the target object to ultimately render a combined slice image of the target object. When the one or more processors 212 of the computing system 210 execute the defect detection container 226, one or more defects in the target object may be detected in the slice images and / or the combined slice images and / or the combined image. In at least some embodiments, coordinates of the defects may be identified, and defect indicators 230 corresponding to the coordinates of the detected defects may be generated. The computing system 210 may cause the projector 191 of the 3D scanner 134 to project the defect indicators 230 onto the target object. For example, a crosshair, an arrow, a circle, etc. In some aspects, at least one defect indicator may be projected onto the target object to outline the defect. That is, the projected defect indicator may outline the perimeter of the defect. This may make the area of ​​the defect readily apparent to an operator reviewing the defect on the target object.

[0045]

[0058] Further, in some aspects, the instructions 216 may include a digital twin creator 232. When one or more processors of the computing system 210 execute the digital twin creator 232, a digital twin 234 of the target object may be created, for example, with defects noted on the target object. The digital twin 234 may be a digital representation of a portion of the target object (e.g., the lower quadrant of an airplane fuselage) or the entire target object (e.g., the entire airplane fuselage). The digital twin 234 may be used for several purposes, such as providing a baseline assessment of the target object in final assembly. The target object in final assembly can be compared to future scans of the target object (e.g., after rework, at delivery, after being put into the field, etc.). Furthermore, the digital twin 234 may be used for several other reasons.

[0046]

[0059] In some further embodiments, the instructions 216 may include an autonomous movement module 236. In such embodiments, the inspection cart 100 (FIGS. 1-3) may be an autonomous vehicle. As one example, the inspection cart 100 may be a self-driving cart that can be operated autonomously or without human intervention. As another example, the inspection cart 100 may be a self-guided cart that can move autonomously, for example, along a trajectory from one location to another. The autonomous movement module 236 may include the logic necessary to move the inspection cart 100 from one scanning location to another scanning location based on sensor feedback arranged on the inspection cart 100, for example, sensor feedback from a camera disposed on board the inspection cart 100.

[0047]

[0060] The data repository 238 can store various captured images and / or stitched images, digital twin 234, etc. of the target object. Such features can be stored in memory for future use and analysis. The predictive assembly system 240 can be communicatively coupled to other elements of the inspection system 200. The predictive assembly system 240 can call or receive the captured and / or stitched images, digital twin 234, etc., and by understanding the detected defects, the predictive assembly system 240 can provide assembly predictions 242, such as the 3D shape of a shim that can be placed at a joint between two interfacing components to provide enhanced mechanical properties at the joint. In this regard, an integrated approach to inspection and assembly can be realized.

[0048]

[0061] The inspection system 200 of FIG. 4 will be generally described, and exemplary ways in which the inspection system 200 may be used to inspect an airplane fuselage for defects will be provided below.

[0049]

[0062] Figure 5 is a flow diagram of an exemplary method 300 of inspecting a target object using an inspection system. Method 300 will be described in the context of inspection system 200 of Figure 4 inspecting an airplane fuselage. Accordingly, Figures 1-4 may also be referenced in addition to Figure 5. However, it will be understood that method 300 may be performed by inspection systems and inspection carts having configurations different from inspection system 200 and inspection cart 100 of Figures 1-4.

[0050]

[0063] At 302, method 300 may include initiating scanning at least a slice of a target object, which in this example is an airplane fuselage. Some airplane fuselages have a significantly larger area relative to the FOV of a 3D scanner, and therefore, the longitudinal length of the airplane fuselage may be scanned slice by slice. That is, the airplane fuselage may be divided into multiple slices along its longitudinal length. For example, FIG. 6 is a schematic side view of an airplane fuselage 400 illustrating one exemplary manner in which the fuselage 400 may be divided into slices. The fuselage 400 has a forward end 402 and an aft end 404 and extends along a longitudinal direction L1. As shown, the fuselage 400 is divided into multiple slices 406, i.e., slices S1-S14, which in this example represent fourteen (14) slices. The slices 406 are divided along the longitudinal direction L1. The slices 406 may cover the complete longitudinal length of the fuselage 400 or a portion thereof. However, in other exemplary embodiments, slices 406 may be divided along a portion of the longitudinal length of fuselage 400 .

[0051]

[0064] Generally, the inspection cart 100 may be positioned relative to a given slice of the torso 400, the given slice may be scanned by the 3D scanner 134 to capture an image of the given slice of the torso 400, and then the inspection cart 100 may be moved or repositioned relative to a subsequent slice (e.g., an adjacent slice) of the torso 400. With the inspection cart 100 positioned relative to the subsequent slice, the subsequent slice may be scanned by the 3D scanner 134 to capture an image of the subsequent slice, and then repositioned relative to another subsequent slice. This process may be repeated, for example, slice by slice, until, for example, the complete longitudinal length of the torso 400 has been scanned. In at least some embodiments, the slices 406 are defined with a width smaller than the FOV of the 3D scanner 134. For example, whereby the captured images of the slices (or slice images) overlap. This may be advantageous for stitching the slice images together into a combined slice image. At least a portion of the method 300 may be repeated (eg, the scanning aspect), so that scanning of subsequent slices of the target object may also be initiated at 302 .

[0052]

[0065] At 304, the method 300 may include positioning an inspection cart of the inspection system relative to the target object. For example, the inspection cart 100 may be positioned relative to a given slice of the torso 400 to be scanned. The inspection cart may be moved manually, or the inspection cart 100 may move autonomously. In some exemplary embodiments, the inspection cart 100 may be positioned relative to the torso 400 by performing a “standoff process” to ensure the inspection cart 100 is an appropriate distance from the torso 400. In an exemplary standoff process, lasers may be emitted by the first laser emitter 194 and the second laser emitter 195 of the 3D scanner 134. When the lasers converge at a point or within a predetermined distance of each other, it may be determined that the focal distances between the first camera 192 and the second camera 193 and the torso 400 are within a predetermined range or specification. This may ensure that the cameras are positioned sufficiently to capture images of the torso 400 during scanning. In at least some embodiments, the standoff process is performed for each slice of the torso 400 with the 3D scanner positioned in the same position along the trajectory 154, e.g., in the upper or lower position. This may facilitate continuity of the captured images and facilitate stitching the slice images together into a combined slice image. Once the inspection cart 100 is positioned relative to the slice of the torso 400, e.g., with the appropriate standoff, scanning of the slice may begin.

[0053]

[0066] At 306, the method 300 may include translating the 3D scanner relative to the target object. For example, once the inspection cart 100 is positioned relative to the slice of the torso 400, at 306 the 3D scanner 134 may be translated, for example, along the translation path TP. Specifically, the electric motor 164 may be activated to drive the gearbox 166. The gearbox 166 in turn drives the belt of the belt pulley system 168. When the belt is driven, the carriage 132 supporting the 3D scanner 134 may be moved along the track 154, or rather, along the translation path TP. Thus, the 3D scanner 134 is translated. The 3D scanner 134 may, for example, start at one end of the track 154 and translate to the other end of the track. Alternatively, the 3D scanner 134 may be moved along the translation path TP from a predetermined start position to a predetermined end position.

[0054]

[0067] At 308, the method 300 may include scanning the target object using the 3D scanner to capture images of at least a slice of the target object as the 3D scanner is translated at 306. In that case, optical fiducial markers integrated into the inspection cart are positioned within the FOV of the 3D scanner during scanning. For example, the projector 191 of the 3D scanner 134 may project light onto the torso 400. With the light projected onto the torso 400, the first camera 192 and the second camera 193 may capture images of at least a slice of the torso 400 as the 3D scanner 134 is translated. Multiple images may be captured for a given slice. Furthermore, when the first camera 192 and the second camera 193 capture the image, the first fiducial marker rail 174 and the second fiducial marker rail 176, and the first finger 188 and the second finger 190 are positioned within the FOV of the 3D scanner 134, and as a result, these items are present in the captured image.

[0055]

[0068] By way of example, FIG. 7 shows the inspection cart 100 positioned relative to a slice of the port side of the fuselage 400 (e.g., the third slice S3 of the fuselage 400 in FIG. 6 ). As shown in FIG. 7 , the projector 191 of the 3D scanner 134 projects light onto the underbelly of the fuselage 400, and the 3D scanner 134 is positioned in a lower position along the trajectory 154 ( FIG. 1 ). The FOVs of the first camera 192 and the second camera 193 are labeled as “FOV” in FIG. 7 . Notably, the optical fiducial markers 170, 172 are positioned within the FOV, and therefore the position target 186 ( FIG. 2 ) is also within the FOV. The first camera 192 and the second camera 193 may capture images of the fuselage 400 as the 3D scanner 134 is translated along the trajectory 154, or rather, as the 3D scanner 134 is translated along the translation path TP ( FIG. 3 ). The 3D scanner 134 may capture images of the torso 400, for example, at predefined intervals, upon detecting a given positional target, etc. Eventually, the 3D scanner 134 reaches the end of the trajectory 154 or a predefined position. As shown in FIG. 8 , the 3D scanner 134 is shown in an upper position along the trajectory 154. At this point, the 3D scanner 134 traverses along the translation path TP from a lower position ( FIG. 7 ) to an upper position ( FIG. 8 ), capturing images along the way. In FIG. 8 , the FOV is at or just above the window line WL of the torso 400. The window line WL extends along the longitudinal length L1 defined by the torso 400. Thus, in scanning a slice of the torso 400 in FIGS. 7 and 8 , the torso 400 is scanned as follows: the captured image of the slice extends from the midline of the lower abdomen of the torso 400 to the window line WL of the torso 400. At this point, the slice scanned is a slice of the port side, lower quadrant of the fuselage.

[0056]

[0069] As shown in Figures 7 and 8, images of the torso 400 are captured by the first camera 192 and the second camera 193 as the 3D scanner 134 is translated along the trajectory 154. The captured images 228 are shown in Figure 9. In particular, a first image C1 of the torso 400 may be captured with the 3D scanner 134 at a first position, for example, at the bottom of the trajectory 154 as shown in Figure 7. Then, a second image C2 of the torso 400 may be captured with the 3D scanner 134 at a second position along the trajectory 154 that is different from the first position. Then, a third image C3 of the torso 400 may be captured with the 3D scanner 134 at a third position along the trajectory 154 that is different from the first and second positions. This process may continue for capturing subsequent images, for example, a fourth image C4, a fifth image C5, a sixth image C6, a seventh image C7, an eighth image C8, and a ninth image C9. A ninth image C9 may be captured by the first camera 192 and the second camera 193 when the 3D scanner 134 is positioned above the track 154, for example, as shown in FIG. 8 . In the captured images C1-C9, the first and second fiducial marker rails 174, 176 and their first and second fingers 188, 190 are visible, and as a result, the position target 186 is also present in the captured images C1-C9. The position target 186 may be used to stitch the captured images 228 together at 310. In other aspects, more or fewer than nine (9) images may be captured during the scanning of a slice of the torso 400.

[0057]

[0070] At 310, the method 300 may include stitching together captured images of the target object acquired during the scan to create slice images. The captured images are stitched together according to position targets provided by the optical fiducial markers. For example, one or more processors 212 of the computing system 210 may execute the image stitching container 224, whereby captured images 228, such as images C1-C9 in FIG. 9, are stitched together. The placement of the position targets 186 in the captured images 228 may be identified so that the position of the 3D scanner 134 may be determined. As described above, the position targets 186 may be uniquely positioned (e.g., in a unique pattern) on the first and second fiducial marker rails 174 and 176 and the first and second fingers 188 and 190, thereby allowing their positions to be easily determined. This facilitates rapid processing of the position of the 3D scanner 134. Based on the known position of the 3D scanner 134, the captured images 228 can be stitched together to form slice image SL3, as shown in FIG. 10 . Slice image SL3 can represent, for example, a complete image of the third slice S3. Notably, the first and second optical fiducial markers 170 and 172, and the first and second fingers 188 and 190, can be removed or masked from slice image SL3. For example, because stereo vision is obtained through multiple positions, the first and second optical fiducial markers 170 and 172, and the first and second fingers 188 and 190 can be filtered out of slice image SL3.

[0058]

[0071] At 312, method 300 may include detecting one or more defects in the target object based on the slice image. For example, one or more processors 212 of computing system 210 may execute defect detection container 226, whereby a slice of fuselage 400 in slice image SL3 of FIG. 10 may be analyzed for defects. As shown in FIG. 10 , in this example, defects 246, including first defect 246A and second defect 246B, have been detected for this slice of fuselage 400. In executing defect detection container 226, one or more processors 212 may ignore known features so as not to identify the known features as defects. For example, in executing defect detection container 226, one or more processors 212 may receive data indicating the locations of known features in fuselage 400 that may appear as defects, such as window notches, door notches, rivets, etc. This data may be processed to ensure that such known features are not identified as defects.

[0059]

[0072] At 314, method 300 may include projecting, by the 3D scanner, one or more defect indicators onto the target object to indicate each of the one or more detected defects in the target object. In at least some embodiments, coordinates of the detected defects 246 may be identified, and defect indicators 230 may be generated corresponding to the coordinates of the detected defects 246. The computing system 210 may cause the projector 191 of the 3D scanner 134 to project the defect indicators 230 onto the target object as crosshairs, arrows, circles, etc. In some aspects, the defects 246 may be classified as one of a plurality of defect types, e.g., defect type 1, defect type 2, etc. The defect indicators projected onto the fuselage 400 may be projected in a color associated with the defect type to which the defect was classified. In some aspects, at least one defect indicator may be projected onto the fuselage 400 to outline the defect. In other words, the projected defect indicator may outline the perimeter of the defect. This allows the area of ​​the defect to be easily identified to an operator reviewing the defect in the fuselage 400. By way of example, FIG. 11 shows a first defect indicator 230A being projected onto the fuselage 400 by the projector 191 of the 3D scanner 134. The first defect indicator 230A projected onto the fuselage 400 corresponds to the first detected defect 246A shown in FIG. 10. The first defect indicator 230A outlines the periphery of the first detected defect 246A, easily identifying the area of ​​the first detected defect 246A to the operator. The 3D scanner 134 may be translated along the trajectory 154, whereby defect indicators 230 corresponding to each of the detected defects 246 may be projected onto the fuselage 400.

[0060]

[0073] In some aspects, steps 312 and 314 may be completed at a later stage, such as after each slice of torso 400 has been scanned and the slice images stitched together into a combined slice image.

[0061]

[0074] At 316, method 300 may include determining whether each relevant slice of the target object has been scanned, e.g., according to steps 302 through 314. When each relevant slice has not been scanned, method 300 may repeat from 302, whereby one or more subsequent slices may be scanned. When each relevant slice has been scanned, method 300 may proceed to 318. For example, method 300 may proceed to 318 when slices S1-S14 (FIG. 6) of fuselage 400 have been scanned, or when scanned from nose to tail. Thus, steps 302 through 314 may be repeated for a predetermined length of the target object, e.g., the complete longitudinal length of fuselage 400.

[0062]

[0075] At 318, the method 300 may include stitching the slice images together into a combined slice image. For example, one or more processors 212 of the computing system 210 may stitch the slice images together into the combined slice image. By way of example, FIG. 12 shows a plurality of slice images 252, each associated with one of the predefined slices of the torso 400. While slice images SL1, SL3, SL4, SL5, and SL14 associated with slices S1, S3, S4, S5, and S14 are shown in FIG. 12, it will be understood that slice images SL1-SL14 may be stitched together to form the combined slice image 254. In at least some aspects, the slices are defined with a width smaller than the FOV of the 3D scanner 134. For example, the slice images overlap. This may be advantageous for stitching the slice images together into the combined slice image.

[0063]

[0076] At 320, method 300 may include creating a second combined slice image associated with a second side of the target object. In that case, the combined slice image created at 318 is the first combined slice image associated with the first side of the target object. For example, combined slice image 254 may be the first combined slice image 254 associated with the first side (e.g., the port side) of fuselage 400. A second combined slice image associated with the second side (e.g., the starboard side) of fuselage 400 may be created using, for example, steps 302-318. The second combined slice image may be created to have the same or similar shape as fuselage 400 in first combined slice image 254 presented in FIG. 12 , except that the second combined slice image will identify defects, if any, associated with the second side of fuselage 400.

[0064]

[0077] At 322, method 300 may include storing the combined slice image and / or multiple combined slice images in a data repository. For example, first combined slice image 254 (and in some cases, second combined slice image, etc.) may be stored in a data repository, e.g., in one or more non-transitory memory devices of the data repository. The images may be stored for analysis purposes. For example, predictive assembly system 240 may recall or receive the captured and / or stitched images. Once defects are identified, predictive assembly system 240 may provide one or more assembly predictions 242. The assembly predictions may provide instructions on how to repair the detected defects, such as by providing instructions on how to machine surfaces, build shims to strengthen joints, set rivets according to specifications, rework specific areas to remove dents, etc. In this regard, an integrated approach to inspection and assembly may be realized.

[0065]

[0078] At 324, method 300 may include creating a digital twin of the target object based at least in part on the first merged slice image and, in some cases, one or more other merged slice images. For example, one or more processors 212 of computing system 210 may execute digital twin creator 232 to create a digital twin 234 of fuselage 400. The created digital twin 234 may identify detected defects in fuselage 400. Digital twin 234 may be used for several purposes, such as providing a baseline assessment of fuselage 400 in final assembly, which can be compared to future scans of fuselage 400 (e.g., after rework, at delivery, after entry into the field, etc.). The merged slice image or multiple merged slice images can be retrieved, for example, from data repository 238 or memory device 214.

[0066]

[0079] In some embodiments, to facilitate further efficiency in inspecting the fuselage, the inspection system 200 may include at least one inspection cart associated with a first side (e.g., the port side) of the fuselage 400 and at least one inspection cart associated with a second side (e.g., the starboard side) of the fuselage 400. For example, FIG. 13 shows a front-to-aft view of the fuselage 400, showing a first inspection cart 100A scanning and capturing images of various slices of the port side of the fuselage 400 to detect defects in the fuselage 400, and also showing a second inspection cart 100B scanning and capturing images of various slices of the starboard side of the fuselage 400 to detect defects in the fuselage 400. The first inspection cart 100A and the second inspection cart 100B may be configured and operate in a manner similar to the inspection cart 100 described herein. In some embodiments, the first inspection cart 100A and the second inspection cart 100B may simultaneously perform multiple aspects of the method 300, which may provide further efficiency for inspecting the fuselage 400 for defects.

[0067]

[0080] In still other embodiments, the inspection cart of the inspection system 200 may have an "encirclement tower" that allows the torso to be scanned so that the captured image of a given slice extends from at least the midline of the lower abdomen of the torso to the midline of the upper torso.

[0068]

[0081] By way of example, FIG. 14 is a perspective view of an inspection cart 100C with an encircling tower in a scanner / motivator configuration. The inspection cart 100C includes a tower 130C supporting a track 154C along which a 3D scanner 134C can move. The 3D scanner 134C can be supported, for example, by a carriage. The track 154C is non-linear, or in this example, curved to complement the shape of the fuselage 400. In this regard, the path of movement of the 3D scanner 134C is also curved or non-linear. The track 154C is connected to its top by a roof 111. The roof 111 is supported by a rear post 113. Although not shown in FIG. 14, a curved optical fiducial marker rail shaped complementarily to the track 154C can be provided, thereby positioning the target within the FOV of the 3D scanner 134C.

[0069]

[0082] 14 shows a sequence illustrating one exemplary way in which the 3D scanner 134C can be moved. At a first position P1, the 3D scanner 134C has a first field of view FOV1 and captures images of a portion of the lower abdomen of the torso 400. At a second position P2, the 3D scanner 134C has a second field of view FOV2 and captures images of a lower quadrant of the torso 400. At a third position P3, the 3D scanner 134C has a third field of view FOV3 and captures images of a side of the torso 400. At a fourth position P4, the 3D scanner 134C has a fourth field of view FOV4 and captures images of an upper quadrant of the torso 400. At a fifth position P5, the 3D scanner 134C has a fifth field of view FOV5 and captures images of the upper side of the torso 400. Thus, the inspection cart 100C is positioned to capture an image of a given slice of the torso 400 from at least the midline of the lower abdomen of the torso 400 to the midline of the upper part of the torso 400. In this regard, the slice image created from the image may represent the lower and upper quadrants of one side of the torso 400. This scan range may promote further efficiency in inspecting the torso and may enable detection of defects in both the lower and upper quadrants of the torso.

[0070]

[0083] FIG. 15 is a perspective view of an inspection cart 100D having an encircling tower in a roller coaster-style conveyor configuration. The inspection cart 100D includes a tower 130D supporting a track 154D along which a 3D scanner 134D can travel. The 3D scanner 134D is shown in FIG. 15 at various positions along the track 154D. The 3D scanner 134D can be supported, for example, by a carriage. The track 154D is non-linear, or in this embodiment, curved to complement the shape of the fuselage 400. In this regard, the path of travel of the 3D scanner 134D is also curved or non-linear. The track 154D is supported by a forward brace 115 and a rear brace 117. The forward brace 115 and the rear brace 117 are supported by the chassis 110D. The track 154D includes a backbone 119, first and second load rails 121 and 123 (along which the carriage or 3D scanner 134D travels), and load rail supports 125 that connect the first and second load rails 121 and 123 to the backbone 119 and provide support thereto. The track 154D is uniquely positioned to accommodate a relatively large fuselage such that, when the inspection cart 100D is in position relative to the fuselage 400, the lower end 127 of the track 154D is positioned below the fuselage 400 and the upper end 129 is positioned above the fuselage 400. In this respect, the track 154D "wraps" around a portion of the fuselage 400. Although not shown in FIG. 15 , a curved optical fiducial marker rail shaped complementarily to the track 154D may be provided, thereby positioning the target within the FOV of the 3D scanner 134D.

[0071]

[0084] The inspection cart 100D is positioned to capture an image of a given slice of the torso 400, from at least the midline of the lower abdomen of the torso 400 to the midline of the upper part of the torso 400. In this regard, slice images created from the images can represent the lower and upper quadrants of one side of the torso 400. This scan range can promote greater efficiency in inspecting the torso and can enable detection of defects in both the lower and upper quadrants of the torso. In at least some embodiments, the inspection cart 100D can be an automated guided vehicle (AGA). In such embodiments, the inspection cart 100D can autonomously move from one scan position to the next.

[0072]

[0085] In the present disclosure, reference is made to various embodiments. However, it should be understood that the disclosure is not limited to the particular described embodiments. Instead, any combination of the following features and elements, whether associated with various embodiments or not, is contemplated for implementing and practicing the teachings provided herein. Furthermore, when elements of an embodiment are described in the form of "at least one of A and B," it should be understood that embodiments including element A only, element B only, and elements A and B are each contemplated. Furthermore, while some embodiments may realize other potential solutions and / or advantages over the prior art, whether or not a particular advantage is realized by a given embodiment does not limit the disclosure. Accordingly, the embodiments, features, and advantages disclosed herein are merely exemplary and should not be considered elements of or limit the scope of the appended claim(s) unless expressly recited in the claim(s).

[0073]

[0086] As will be appreciated by one of ordinary skill in the art, aspects described herein may be embodied as a system, method, and / or computer program product. Accordingly, aspects may take the form of entirely hardware aspects, entirely software aspects (including firmware, resident software, microcode, etc.), or aspects combining software and hardware aspects, all of which may be broadly referred to herein as "circuits," "modules," or "systems." Furthermore, aspects described herein may take the form of a computer program product embodied in one or more computer-readable storage medium(s) having computer-readable program code embodied therein.

[0074]

[0087] The program code embodied in the computer readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, fiber optic cable, RF, etc., or any suitable combination thereof.

[0075]

[0088] Computer program code for carrying out operations of aspects of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" programming language or similar programming languages. The program code may run entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider).

[0076]

[0089] Aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to aspects of the present disclosure. It will be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose or special-purpose computer or other programmable data processing device to produce a machine. These instructions, executed via the processor of the computer or other programmable data processing device, thereby create means for performing the function(s) / acts identified in the block(s) of the flowcharts and / or block diagrams.

[0077]

[0090] These computer program instructions may also be stored on a computer-readable medium that may direct a computer, other programmable data processing apparatus, or other device to function in a particular manner. The instructions stored in the computer-readable medium thereby produce an article of manufacture. The instructions include instructions that implement the functions / acts identified in the flowchart and / or block diagram block(s).

[0078]

[0091] Computer program instructions may also be loaded into a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be executed on the computer, other programmable data processing apparatus, or other device to generate computer-implemented processes, whereby the instructions executed on the computer, other programmable data processing apparatus, or other device provide steps for performing the functions / acts identified in the flowchart and / or block diagram block(s).

[0079]

[0092] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various aspects of the present disclosure. As such, each block in the flowcharts and block diagrams may represent a module, segment, or portion of code, including one or more executable instructions for implementing specific logical function(s). In some alternative implementations, the functions shown in the blocks need not occur in the order depicted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially simultaneously, or the blocks may be executed in reverse or out of order, depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a special-purpose hardware-based system that performs particular functions or functions, or by a combination of special-purpose hardware and computer instructions.

[0080]

[0093] While the foregoing is directed to aspects of the disclosure, other and further aspects of the disclosure may be devised without departing from the basic scope thereof, the scope of which is defined by the following claims.

Claims

1. translating (306) a 3D scanner (134) relative to the target object (400), the 3D scanner (134) being disposed within the inspection cart (100); scanning (308) the target object (400) using the 3D scanner (134) to capture images (228) of at least a slice of the target object (400) as the 3D scanner (134) is translated, wherein optical fiducial markers (170, 172) integrated into the inspection cart (100) are positioned within a field of view (FOV) of the 3D scanner (134) during the scanning (308); stitching together (310) the captured images (228) of the target object (400) acquired during the scan (308) to create a slice image (SL3), wherein the captured images (228) are stitched together according to positional targets (186) provided by the optical fiducial markers (170, 172); and The method (300) includes detecting (312) one or more defects (246) in the target object (400) based on the slice image (SL3).

2. 10. The method (300) of claim 1, further comprising projecting (314) one or more defect indicators (230) onto the target object (400) by the 3D scanner (134) to indicate each of the one or more defects (246) detected in the target object (400).

3. 3. The method of claim 2, wherein at least one defect indicator of the one or more defect indicators is projected onto the target object to outline a defect of the one or more defects detected in the target object.

4. 2. The method (300) of claim 1, wherein the 3D scanner (134) is translated along a linear path (TP) oriented at an angle to a vertical direction (V).

5. 5. The method (300) of claim 4, wherein the optical reference markers (170, 172) include a first reference marker rail (174) and a second reference marker rail (176) spaced apart from each other and arranged parallel to the linear path (TP).

6. 6. The method (300) of claim 5, wherein the position targets (186) are disposed on first fingers (188) extending from the first reference marker rail (174) and second fingers (190) extending from the second reference marker rail (176), the first fingers (188) being spaced apart from one another along the first reference marker rail (174) and the second fingers (190) being spaced apart from one another along the second reference marker rail (176).

7. 2. The method (300) of claim 1, further comprising masking the optical reference markers (170, 172) in the slice image (SL3) so that the optical reference markers (170, 172) do not appear in the slice image (SL3).

8. The slice of the target object (400) is a first slice, and the method (300) further comprises: a) moving (302) the examination cart (100) so that the examination cart (100) is aligned with a subsequent slice of the target object (400); b) translating (306) the 3D scanner (134) relative to the target object (400) while the examination cart (100) is aligned with the subsequent slice; c) scanning (308) the target object (400) using the 3D scanner (134) to capture an image (228) of the subsequent slice of the target object (400) as the 3D scanner (134) is translated with the inspection cart (100) aligned with the subsequent slice, wherein the optical fiducial markers (170, 172) are positioned within the field of view (FOV) of the 3D scanner (134) during the scanning (308) of the subsequent slice; d) stitching together (310) the captured images (228) of the subsequent slices of the target object (400) to create subsequent slice images (SL3), wherein the captured images (228) of the subsequent slices are stitched together according to the positional targets (186) provided by the optical fiducial markers (170, 172); and 2. The method (300) of claim 1, further comprising: e) detecting (312) one or more defects (246) on the subsequent slice of the target object (400) based on the subsequent slice image (SL3).

9. repeating steps a) through e) for a predetermined length of said target object (400); and 9. The method (300) of claim 8, further comprising stitching (318) the slice image (SL3) of the first slice and each subsequent slice image (SL3) of the subsequent slices into a combined slice image (SL3).

10. The combined slice image (SL3) is a first combined slice image (SL3) associated with a first side of the target object (400), and the method (300) further comprises:

10. The method (300) of claim 9, comprising generating (320) a second combined slice image (SL3) associated with a second side of the target object (400).

11. The method (300) of claim 10, further comprising creating (324) a digital twin (234) of the target object (400) based at least in part on the first combined slice image (SL3) and the second combined slice image (SL3).

12. 10. The method (300) of claim 8, wherein the inspection cart (100) moves autonomously from one slice of the target object (400) to another slice.

13. 2. The method of claim 1, wherein the target object is a torso, and the torso is scanned such that the captured image of the slice extends from a midline of the lower abdomen of the torso to a window line of the torso.

14. 2. The method of claim 1, wherein the target object is a torso, and the torso is scanned such that the captured image of the slice extends from a midline of the lower abdomen of the torso to a midline of the upper part of the torso.

15. An inspection system (200) comprising an inspection cart (100), the inspection cart (100) comprising: a tower (130) supporting a track (154); a carriage (132) movable along said track (154), the carriage (132) supporting a 3D scanner (134) having at least two cameras (192, 193) and a projector (191); An inspection system (200) comprising optical reference markers (170, 172) positioned relative to the trajectory (154) such that the optical reference markers (170, 172) are within a field of view (FOV) of the 3D scanner (134) during scanning of a target object (400).

16. The method further comprises one or more processors (212) and one or more non-transitory memory devices (214) storing a program, the program, when executed by any combination of the one or more processors (212), causing the one or more processors (212) to perform steps, the steps including: moving the carriage (132) along the track (154) such that the 3D scanner (134) is moved relative to the target object (400); causing the 3D scanner (134) to scan the target object (400) to capture images (228) of at least a slice of the target object (400) as the 3D scanner (134) is moved; stitching together the captured images (228) of the slices of the target object (400) to create slice images (SL3), the captured images (228) being stitched together according to positional targets (186) provided by the optical fiducial markers (170, 172); and The inspection system (200) of claim 15, further comprising detecting one or more defects (246) in the target object (400) based on the slice image (SL3).

17. The process comprises:

17. The inspection system (200) of claim 16, further comprising causing the projector (191) of the 3D scanner (134) to project one or more defect indicators (230) onto the target object (400) to indicate each of the one or more defects (246) detected in the target object (400).

18. 16. The inspection system (200) of claim 15, wherein the track (154) is a linear track (154), and the optical reference markers (170, 172) include a first reference marker rail (174) and a second reference marker rail (176) spaced apart from each other and arranged parallel to the linear track (154).

19. 16. The inspection system (200) of claim 15, wherein the target object (400) is a fuselage (400) and the trajectory (154) is a curved trajectory (154D) extending above and below the fuselage (400).

20. a tower (130) supporting a track (154); a carriage (132) movable along said track (154), the carriage (132) supporting a 3D scanner (134) having at least two cameras (192, 193) and a projector (191); An inspection cart (100) comprising optical reference markers (170, 172) positioned relative to the trajectory (154) such that the optical reference markers (170, 172) are within a field of view (FOV) of the 3D scanner (134) during scanning of a target object (400).

Images