Method and apparatus for measuring the concentricity of a fastener

Abstract

To provide systems and methods for measuring concentricity and flushness of a fastener installed in a part.SOLUTION: The systems and methods inspect a fastener 12 installed at least partially through a hole 14 in a part, by measuring concentricity and flushness of the fastener 12 and / or detecting foreign object debris. The systems include an x-ray imaging system 18, a first camera device 20, a second camera device 22, a first support structure 26, and at least one processing unit 24. The first camera device 20 produces a first image of the fastener from a first vantage point, and the second camera device produces a second image of the fastener from a second vantage point. The systems inspect the fastener based on the x-ray image and / or the 3D image. The systems may be automated and mounted on robot arms to be positioned relative to the fasteners being inspected.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure generally relates to systems and methods for inspecting fasteners, and more particularly to systems and methods for measuring the concentricity and flatness of fasteners attached within a component.

Background Art

[0002] Assemblies such as aircraft can include hundreds or thousands of fasteners, and it typically takes a significant amount of time to inspect and confirm that the fasteners are properly installed. For example, for proper installation of fasteners for a given assembly, the fasteners may need to be orthogonal to the surface to which they are fastened, such that fasteners installed at too great an angle to the surface would be considered defective. When the installation of fasteners is automated, for example, using a robot to drill holes and attach and secure rivets, the ability to perform in-process open hole inspections may be lost. In an automated process, current inspection techniques for inspecting such installed fasteners are of a destructive nature. For example, measurement of the rivet surface is performed using a manual probe that gives a point measurement after the fastener has been removed for inspection. Further, the above techniques tend to be time-consuming and costly and are not very repeatable due to operator dependence.

Summary of the Invention

[0003] The systems and methods disclosed herein can be configured to provide non-destructive, rapid, automated inspection of installed fasteners within an assembly such as an aircraft component. Such systems and methods can be configured to measure the concentricity of the fasteners, the flatness of the fasteners with respect to the surface, and / or detect the presence of foreign debris.

[0004] In one example, a system for inspecting a fastener installed at least partially through a hole in a component may include an X-ray imaging system, a first camera device, a second camera device, a first support structure, and at least one processing unit. The X-ray imaging system may be oriented and configured to generate an X-ray image of the fastener. The first camera device can be positioned and oriented to be configured to generate a first image of the fastener from a first viewpoint, and the second camera device can be positioned and oriented to be configured to generate a second image of the fastener from a second viewpoint. The X-ray imaging system, the first camera device, and the second camera device may be coupled to the first support structure. The first support structure may support the first camera device and the second camera device such that a three-dimensional image of the fastener can be created from the first image and the second image, and may be configured to position the first camera device and the second camera device with respect to the component and the fastener. The at least one processing unit can be configured to generate a three-dimensional image of the fastener from the first image and the second image, and further can be configured to determine concentricity and / or flatness of the fastener based on the X-ray image and the three-dimensional image.

[0005] The disclosed method for inspecting a fastener installed at least partially through a hole in a component generally includes creating an X-ray image of the fastener via an X-ray imaging system and measuring concentricity of the fastener using the X-ray image. Additionally or alternatively, the disclosed method may include generating a three-dimensional image configuration of the fastener using a first image of the fastener and a second image of the fastener, wherein the first image is taken from a first viewpoint via a first camera device and the second image is taken from a second viewpoint via a second camera device, and measuring flatness of the fastener using the three-dimensional image. Also disclosed is a computer-readable medium having non-transitory computer-readable instructions that, when executed by a processing unit, cause the processing unit to perform the disclosed method. BRIEF DESCRIPTION OF THE DRAWINGS

[0006]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

DETAILED DESCRIPTION OF THE INVENTION

[0007] FIG. 1, FIG. 2, and FIG. 5 provide non-limiting examples of system 10 according to the present disclosure. Elements that serve the same or at least substantially the same purpose are denoted with like reference numerals in each of FIG. 1, FIG. 2, and FIG. 5, and details of these elements may not be described herein when referring to each of FIG. 1, FIG. 2, and FIG. 5. Similarly, not all elements are labeled in each of FIG. 1, FIG. 2, and FIG. 5, but the reference numerals associated therewith may be used consistently herein. Elements, components, and / or features described herein with reference to one or more of FIG. 1, FIG. 2, and FIG. 5 may be included in and / or used in any of FIG. 1, FIG. 2, and FIG. 5 without departing from the scope of the present disclosure. Generally, elements that are likely to be included in a given (i.e., particular) embodiment are shown in solid lines, and elements that are optional in a given embodiment are shown in dashed lines. However, elements shown in solid lines are not necessarily essential to all embodiments, and elements shown in solid lines may be omitted from a particular embodiment without departing from the scope of the present disclosure.

[0008] Referring to FIGS. 1, 2, and 5, system 10 is configured to inspect one or more fasteners 12 mounted at least partially through each hole 14 in component 16. In this specification, embodiments will be described with reference to one or more fasteners 12 (e.g., rivets in a wing panel) mounted within an aircraft component. However, system 10 can be used to inspect fasteners 12 mounted in a wide variety of components 16 from any industry including, but not limited to, automotive, aerospace, marine, OEM, military, construction, transportation, shipping, protection, and cargo industries. Fastener 12 can be configured to be at least substantially flush with one side surface of component 16, at least substantially flush with both side surfaces of component 16, and / or extend beyond one or both side surfaces of component 16, as schematically shown in FIG. 1. System 10 is configured to non-destructively inspect fastener 12, i.e., fastener 12 can be inspected without being damaged or removed from component 16. System 10 generally uses a combination of x-ray imaging and image processing, combining computer vision and image processing algorithms for real-time detection and determination to inspect fastener 12. In some examples, system 10 can be further configured to access only one side surface of component 16 to inspect fastener 12.

[0009] System 10 generally includes an x-ray imaging system 18, a first camera device 20, and a second camera device 22. X-ray imaging system 18 is oriented and configured to generate an x-ray image of one or more fasteners 12 within component 16. In some examples, x-ray imaging system 18 can be configured to generate an x-ray image of one fastener 12 (e.g., fastener 12a) at a time and then be moved relative to component 16 to generate other x-ray images of other fasteners (e.g., fastener 12b). In some examples, x-ray imaging system 18 can be configured to generate an x-ray image that includes a plurality of fasteners (e.g., fasteners 12a, 12b, and 12c) within one x-ray image.

[0010] Similarly, the first camera device 20 is arranged and oriented such that it is configured to generate a first image of one or more fasteners 12 within the component 16 from a first perspective. In some examples, the first camera device 20 may be configured to generate a corresponding first image of one fastener 12 (e.g., fastener 12a) at a time, and then may be moved relative to the component 16 to generate other first images of other fasteners (e.g., fastener 12b). In some examples, the first camera device 20 may be configured to generate a first image including a plurality of fasteners (e.g., fasteners 12a, 12b, and 12c) within one X-ray image. Similarly, the second camera device 22 is arranged and oriented such that it is configured to generate a second image of one or more fasteners 12 within the component 16 from a second perspective. In some examples, the second camera device 22 may be configured to generate a corresponding second image of one fastener 12 (e.g., fastener 12a) at a time, and then may be moved relative to the component 16 to generate other second images of other fasteners (e.g., fastener 12b). In some examples, the second camera device 22 may be configured to generate a second image including a plurality of fasteners (e.g., fasteners 12a, 12b, and 12c) within one second image.

[0011] The system 10 also includes at least one processing unit 24 configured to receive information (e.g., X-ray images, first images, and second images) from the X-ray imaging system 18, the first camera device 20, and the second camera device 22. The information can be transmitted wirelessly to the processing unit 24, or the processing unit 24 can be directly electrically connected to the X-ray imaging system 18, the first camera device 20, and / or the second camera device 22. The processing unit 24 may be configured to create a three-dimensional image of one or more fasteners 12 from the information, for example using stereo vision technology. The processing unit 24 is configured to inspect the fasteners 12 based on the X-ray images received from the X-ray imaging system 18 and / or based on the three-dimensional images of the fasteners 12, as described in more detail herein.

[0012] System 10 also includes a first support structure 26 to which an X-ray imaging system 18, a first camera device 20, and a second camera device 22 are coupled. The first support structure 26 is configured to support the first camera device 20 and the second camera device 22 and to position the first camera device 20 and the second camera device 22 relative to the component 16 and the fastener 12 being inspected such that a three-dimensional image of the fastener 12 can be created from a first image generated by the first camera device 20 and a second image generated by the second camera device 22. Generally, the first support structure 26 serves to position the first camera device 20 and the second camera device 22 such that the first camera device 20 and the second camera device 22 image the fastener 12 from different angles or different viewpoints. The support structure 26 generally supports the X-ray imaging system 18 at a position between the first camera device 20 and the second camera device 22, but system 10 is not limited to this configuration. In some examples, the first support structure 26 can include two or more support structures that are connected together or moved separately from each other to position the X-ray imaging system 18, the first camera device 20, and the second camera device 22 relative to the component 16 and the fastener 12.

[0013] The first support structure 26 can be coupled to the first robot 28, and the first robot 28 can be configured to control, orient, move, and / or position the first support structure 26, for example, via the first robot arm 30 (FIG. 2). The first robot arm 30 can be configured to move the first support structure 26 relative to the component 16, thereby adjusting the positions of the X-ray imaging system 18, the first camera device 20, and the second camera device 22 relative to the fastener 12. The system 10 can also include an X-ray detector 32, and the X-ray detector 32 can be coupled to the second support structure 34 and / or the second robot 36 (e.g., via a second robot arm 38 as shown in FIG. 2). The second robot arm 38 can be configured to move the second support structure 34 relative to the component 16, thereby adjusting the position of the X-ray detector 32 relative to the fastener 12. As shown in FIGS. 1 and 2, the X-ray imaging system 18 can be disposed on one side of the component 16, and the X-ray detector 32 can be disposed on the opposite side of the component 16. Correspondingly, the first robot arm 30 can be disposed on one side of the component 16, and the second robot arm 38 can be disposed on the opposite side of the component 16. In a system 10 including the first robot 28 and the second robot 36, the first robot 28 can be a command robot, a primary robot, or a command robot that instructs the movement of the second robot 36, and the second robot 36 can be a secondary robot or a following robot. In some examples, the first robot 28 and the second robot 36 can be calibrated such that they move simultaneously and in parallel, where the second robot arm 38 positions the X-ray detector 32 to face the first support structure 26 and the X-ray imaging system 18 disposed by the first robot arm 30 on the opposite side of the component 16.

[0014] System 10 is configured to measure the concentricity of one or more fasteners 12 and / or the coplanarity of one or more fasteners 12 with respect to component 16. Since the fasteners 12 are configured to be perpendicular to component 16 when properly attached in the embodiments disclosed herein, as used herein, "concentricity" of the fasteners 12 refers to the concentricity between two sides of the fastener. Thus, when properly attached (i.e., such that the longitudinal axis 40 of the fastener 12 is perpendicular to the plane defined by component 16), the circular plane defined by the first side 42 of the fastener 12 is substantially concentric with the circular plane defined by the second side 44 of the fastener 12. As used herein, the concentricity between the first side 42 and the second side 44 of the fastener 12 generally also indicates the concentricity with respect to the hole 14 in which the fastener 12 is installed.

[0015] Referring to FIGS. 3-4, the processing unit 24 (FIG. 1) can be configured to detect a first circle 66 and a second circle 68 from an X-ray image 70 of the fastener 12. The first circle 66 corresponds to the first side 42 of the fastener 12 (e.g., the head of the fastener 12), and the second circle 68 corresponds to the second side 44 of the fastener 12 (e.g., the shaft of the fastener 12b). For example, the processing unit 24 can include instructions for detecting features of a circle whose nominal diameter is the nominal diameter of the fastener 12 (e.g., the head or shaft of the fastener 12). The processing steps can include basic noise removal (Gaussian blurring, median filter, etc.), Canny edge detection, and / or Hough Circle detection.

[0016] The processing unit 24 is further configured to determine a first center 72 of the first circle 66 and a second center 74 of the second circle 68, and is configured to determine an offset or distance between the first center 72 and the second center 74. The processing unit 24 can then determine whether the offset is within a range of a predetermined pass / fail threshold offset distance. If the first center 72 is farther from the second center 74 than the predetermined pass / fail threshold offset distance, this may indicate that the fastener 12 is not sufficiently perpendicular to the component 16, so that the first circle 66 and the second circle 68 are not sufficiently concentric, and thus the fastener 12 is not properly installed within the component 16. In other words, the greater the offset between the first center 72 and the second center 74, the more the fastener 12 is tilted with respect to the component 16. FIG. 6 shows this concept in a different way, showing a fastener 12d properly installed within the component 16, while the fastener 12e is tilted with respect to the component 16 and is thus improperly installed. X-ray images 70 of each of the fasteners 12d, 12e are also shown. For example, the X-ray image 70d of the fastener 12d shows that the first circle 66 and the second circle 68 are substantially concentric, which would correspond to a "pass" result for the measurement of the concentricity b of the fastener 12. On the other hand, the X-ray image 70e of the fastener 12e reveals that the first circle 66 is significantly displaced from the second circle 68 (due to the tilt of the fastener 12e with respect to the component 16), and thus the X-ray image 70e is an example of an image that can be interpreted as showing an improperly installed fastener according to a certain threshold criterion. Correspondingly, the fastener 12e is shown as being improperly installed and can be flagged for repair (e.g., reinstallation), removal, and / or replacement.

[0017] The predetermined threshold offset distance can be stored, for example, in the non-transitory computer-readable memory 76 (also referred to herein as memory 76) (FIG. 1) of the system 10. In this way, the system 10 can determine the quality of the attachment of the fastener 12, which is determined by the concentricity of the two sides 42, 44 of the fastener 12. The processing unit 24 may include an X-ray image processing module for determining such concentricity of the fastener 12. Since the X-ray imaging system 18 can be configured to create X-ray images of both the first side 42 and the second side 44 of the fastener 12 in one X-ray image, it can be said that the system 10 is configured to inspect the first side 42 (also referred to herein as the first portion 42) and the second side 44 (also referred to herein as the second portion 44) of the fastener 12 at least substantially simultaneously.

[0018] The system 10 can be used to inspect the fastener 12 as to whether the fastener is configured to be mounted within the hole 14 flush with the component 16. Herein, "flushness" refers to the degree to which the fastener 12 protrudes beyond the surface of the component 16. In other words, in some examples, the fastener 12 is completely within the scope of the hole 14 when properly mounted, while in other examples, at least a portion of the fastener 12 can be disposed outside the hole 14 when the fastener 12 is correctly mounted. For example, in FIG. 1, the fastener 12b is completely mounted within the hole 14b such that the first side or first portion 42b of the fastener 12b is disposed adjacent to the first surface 46 of the component 16 and the second side or second portion 44b of the fastener 12b is disposed adjacent to the second surface 48 of the component 16, where the first surface 46 faces in the same direction as the first side of the component 16 and the second surface 48 faces in the same direction as the second side of the component 16. On the other hand, the first side 42 and the second side 44 of the fastener 12a are disposed outside their respective holes 14, whereby the first side 42 and the second side 44 protrude beyond the first surface 46 and the second surface 48 of the component 16, respectively.

[0019] In some examples, system 10 is configured to inspect the height of the head of fastener 12 relative to the first surface 46 and / or the second surface 48 of component 16, thereby measuring the flatness of fastener 12. The processing unit 24 may include a three-dimensional reconstruction module configured to determine the flatness of fastener 12 by creating a three-dimensional image of fastener 12 and the surface of component 16. For example, once a three-dimensional image of fastener 12 is created from two or more images of fastener 12, a point cloud of the three-dimensional surface may be generated by processing unit 24. Based on the three-dimensional image, processing unit 24 may be configured to identify the surface of fastener 12 (e.g., the surface corresponding to the first side surface 42) and the surface of component 16 (e.g., the first surface 46). In this case, processing unit 24 may align a first plane with the surface of fastener 12 and a second plane with the surface of component 16 to determine the height difference between the first plane and the second plane, thereby measuring the flatness of fastener 12 relative to the first surface 46 of component 16. In some examples, one or more filters may be applied by processing unit 24 to the point cloud of the three-dimensional surface of fastener 12 to identify these two surfaces.

[0020] Additionally or alternatively, system 10 may be configured to detect debris of foreign matter around fastener 12, such as debris generated during the installation of fastener 12. The processing unit 24 may include a two-dimensional processing module configured to detect debris of foreign matter around fastener 12. In some examples of system 10, processing unit 24 may be configured to use image processing techniques (e.g., image classification algorithms) to identify an object having a surface area different from the surface area of the head of fastener 12 based on an X-ray image of fastener 12, a three-dimensional reconstruction of fastener 12 and / or component 16, and / or a two-dimensional image of fastener 12 and / or component 16. For example, processing unit 24 may detect features of a circle having a diameter that matches the known diameter of the head of fastener 12 and apply a high-pass filter to eliminate the background, thereby detecting debris of foreign matter adjacent to or near fastener 12.

[0021] The memory 76 of the processing unit 24 can store non-transitory computer-readable instructions, which, when executed by the processing unit 24, cause the processing unit 24 to measure the concentricity of the fastener 12, measure the flatness of the fastener 12 with respect to the component 16, and / or detect debris of foreign matter near the fastener 12. Pass / fail threshold criteria for fastener inspection can be stored in the memory 76. Further, the above instructions, when executed by the processing unit 24, can be configured to cause the processing unit 24 to automatically mark, indicate, or flag any detected defective fastener 12.

[0022] The system 10 is configured to inspect a plurality of fasteners 12 mounted within the component 16, and the first robot 28 can be configured to scan the component 16 while the plurality of fasteners 12 are being inspected to determine the contour of the component 16. In this way, the processing unit 24 can determine the movement of the first robot arm 30 based on the scan of the component 16 to position the X-ray imaging system 18, the first camera device 20, and the second camera device 22 for each fastener 12 of the plurality of fasteners being inspected. In some examples, the first support structure 26 is configured to be moved relative to the component 16 during the intervals between subsequent fastener inspections so that the X-ray imaging system 18, the first camera device 20, and the second camera device 22 are correctly positioned and oriented for each corresponding fastener 12 being inspected. The second robot 36 is configured to determine the contour of the component 16 and, thereby, determine the movement of the second robot arm 38 to position the X-ray detector 32 for each fastener 12 of the plurality of fasteners being inspected or relative to the first support structure 26, and can be configured to scan the component in synchronization with the first robot 28 while the plurality of fasteners 12 are being inspected.

[0023] As shown in FIG. 2, the command robot 28 and / or the follower robot 36 can be disposed on the sliding stage 90. The sliding stage 90 can be configured to slide or move within the work space environment to position the first robot 28 and the second robot 36 for inspecting the fasteners 12 within a plurality of different spaced-apart components 16. Additionally or alternatively, the sliding stage 90 can be configured to enable movement of the first robot 28 and / or the second robot 36 relative to the component 16 such that the system 10 can be positioned and oriented as desired for inspecting the fasteners 12.

[0024] In some systems 10, the first support structure 26 can be or can include a first rigid linear platform. For example, the first support structure 26 can be or can include a C-beam rail. Similarly, the second support structure 34 can be or can include a second rigid linear platform, such as a C-beam rail. An example of a structure that can function as the first support structure 26 and / or the second support structure 34 is shown in FIG. 7. The X-ray imaging system 18, the first camera device 20, and the second camera device 22 can be coupled collinearly to the first support structure 26.

[0025] Continuing to refer to FIGS. 1, 2, and 5, in some examples, the X-ray imaging system 18 can be a portable, movable, and / or hand-held X-ray imaging system 18, but generally, the X-ray imaging system 18 can be any X-ray system configured to generate an X-ray image of the fastener 12. In some examples, the first camera device 20 and / or the second camera device 22 can be the first pinhole camera and the second pinhole camera, but generally, the first camera device 20 and the second camera device 22 can be any camera device configured to generate an image of the fastener 12. The system 10 can generally be portable and can be mounted on various robotic arms at different work cells or locations.

[0026] If, as a result of the inspection, the fastener is deemed to not meet the desired criteria (unsatisfactory), the system 10 may be configured to automatically mark or indicate fasteners with incomplete or insufficient attachment. Such fasteners that fail the inspection may be flagged for manual inspection and / or removal from the component 16. In some examples, the system 10 may be configured to inspect each fastener 12 within a given component 16. In other examples, the system 10 may be configured to inspect only a subset of the fasteners 12 within a given component 16, but the system 10 may be configured to inspect additional fasteners 12 in the vicinity of fasteners with incomplete attachment. The first robotic arm 30 can also be used to attach the fasteners 12 within the component 16, and thus the system 10 can inspect such fasteners 12 in real time while they are being attached. The system 10 can also be configured for the automatic examination of X-ray images generated by the X-ray imaging system 18, whereby the concentricity of the fastener can be determined in real time while the fastener is being attached.

[0027] As described above, the X-ray imaging system 18 can be disposed between the first camera device 20 and the second camera device 22. In some examples, the first camera device 20 and the second camera device 22 are symmetrically disposed on both sides of the X-ray imaging system 18 such that the first camera device 20 and the second camera device 22 can be at least substantially equidistant from the X-ray imaging system 18. Additionally or alternatively, the first camera device 20 and the second camera device 22 can be symmetrically disposed on both sides of the center of the hole 14 of the fastener 12 being inspected.

[0028] Referring to FIG. 5, the distance 50 between the first camera device 20 and the second camera device 22, the first angle 52 of the first camera device 20 with respect to the first support structure 26 (FIG. 2), and / or the second angle 54 of the second camera device 22 with respect to the first support structure 26 can be selectively adjusted to place the fastener 12 within the range of the first central region 56 of the first field of view 58 of the first camera device 20 and within the range of the second central region 60 of the second field of view 62 of the second camera device 22. The distance 50 (and / or the angles 52 and 54) between the first camera device 20 and the second camera device 22 can be selected such that the fastener 12 is positioned at the center of the first field of view 58 and the second field of view 62. In some examples, the distance 50 can be about 6 inches, about 8 inches, about 10 inches, about 12 inches, about 16 inches, about 20 inches, about 24 inches, about 28 inches, about 32 inches, about 36 inches, and / or greater than 36 inches. The distance 50 can vary depending on the size of the component 16, the size of the fastener 12, and / or the first angle 52 and the second angle 54. In some examples, the first angle 52 and the second angle 54 can be at least substantially equal. In other examples, the first angle 52 can be different from the second angle 54. In some examples, the first angle 52 and / or the second angle 54 can be about 45 degrees. In other examples, the first angle 52 and / or the second angle 54 can be between 0 degrees and 45 degrees, between 45 degrees and 90 degrees, between 90 degrees and 135 degrees, and / or between 135 degrees and 180 degrees.

[0029] Additionally or alternatively, the first standoff distance 64 between the first support structure 26 and the fastener 12 can be optimized for both the stereoscopic photograph and the X-ray photograph. For example, the first standoff distance 64 can be at least 6 inches, at least 8 inches, at least 10 inches, at least 12 inches, at least 14 inches, at least 16 inches, at least 18 inches, and / or at least 20 inches. In a specific example, the first standoff distance 64 can be between 12 inches and 16 inches. The first standoff distance 64 can be selected or set relative to the distance 50 between the first camera device 20 and the second camera device 22, taking into account the first angle 52 and the second angle 54. The first standoff distance 64 can be less than the distance 50, for example, about 75% of the distance 50, about 50% of the distance 50, and / or about 25% of the distance 50. In some examples, the first standoff distance 64 can be 25 - 75% of the distance 50. In other examples, the first standoff distance 64 can be greater than the distance 50. Additionally or alternatively, the second standoff distance between the X-ray detector 32 (FIG. 2) and the fastener 12 can be selectively adjusted according to the desired external magnification of the X-ray image generated by the X-ray imaging system 18. The first standoff distance 64 can be selectively adjusted via the movement of the first robotic arm 30, and the second standoff distance can be selectively adjusted via the movement of the second robotic arm 38 (FIG. 2).

[0030] Generally, system 10 can include a processing unit 24, where, during operation, the processing unit 24 executes computer-readable instructions (stored in the memory 76 of the processing unit 24) on the fastener 12 to detect any fastener with an insufficient attachment, and if a fastener with an insufficient attachment is detected, the processing unit 24 can automatically indicate the fastener that failed the inspection. Thus, system 10 can function as an automated real-time fastener attachment and inspection system. In a specific example, it is possible to utilize the mounted processing unit 24, such as a Raspberry Pi, to provide various instructions and data sets and perform an analysis of the fastener 12. The processing unit 24 may be mounted and disposed on a first robot 28 coupled to a support structure 26 and / or may be mounted and disposed on a second robot 36. In other examples, the processing unit 24 may be disposed away from the first robot 28 and the second robot 36. The processing unit 24 may be incorporated into the first robot 28 and / or the second robot 36 during manufacturing. In other examples, the first robot 28 and / or the second robot 36 may be equipped (e.g., incorporated) with the processing unit 24 after its initial manufacture.

[0031] Referring now to FIG. 7, a non-limiting example of the first support structure 26 and / or the second support structure 34 in the form of a C-beam rail 78 is shown. Optionally, the reference numerals of the schematic diagrams of FIGS. 1, 2, and 5 are used to refer to the corresponding components in FIG. 7, but the examples of FIGS. 1, 2, and 5 are non-limiting and do not limit the first support structure 26 or the second indicating structure 34 to the illustrated embodiment of FIG. 7. That is, the first support structure 26 or the second indicating structure 34 is not limited to the illustrated C-beam rail 78, but may include any number of various aspects, configurations, features, characteristics, etc. of the first support structure 26 or the second indicating structure 34 shown in the schematic diagrams of FIGS. 1, 2, and 5 and / or the embodiment of FIG. 7 and described with reference thereto, and variations thereof, although it is not necessary to include all such aspects, configurations, features, characteristics, etc. For the sake of brevity, each of the components, parts, portions, aspects, regions, etc. described above or variations thereof may not be described, illustrated, and / or shown again with respect to the C-beam rail 78, but is within the scope of the present disclosure and the features, variations, etc. described above may be utilized therewith.

[0032] FIG. 7 shows an example of the first support structure 26 and / or the second support structure 34 in the form of a C-beam rail 78. The C-beam rail 78 includes a plurality of holes 80 spaced along the length 82 of the C-beam rail 78. For example, the C-beam rail 78 may include a plurality of holes 80 along the first or upper lip or flange 84 and along the second or lower lip or flange 86. In some examples, one or more of the holes 80 are threaded. The holes 80 can be said to include a first plurality of holes 80 along the upper lip 84 and a second plurality of holes 80 along the lower lip 86, where each hole 80 is spaced from each other at each of the upper lip 84 and the lower lip 86, whereby a set screw can be inserted through one or more of the holes 80 in the upper lip 84 and one or more of the holes 80 in the lower lip 86 to limit or inhibit the movement of the X-ray imaging system 18, the first camera device 20, and / or the second camera device 22 via the X-ring 88. The ring 88 can be configured to move linearly along the length 82 of the C-beam rail 78 such that a set screw (or bolt, post, pin, etc.) slides longitudinally along the length 82 of the C-beam rail 78 unless it is inserted into the hole 80 to prevent such sliding of the ring 88. For example, a first set screw inserted into the hole 80a and a second set screw inserted into the hole 80b can substantially limit or inhibit the linear movement of the ring 88a along the C-beam rail 78.

[0033] In some examples, the X-ray imaging system 18 can be coupled to one of the rings 88 (e.g., ring 88a), the first camera device 20 can be coupled to one of the rings 88 (e.g., ring 88b), and the second camera device 22 can be coupled to one of the rings 88 (e.g., ring 88c). In this way, the linear movement of the ring 88 causes the movement of the devices coupled to each ring 88. In some examples, the angle and position of the X-ray imaging system 18 can be adjusted relative to the fastener 12 via the ring 88a. Similarly, the angle and position of the first camera device 20 can be adjusted relative to the fastener 12 via the ring 88b, and the angle and position of the second camera device 22 can be adjusted relative to the fastener 12 via the ring 88c. When moving the first camera device 20 and / or the second camera device 22 (and / or when changing the focus of the first camera device 20 and / or the second camera device 22), the system 10 can be configured to enable correction of lens distortion and calibration of the first camera device 20 and / or the second camera device 22 (e.g., calibration for pixel-to-inch conversion). For example, the calibration of the first camera device 20 and / or the second camera device 22 can be performed using one or more images of a standard checkerboard pattern and a calibration module stored in the processing unit 24. Similarly, when moving the X-ray imaging system 18, the X-ray parameters can be set to optimize the image quality of the resulting X-ray image. For example, an image quality indicator can be disposed on the component 16 and / or the fastener 12 to verify the image quality of the X-ray image.

[0034] Figures 8-9 schematically provide flowcharts representing non-limiting examples of the method according to the present disclosure. In Figures 8-9, some steps are enclosed by dotted lines. This indicates that such steps may be optional or may correspond to an optional version of the method according to the present disclosure. That said, not all methods according to the present disclosure are required to include the steps enclosed by solid lines. The methods and steps shown in Figures 8-9 are not limiting, and as will be apparent from the description herein, other methods and steps having more or fewer steps than the illustrated steps are also included within the scope of the present disclosure.

[0035] Figure 8 shows a method 100 for inspecting a fastener (e.g., fastener 12) attached at least partially through a hole (e.g., hole 14 of part 16) in a part. Method 100 generally includes, at 102, creating an X-ray image of the fastener via an X-ray imaging system (e.g., X-ray imaging system 18), and at 104, measuring the concentricity of the fastener using the X-ray image. Measuring the concentricity at 104 may be performed by one or more processing units (e.g., processing unit 24). Measuring the concentricity at 104 may include detecting a first circle corresponding to a first side of the fastener (e.g., the head of the fastener, e.g., first side 42), detecting a second circle corresponding to a second side of the fastener (e.g., the shaft of the fastener, e.g., second side 44), determining a first center of the first circle (e.g., first center 72 of first circle 66), determining a second center of the second circle (e.g., second center 74 of second circle 68), and determining an offset distance between the first center and the second center. Measuring the concentricity at 104 may also include determining whether the offset distance is within a predetermined pass / fail threshold offset distance and / or indicating whether the offset distance is within the range of the predetermined pass / fail threshold offset distance for each fastener being inspected.

[0036] Method 100 may include, at 120, determining whether a particular fastener is pass / fail (e.g., whether the fastener requirements or mounting criteria are met overall, whether it is properly mounted). For example, by measuring concentricity at 104, if it is shown that the offset distance is not within the range of a predetermined pass / fail threshold offset distance, that may be sufficient criteria for a "pass" determination at 120. Determining pass / fail for a given fastener at 120 generally includes determining whether the fastener is properly mounted for a given environment, and may be based on measuring concentricity at 104, measuring flatness at 108, and / or inspecting for foreign debris at 110. In some examples, determining pass / fail for a given fastener at 120 includes performing a quality threshold calculation.

[0037] Determining pass / fail of a fastener at 120 may include indicating which fasteners are properly mounted and / or which fasteners are improperly mounted. For example, a fastener with improper mounting may be indicated at 120 by recording or noting the location of the fastener with improper mounting, physically marking the fastener with improper mounting, issuing an alert to the worker of the fastener with improper mounting, recording or noting the identification number or other identifier of the fastener with improper mounting, and / or by any other means indicating that a particular fastener is improperly mounted. In some methods 100, at 126, after it is determined at 120 that the corresponding fastener is insufficiently mounted, the corresponding fastener can be removed from the part and the corresponding fastener can be replaced with a newly mounted fastener on the part.

[0038] Method 100 may also include, at 106, creating a three-dimensional image or three-dimensional reconstruction of the fastener using a first image of the fastener and a second image of the fastener. The first image of the fastener is taken from a first perspective (e.g., by the first camera device 20), and the second image of the fastener is taken from a second perspective (e.g., by the second camera device 22). The three-dimensional image of the fastener may be created at 106 by the processing unit using stereovision (stereoscopy), structured light projection, laser scanning, and / or any other suitable technique. For example, creating a three-dimensional image of the fastener at 106 may include imaging two images of the fastener (e.g., via the first camera device and the second camera device), reconstructing the three-dimensional surface of the fastener, and generating a point cloud of the three-dimensional surface using stereovision techniques. Additionally or alternatively, creating a three-dimensional image of the fastener at 106 may include calibrating the first camera device and the second camera device relative to each other and relative to the fastener such that an image configured to generate a three-dimensional reconstruction of the fastener mounted within the component is captured.

[0039] At 108, the flatness of the fastener with respect to the surface of the component may be measured by the processing unit using the three-dimensional image of the fastener. For example, measuring the flatness of the fastener at 108 may include identifying a first surface of the fastener, identifying a second surface of the component, aligning a first plane with the first surface, aligning a second plane with the second surface, determining the height difference between the first plane and the second plane, thereby measuring the flatness of the fastener. One or more filters may be applied to the point cloud of the three-dimensional surface of the fastener to identify the first surface and / or the second surface.

[0040] Additionally or alternatively, at 110, the vicinity of the fastener can be inspected for foreign object debris. For example, detecting foreign object debris at 110 can include using image processing techniques (e.g., image classification algorithms) to identify an object including a surface area different from the surface area of the head of the fastener based on an X-ray image of the fastener, a three-dimensional reconstruction of the fastener and the component surface, and / or a two-dimensional image of the fastener and the component surface. In a specific example, inspecting for foreign object debris at 110 can include applying image processing to detect features of a circle having a diameter that matches a known diameter of the head of the fastener, and applying a high-pass filter to eliminate the background, thereby detecting foreign object debris adjacent to or in the vicinity of the fastener.

[0041] In some methods 100, at 112, an X-ray imaging system can be coupled to a first support structure (e.g., the first support structure 26). At 112, coupling the X-ray imaging system to the first support structure can include coupling the first camera device and / or the second camera device to the first support structure as well. At 114, the X-ray imaging system, together with the first camera device and the second camera device, can be moved and / or positioned relative to components and / or fasteners, for example, by coupling the first support structure to a robotic arm and moving the robotic arm to position the X-ray imaging system, the first camera device, and the second camera device relative to the fasteners and components. Some methods 100 can include scanning the components at 116 before moving and / or positioning the X-ray imaging system at 114, where scanning the components informs the movement of the robotic arm and thus the movement of the X-ray imaging system. Scanning the components at 116 can include determining the contour of the components and planning the movement of the first robotic arm to position the X-ray imaging system for each of a plurality of fasteners being inspected. In some methods 100, scanning the components at 116 can be performed in parallel with measuring concentricity at 104, measuring flatness at 108, and / or inspecting for foreign fragments at 110. For example, it is possible that while the processing unit of the disclosed system is determining concentricity at the current position, other areas of the component are being scanned in preparation for measuring concentricity at the next position of the subsequent fastener being measured.

[0042] Method 100 can include positioning the X-ray imaging system on a first side of the component via movement of the X-ray imaging system at 114, and may further include coupling an X-ray detector (e.g., X-ray detector 32) to a second support structure at 118 and positioning the X-ray detector on a side of the component opposite the X-ray imaging system. Coupling and positioning the X-ray detector at 118 can include selectively adjusting a stand-off distance between the X-ray detector and the component, e.g., based on a desired external magnification of the X-ray image. The second support structure can be coupled to a second robotic arm, e.g., coupled to a second robot, whereby movement of the X-ray detector relative to the component and the fastener (and relative to the first robotic arm and the X-ray imaging system) can be controlled. The second robot can be a follower robot relative to a command robot that controls movement of the first support structure. Thus, the two robotic arms can be configured to move in a coordinated manner such that the X-ray imaging system and the X-ray detector are positioned relative to each other and relative to the fastener to create an X-ray image of the fastener as described herein.

[0043] In some methods 100, for a plurality of fasteners within a given component, flatness and concentricity are measured, and / or the presence of debris of foreign objects is detected. In other words, measuring concentricity at 104, measuring flatness at 108, and / or inspecting the vicinity of the fastener for debris of foreign objects at 110 are performed multiple times, for example, for each fastener being inspected. In the same example, method 100 may include moving and / or positioning the X-ray imaging system at 114 between each performance of measuring concentricity at 104, measuring flatness at 108, and / or inspecting the vicinity of the fastener for debris of foreign objects at 110. For example, at 104, the concentricity of the first fastener can be measured, at 108, the flatness of the first fastener can be measured, and / or at 110, the first fastener can be inspected for debris of foreign objects in its vicinity, and then, before the concentricity of the second fastener is measured at 104, the flatness of the second fastener is measured at 108, and / or the second fastener is inspected for debris of foreign objects in its vicinity at 110, the X-ray imaging system can be moved and / or positioned at 114.

[0044] In some methods 100, fasteners can be inspected on a zone-by-zone basis. For example, a part having a plurality of fasteners can be divided into two or more different zones, where there are a plurality of fasteners within each zone. When inspecting the fasteners within the part, the disclosed system can inspect a predetermined number (one or more) of fasteners within a given zone, rather than all zones within the part. If the inspected fasteners within a zone are properly attached, at 124, the system can move to a different zone and inspect one or more fasteners within that zone. On the other hand, if it is found that one or more fasteners within a given zone are improperly attached, at 122, one or more other fasteners within the same zone can be inspected beyond the original number of fasteners to be inspected. In this way, the disclosed system inspects a sample of fasteners within the part and can increase sampling within an area or zone where one or more fasteners are determined to be improperly attached. Thus, measuring concentricity at 104 can include measuring the concentricity of a first fastener within a first zone of the part and measuring the concentricity of a second fastener within a second zone of the part. Measuring concentricity at 104 can include measuring the concentricity of at least one fastener in each of a plurality of zones of the part.

[0045] Measuring concentricity at 104, moving and / or positioning an X-ray system at 114, scanning components at 116, creating a three-dimensional image of a fastener at 106, measuring flatness at 108, inspecting for foreign object debris at 110, and / or determining pass / fail status of a fastener at 120 can be implemented by non-transitory computer-readable instructions stored in a computer-readable medium and / or memory of a processing unit of the disclosed system and executable by the processing unit of the system. FIG. 9 schematically shows an exemplary algorithm 200 that can be executed by a processing unit 24 and stored in a memory 76 to inspect one or more fasteners 12 attached within a component 16. Briefly, as indicated by "move to position" at 202, the system can be positioned relative to the fastener being inspected. At 204, an X-ray imaging system control module of the processing unit can be actuated to trigger collection of an X-ray image of the fastener. At 206, an X-ray image processing module of the processing unit is actuated and can perform a concentricity calculation at 208. Either concurrently or in a series of flows, at 210, a vision camera control module of the processing unit can be actuated to trigger collection of optical images by a first camera device and a second camera device. At 212, a two-dimensional image processing module of the processing unit is actuated and can perform detection of foreign object debris at 214. Further, at 216, a three-dimensional reconstruction module of the processing unit is actuated and can perform flatness estimation at 218. At 220, a quantitative quality metric assessment is performed by the processing unit, and at 224, it is determined whether a given fastener being inspected passes all measured criteria. If all measured criteria have a "pass" result, the inspected fastener is considered to be properly attached, but if one or more measured criteria have a "fail" result, the inspected fastener is considered to be improperly attached and flagged for removal or repair.

[0046] Non-exclusive examples of the subject matter of the invention according to the present disclosure are described in the paragraphs listed below.

[0047] A1. A system (10) for inspecting a fastener (12) attached at least partially through a hole (14) in a component (16), an X-ray imaging system (18) oriented and configured to generate an X-ray image (70) of the fastener (12), a first camera device (20) arranged and oriented to generate a first image of the fastener (12) from a first perspective, a second camera device (22) arranged and oriented to generate a second image of the fastener (12) from a second perspective, a first support structure (26) to which the X-ray imaging system (18), the first camera device (20), and the second camera device (22) are coupled, the first support structure (26) being configured to support the first camera device (20) and the second camera device (22) such that a three-dimensional image of the fastener (12) can be created from the first image and the second image, and to position the first camera device (20) and the second camera device (22) with respect to the component (16) and the fastener (12), at least one processing unit (24) configured to generate a three-dimensional image of the fastener (12) from the first image and the second image, the at least one processing unit (24) being further configured to inspect the fastener (12) based on the X-ray image (70) and the three-dimensional image, A system (10) comprising.

[0048] A1.1. The system (10) according to paragraph A1, wherein the system (10) is configured to measure the concentricity of the fastener (12).

[0049] A1.2. The system (10) according to paragraph A1 or A1.1, wherein the system (10) is configured to measure the flatness of the fastener (12) with respect to the component (16).

[0050] The system (10) according to any one of paragraphs A1 to A1.2, further comprising an X-ray detector (32), wherein the X-ray imaging system (18) is arranged on a first side of the component (16), and the X-ray detector (32) is arranged on a second side of the component (16), whereby the X-ray imaging system (18) and the X-ray detector (32) are present on opposite sides of each other with the component (16) therebetween.

[0051] The system (10) according to paragraph A2, wherein the X-ray detector (32) is coupled to a second support structure (34).

[0052] The system (10) according to any one of paragraphs A1 to A3, wherein the first support structure (26) includes a linear platform of a first rigidity.

[0053] The system (10) according to any one of paragraphs A1 to A4, wherein the first support structure (26) includes a C-beam rail (78).

[0054] The system (10) according to any one of paragraphs A1 to A4.1, wherein the first support structure (26) includes a plurality of holes (14) spaced along the length of the first support structure (26).

[0055] The system (10) according to paragraph A4.2, wherein the plurality of holes (14) includes a plurality of threaded holes (14).

[0056] The plurality of holes (14) includes a first plurality of holes (14) spaced along a first lip (84) of the first support structure (26), and the plurality of holes (14) includes a second plurality of holes (14) spaced along a second lip (86) of the first support structure (26). One or more of the first plurality of holes (14) and one or more of the second plurality of holes (14) are inserted through a set screw configured to limit or suppress movement of the X-ray imaging system (18), the first camera device (20), and the second camera device (22), the system (10) according to paragraph A4.2 or A4.3, wherein the first plurality of holes (14) are spaced from the second plurality of holes (14).

[0057] A5. The system (10) according to any one of paragraphs A1 to A4.1, wherein the second support structure (34) includes a second rigid linear platform.

[0058] A5.1. The system (10) according to any one of paragraphs A1 to A5, wherein the second support structure (34) includes a C-beam rail (78).

[0059] A5.2. The system (10) according to any one of paragraphs A1 to A5.1, wherein the second support structure (34) includes a plurality of holes (14) spaced along the length of the second support structure (34).

[0060] A6. The system (10) according to any one of paragraphs A1 to A5.2, wherein the first support structure (26) is coupled to the first robotic arm (30).

[0061] A7. The system (10) according to paragraph A6, wherein the first robotic arm (30) moves the first support structure (26) relative to the component (16), thereby adjusting the positions of the X-ray imaging system (18), the first camera device (20), and the second camera device (22) relative to the fastener (12).

[0062] A7.1. The system (10) according to paragraph A7, wherein the first robotic arm (30) is coupled to a command robot (28).

[0063] A7.2. The system (10) according to paragraph A7.1, wherein the command robot (28) is disposed on a sliding stage (90).

[0064] System (10) is configured to inspect a plurality of fasteners (12) mounted within a component (16), a command robot (28) is, by determining the contour of the component (16), determining the movement of the first robotic arm (30), and for positioning the X-ray imaging system (18), the first camera device (20), and the second camera device (22) with respect to each fastener (12) of the plurality of fasteners (12) being inspected, the system (10) according to paragraph A7.1 or A7.2, which is configured to scan the component (16) while the plurality of fasteners (12) are being inspected.

[0065] System (10) is configured to inspect a plurality of fasteners (12) mounted within a component (16), a first support structure (26) is, configured to be moved relative to the component (16) between inspections of each successive fastener (12) such that the X-ray imaging system (18), the first camera device (20), and the second camera device (22) are correctly positioned and oriented with respect to each corresponding fastener (12) being inspected, the system (10) according to any one of paragraphs A1 to A7.3.

[0066] System (10) according to any one of paragraphs A1 to A7.4, wherein a second support structure (34) is coupled to a second robotic arm (38).

[0067] System (10) according to paragraph A8, wherein the second robotic arm (38) is configured to move the second support structure (34) relative to the component (16), thereby adjusting the position of the X-ray detector (32) with respect to the fastener (12).

[0068] System (10) according to paragraph A9, wherein a first robotic arm (38) is coupled to a command robot (36).

[0069] The system (10) according to paragraph A9.1, wherein a following robot (36) is arranged on a sliding stage (90).

[0070] The system (10) is configured to inspect a plurality of fasteners (12) attached within a component (16), and the following robot (36) determines the movement of a second robot arm (38) by determining the contour of the component (16) and arranges an X-ray detector (32) for each fastener (12) of the plurality of fasteners (12) being inspected or with respect to a first support structure (26). The system (10) according to paragraph A9.1 or A9.2, which is configured to scan the component (16) synchronously with a command robot (28) while the plurality of fasteners (12) are being inspected.

[0071] The system (10) according to any one of paragraphs A1 to A9.3, wherein a first robot arm (30) is arranged on a first side of the component (16) and a second robot arm (38) is arranged on a second side of the component (16).

[0072] The system (10) according to any one of paragraphs A1 to A10, wherein the component (16) includes a panel.

[0073] The fastener (12) is attached so that a first portion of the fastener (12) is adjacent to a first surface (46) of the component (16) and a second portion of the fastener (12) is adjacent to a second surface (48) of the component (16), penetrates completely through a hole (14), and the first surface (46) faces a first side of the component (16) and the second surface (48) faces a second side of the component (16). The system (10) according to any one of paragraphs A1 to A11.

[0074] The system (10) according to paragraph A12, wherein the system (10) is configured to inspect at least substantially simultaneously the first part of the fastener (12) and the second part of the fastener (12).

[0075] The system (10) according to any one of paragraphs A1 to A12.1, wherein the system (10) is configured to nondestructively inspect the fastener (12).

[0076] The system (10) according to any one of paragraphs A1 to A13, wherein the system (10) is configured to detect the concentricity of the fastener (12), the orientation of the fastener (12), and fragments of foreign matter generated during the attachment of the fastener (12).

[0077] The system (10) according to any one of paragraphs A1 to A14, wherein the system (10) is configured to inspect the height of the head of the fastener (12) with respect to the first surface (46) of the component (16) and / or with respect to the second surface (48) of the component (16).

[0078] The system (10) according to any one of paragraphs A1 to A14.1, wherein the system (10) is configured to inspect the fastener (12) by accessing only one side of the component (16).

[0079] The system (10) according to any one of paragraphs A1 to A15, wherein the X-ray imaging system (18) includes a portable, movable, and / or handheld X-ray imaging system (18).

[0080] The system (10) according to any one of paragraphs A1 to A16, wherein the first camera device (20) includes a first pinhole camera.

[0081] The system (10) according to any one of paragraphs A1 to A17, wherein the second camera device (22) includes a second pinhole camera.

[0082] A19. The system (10) is such that the first angle (52) and / or the first position of the X-ray imaging system (18) relative to the fastener (12) can be selectively adjusted via one or more rings (88) and / or one or more set screws operably coupled to the first support structure (26), the system (10) according to any one of paragraphs A1 to A18.

[0083] A19.1. The X-ray imaging system (18) is coupled to one or more rings (88), the one or more rings (88) are configured to linearly move along the first support structure (26), and the one or more rings (88) are configured to be held in a fixed position via one or more set screws, the system (10) according to any one of paragraphs A1 to A19.

[0084] A20. The system (10) is such that the second angle (54) and / or the second position of the first camera device (20) relative to the fastener (12) can be selectively adjusted via one or more rings (88) and / or one or more set screws operably coupled to the first support structure (26), the system (10) according to any one of paragraphs A1 to A19.1.

[0085] A20.1. The first camera device (20) is coupled to one or more rings (88), the one or more rings (88) are configured to linearly move along the first support structure (26), and the one or more rings (88) are configured to be held in a fixed position via one or more set screws, the system (10) according to any one of paragraphs A1 to A20.

[0086] A21. The system (10) is such that the third angle and / or the third position of the second camera device (22) relative to the fastener (12) can be selectively adjusted via one or more rings (88) and / or one or more set screws operably coupled to the first support structure (26), The system (10) according to any one of paragraphs A1 to A20.1, comprising

[0087] A21.1. The system (10) according to any one of paragraphs A1 to A21, wherein a second camera device (22) is coupled to one or more rings (88), the one or more rings (88) are configured to linearly move along a first support structure (26), and the one or more rings (88) are configured to be held in a fixed position via one or more set screws.

[0088] A22. The system (10) according to any one of paragraphs A1 to A21.1, wherein an X-ray imaging system (18) is disposed between a first camera device (20) and a second camera device (22).

[0089] A23. The system (10) according to any one of paragraphs A1 to A22, wherein the first camera device (20) and the second camera device (22) are symmetrically disposed on both sides of the X-ray imaging system (18).

[0090] A24. The system (10) according to any one of paragraphs A1 to A23, wherein the first camera device (20) and the second camera device (22) are symmetrically disposed on both sides of the center line of the hole (14).

[0091] A25. The system (10) is configured to arrange a fastener (12) within a first central region (56) of a first field of view (58) of the first camera device (20) and within a second central region (60) of a second field of view (62) of the second camera device (22), and configured such that the distance between the first camera device (20) and the second camera device (22), the first angle (52) of the first camera device (20) with respect to the first support structure (26), and the second angle (54) of the second camera device (22) with respect to the first support structure (26) can be selectively adjusted, according to any one of paragraphs A1 to A24.

[0092] The system (10) according to any one of paragraphs A1 to A25, wherein a first stand-off distance between the first support structure (26) and the fastener (12) is optimized for both the stereoscopic photograph and the X-ray photograph.

[0093] The system (10) according to any one of paragraphs A1 to A26, wherein the first stand-off distance is at least 6 inches, at least 8 inches, at least 10 inches, at least 12 inches, at least 14 inches, at least 16 inches, at least 18 inches, and / or at least 20 inches.

[0094] The system (10) according to any one of paragraphs A1 to A27, wherein the first stand-off distance is 12 to 16 inches.

[0095] The system (10) according to any one of paragraphs A1 to A28, wherein the system (10) is configured such that a second stand-off distance between the X-ray detector (32) and the fastener (12) is selectively adjusted according to a desired external magnification of the X-ray image (70).

[0096] The system (10) according to any one of paragraphs A1 to A29, wherein the system (10) is configured to automatically mark a defective fastener (12) if the inspection result of the fastener (12) does not meet the desired criteria.

[0097] The system (10) according to any one of paragraphs A1 to A30, wherein the X-ray imaging system (18), the first camera device (20), and the second camera device (22) are coupled collinearly to the first support structure (26).

[0098] The system (10) according to any one of paragraphs A1 to A31, wherein the first robotic arm (30) is configured to attach the fastener (12) into the component (16), and the system (10) is configured to inspect the fastener (12) in real time.

[0099] The system (10) according to any one of paragraphs A1 to A32, configured for automated inspection of an X-ray image (70) to determine the concentricity of a fastener (12).

[0100] A34. At least one processing unit (24) is configured to detect a first circle (66) corresponding to a first side (42) of the fastener (12) and a second circle (68) corresponding to a second side (44) of the fastener (12), At least one processing unit (24) is further configured to determine a first center (72) of the first circle (66) and a second center (74) of the second circle (68), and to determine an offset distance between the first center (72) and the second center (74), the system (10) according to any one of paragraphs A1 to A33.

[0101] A35. At least one processing unit (24) is configured to determine whether the offset distance is within a range of a predetermined pass / fail threshold offset distance, the system (10) according to paragraph A34.

[0102] A35.1. The predetermined pass / fail threshold offset distance is stored in a non-transitory computer-readable memory (76) of the system (10), the system (10) according to paragraph A35.

[0103] A36. At least one processing unit (24) is configured to determine the quality of attachment of the fastener (12), the system (10) according to any one of paragraphs A1 to A35.1.

[0104] A37. At least one processing unit (24) includes an X-ray image processing module configured to determine the concentricity of the fastener (12), the system (10) according to any one of paragraphs A1 to A36.

[0105] The system (10) according to any one of paragraphs A1 to A37, wherein at least one processing unit (24) includes a two-dimensional processing module configured to detect debris of foreign matter around the fastener (12).

[0106] The system (10) according to any one of paragraphs A1 to A38, wherein at least one processing unit (24) includes a three-dimensional reconstruction module configured to detect the flatness of the fastener (12) with respect to the component (16).

[0107] Further comprising a memory (76) storing non-transitory computer-readable instructions, The non-transitory computer-readable instructions, when executed by at least one processing unit (24), cause the at least one processing unit (24) to measure the concentricity of the fastener (12), measure the flatness of the fastener (12) with respect to the component (16), and / or detect debris of foreign matter near the fastener (12) The system (10) according to any one of paragraphs A1 to A39.

[0108] The system (10) according to paragraph A40, wherein pass / fail threshold criteria for the fastener (12) are stored in the memory (76).

[0109] The system (10) according to paragraph A40 or A41, wherein the instructions, when executed by at least one processing unit (24), cause the at least one processing unit (24) to automatically mark any fastener (12) with a detected defect.

[0110] A method (100) for inspecting a fastener (12) attached at least partially through a hole (14) in a component (16), comprising: creating an X-ray image (70) of the fastener (12) via an X-ray imaging system (18) (102); Measuring the concentricity of the fastener (12) using an X-ray image (70) (104), A method (100) comprising this.

[0111] B2. Measuring the concentricity (104) is performed by at least one processing unit (24), the method (100) described in paragraph B1.

[0112] B3. Further comprising creating a three-dimensional image of the fastener (106) using a first image of the fastener (12) and a second image of the fastener (12), The first image is taken from a first viewpoint via a first camera device (20), and the second image is taken from a second viewpoint via a second camera device (22), the method (100) described in paragraph B1 or B2.

[0113] B4. Creating a three-dimensional image of the fastener (106) is performed by at least one processing unit (24), the method (100) described in paragraph B3.

[0114] B5. Further comprising measuring the flatness of the fastener (108) using the three-dimensional image, the method (100) described in paragraph B3 or B4.

[0115] B6. Measuring the flatness (108) is performed by at least one processing unit (24), the method (100) described in paragraph B5.

[0116] B7. Further comprising inspecting the vicinity of the fastener (12) for foreign object fragments (110), the method (100) described in any one of paragraphs B1 to B6.

[0117] B8. Further comprising coupling an X-ray imaging system (18), a first camera device (20), and a second camera device (22) to a first support structure (26), the method (100) described in any one of paragraphs B1 to B7.

[0118] The method (100) according to paragraph B8, further comprising moving a first support structure (26) relative to a fastener (12) via a first robotic arm (30).

[0119] B10. An X-ray imaging system (18) and an X-ray detector (32) are arranged on opposite sides of a component (16) with the component (16) therebetween, arranging the X-ray imaging system (18) on a first side of the component (16), arranging the X-ray detector (32) on a second side of the component (16), The method (100) according to any one of paragraphs B1 to B9, further comprising.

[0120] B11. The method (100) according to paragraph B10, further comprising coupling an X-ray detector (32) to a second support structure (34) (118).

[0121] B12. The method (100) according to paragraph B11, further comprising moving a second support structure (34) relative to a fastener (12) via a second robotic arm (38).

[0122] B13. In order to measure the concentricity of a plurality of fasteners (12) of a component (16), performing multiple times measuring the concentricity of the fastener (12) (104), moving the X-ray imaging system (18) relative to the component (16) between each performance of measuring the concentricity (104), The method (100) according to any one of paragraphs B1 to B12, further comprising.

[0123] B14. In order to arrange the X-ray imaging system (18) for each of a plurality of fasteners (12) being inspected, scanning the component (16) (116), thereby, determining the contour of the component (16) and planning the movement of the first robotic arm (30), The method (100) according to any one of paragraphs B1 to B13, further comprising.

[0124] The method (100) according to paragraph B14, wherein scanning the component (16) (116) is carried out in parallel with measuring the concentricity of the fastener (12) (104).

[0125] The method (100) according to any one of paragraphs B1 to B14.1, wherein measuring the concentricity (104) is carried out non-destructively.

[0126] B16. Measuring the concentricity (104) comprises detecting a first circle (66) corresponding to a first side (42) of the fastener (12), detecting a second circle (68) corresponding to a second side (44) of the fastener (12), determining a first center (72) of the first circle (66), determining a second center (74) of the second circle (68), determining an offset distance between the first center (72) and the second center (74), and the method (100) according to any one of paragraphs B1 to B15.

[0127] The method (100) according to paragraph B16, further comprising determining whether the offset distance is within a range of a predetermined pass / fail threshold offset distance.

[0128] The method (100) according to paragraph B17, further comprising indicating that there is a defect in the fastener (12) when the offset distance is not within the range of the predetermined pass / fail threshold offset distance.

[0129] The method (100) according to any one of paragraphs B1 to B18, wherein the method (100) is carried out using the system (10) according to any one of paragraphs A1 to A42.

[0130] The first camera device (20) and the second camera device (22) are configured to capture an image configured to generate a three-dimensional reconstruction of a fastener (12) mounted within a component (16). The method (100) according to any one of paragraphs B1 to B19, further comprising calibrating the first camera device (20) and the second camera device (22) relative to each other and relative to the fastener (12).

[0131] Measuring the concentricity of the fastener (12) (104) includes measuring the concentricity of the first fastener (12) within the first zone of the component (16). The method (100) according to any one of paragraphs B1 to B20, further comprising measuring the concentricity of a second fastener (12) within a second zone of the component (16).

[0132] Measuring the concentricity of the fastener (12) (104) includes measuring the concentricity of at least one fastener (12) within each of a plurality of zones of the component (16). The method (100) according to any one of paragraphs B1 to B21.

[0133] Measuring the concentricity (104) The method (100) according to paragraph B22, includes measuring the concentricity of an additional fastener (12) within each of the plurality of zones if another fastener (12) within that zone is determined to be defective.

[0134] The method (100) according to any one of paragraphs B1 to B23, further comprising selectively adjusting a standoff distance between the X-ray detector (32) and the component (16) based on a desired external magnification of the X-ray image (70).

[0135] Determining (120) that the attachment of the corresponding fastener (12) of the component (16) is insufficient, Removing (126) the corresponding fastener (12) from the component (16), Replacing the corresponding fastener (12) with a fastener (12) newly installed within the component (16), The method (100) according to any one of paragraphs B1 to B24, further comprising

[0136] B26. The method (100) according to any one of paragraphs B1 to B25, further comprising generating a three-dimensional reconstruction of the fastener (12) installed within the component (16).

[0137] B27. Generating the three-dimensional reconstruction includes stereo imaging using a first camera device (20) and a second camera device (22), the method (100) according to paragraph B26.

[0138] B28. Generating the three-dimensional reconstruction includes projection of structured light, the method (100) according to paragraph B26 or B27.

[0139] B29. Generating the three-dimensional reconstruction includes laser scanning, the method (100) according to any one of paragraphs B26 to B28.

[0140] B30. Imaging two images of the fastener (12), Reconstructing the three-dimensional surface of the fastener (12), Generating a point cloud of the three-dimensional surface using stereo vision technology, The method (100) according to any one of paragraphs B1 to B29, further comprising

[0141] B31. Identifying a first surface of the fastener (12), Identifying a second surface (48) of the component (16), Aligning a first plane with the first surface, Aligning a second plane with the second surface (48), Measuring the flatness of the fastener (12) by determining the height difference between the first plane and the second plane (108). The method (100) according to any one of paragraphs B1 to B30, further comprising

[0142] B32. The method (100) according to paragraph B31, wherein identifying the first surface includes applying one or more filters to a point cloud of the three-dimensional surface of the fastener (12).

[0143] B33. The method (100) according to paragraph B31 or B32, wherein identifying the second surface includes applying one or more filters to a point cloud of the three-dimensional surface of the component (16).

[0144] B34. Applying image processing to detect a feature of a circle having a diameter that matches a known diameter of the head of the fastener (12); Applying a high-pass filter to remove the background; Detecting fragments of foreign matter near the fastener (12); The method (100) according to any one of paragraphs B1 to B33, further comprising

[0145] B35. The method (100) according to paragraph B34, wherein detecting fragments of foreign matter includes identifying an object having a surface area different from the surface area of the head of the fastener (12).

[0146] C1. When executed by the processing unit (24), the processing unit (24) A computer-readable medium including non-transitory computer-readable instructions for causing the method (100) according to any one of paragraphs B1 to B35 to be performed.

[0147] C2. When executed by the processing unit (24), the processing unit (24) A computer-readable medium including non-transitory computer-readable instructions for causing the concentricity of the fastener (12) attached within the component (16) to be measured using an X-ray image (70) of the fastener (12).

[0148] When the non-transitory computer-readable instructions are executed by the processing unit (24), the processing unit (24) is caused to generate a three-dimensional reconstruction of the fastener (12) mounted within the component (16) using the first image (12) of the fastener and the second image (12) of the fastener, measure the flatness of the fastener (12) with respect to the surface of the component (16) using the three-dimensional reconstruction, The computer-readable medium according to paragraph C2, which further causes the above to be performed.

[0149] When the non-transitory computer-readable instructions are executed by the processing unit (24), the processing unit (24) is further caused to detect fragments of foreign matter near the fastener (12). The computer-readable medium according to paragraph C2 or C3.

[0150] Use of the system (10) according to any one of paragraphs A1 to A42 for measuring the concentricity of the fastener (12) with respect to the hole (14) in which the fastener (12) is mounted.

[0151] Use of the system (10) according to any one of paragraphs A1 to A42 for measuring the flatness of the fastener (12) with respect to the surface of the component (16) in which the fastener (12) is mounted.

[0152] Use of the system (10) according to any one of paragraphs A1 to A42 for detecting fragments of foreign matter near the fastener (12) mounted within the component (16).

[0153] As used herein, the terms "selective" and "selectively" mean that when changing the operation, movement, configuration, or other activity of one or more components of a device, or one or more characteristics of the device, the particular operation, movement, configuration, or other activity is a direct or indirect result of an aspect of the device or a dynamic process of one or more components, and / or a direct or indirect result of a user operation on an aspect of the device or one or more components. Thus, the terms "selective" and "selectively" can characterize an activity that is a direct or indirect result of a user operation on an aspect of the device or one or more components, or can characterize a process that occurs automatically, for example, via the mechanisms disclosed herein.

[0154] As used herein, the terms "adapted" and "configured" mean that an element, component, or other subject matter of an invention is designed and / or intended to perform a given function. Thus, the use of the terms "adapted" and "configured" should not be construed to mean simply that a given element, component, or other subject matter of an invention is "capable of" performing a given function, but rather should be construed to mean that the element, component, and / or other subject matter of the invention has been specifically selected, created, implemented, utilized, programmed, and / or designed for the purpose of performing that function. It is further included within the scope of the disclosure that an element, component, and / or other described subject matter of an invention described as being adapted to perform a particular function can additionally or alternatively be described as being configured to perform that function, and vice versa. Similarly, a subject matter of an invention described as being configured to perform a particular function can additionally or alternatively be described as being operable to perform that function.

[0155] As used herein, the phrase "at least one" in reference to the listing of one or more items means at least one item selected from one or more of the listed items, but does not necessarily include at least one of each specifically recited item among the listed items, and it should be understood that any combination of items in the listed items is not excluded. By this definition, it is also possible for the phrase "at least one" to optionally include items other than the specifically identified items among the listed items, whether or not related to the specifically identified items. Thus, by way of non-limiting example, "at least one of A and B" (or equivalently, "at least one of A or B", or equivalently "at least one of A and / or B") can, in one embodiment, mean at least one A (optionally including more than one A), where B is absent (and optionally including items other than B), and in another embodiment, can mean at least one B (optionally including more than one B), where A is absent (and optionally including items other than A), and in yet another embodiment, can mean at least one A (optionally including more than one A), and at least one B (optionally including more than one B) (and optionally including items other than B). In other words, the phrases "at least one", "one or more", and "and / or" are open-ended expressions that are conjunctive and disjunctive in their operation. For example, the expressions "at least one of A, B, and C", "at least one of A, B, or C", "one or more of A, B, and C", "one or more of A, B, or C", and "A, B, and / or C" can mean only A, only B, only C, A and B together, A and C together, B and C together, A, B, and C together, and optionally, a combination of any of the foregoing with at least one other item.

[0156] The processing unit can be any suitable device configured to perform the functions of the processing unit described herein. For example, the processing unit can include one or more electronic controllers, proprietary controllers, application-specific controllers, personal computers, application-specific computers, display devices, logic devices, memory devices, and / or a memory having a computer-readable medium suitable for storing computer-executable instructions for implementing aspects of the systems and / or methods according to the present disclosure. Additionally or alternatively, the processing unit can include a non-transitory computer-readable storage or memory suitable for storing computer-executable instructions, or software for implementing the methods and / or steps of the methods according to the present disclosure. Examples of such media include CD-ROMs, disks, hard drives, flash memories, and the like. In this specification, storage, memory, devices having computer-executable instructions, and methods implemented by a computer according to the present disclosure, and other methods are considered to be within the scope of the subject matter of an invention considered to be patentable in accordance with 35 U.S.C. § 101.

[0157] As used herein, the phrase "at least substantially" when modifying a degree or relationship includes not only the described "substantial" degree or relationship but also the full range of the described degree or relationship when changing the degree or relationship. A substantial amount of the described degree or relationship can include at least 75% of the described degree or relationship. For example, a first direction that is at least substantially parallel to a second direction includes a first direction that is within an angular deviation of 22.5° with respect to the second direction, and further includes a first direction that is the same as the second direction.

[0158] The various disclosed elements of the devices and steps of the methods disclosed herein are not necessarily required for all of the devices and methods according to the present disclosure, and the present disclosure includes all novel and non-obvious specific combinations and subcombinations of the various elements and steps disclosed herein. Further, one or more of the various elements and steps disclosed herein may define the subject matter of an independent invention separate from the overall disclosed device or method. Thus, such subject matter of the invention need not be related to the specific devices and methods explicitly disclosed herein, and usefulness in devices and / or methods not explicitly disclosed herein may be found by such subject matter of the invention.

[0159] As used herein, the expressions "for example", "as an example", and / or simply the term "example" are intended to convey that the one or more components, features, details, structures, embodiments, and / or methods according to the present disclosure, when used in connection therewith, are exemplary and non-limiting examples of the foregoing components, features, details, structures, embodiments, and / or methods. Accordingly, the described components, features, details, structures, embodiments, and / or methods are not limiting, required, or exclusive / inclusive, and other components, features, details, structures, embodiments, and / or methods including structurally and / or functionally similar and / or equivalent components, features, details, structures, embodiments, and / or methods are also included within the scope of the present disclosure.

Claims

1. A system (10) for inspecting a fastener (12) attached at least partially through a hole (14) in a component (16), an X-ray imaging system (18) oriented and configured to generate an X-ray image (70) of the fastener (12), a first camera device (20) arranged and oriented to generate a first image of the fastener (12) from a first viewing point, a second camera device (22) arranged and oriented to generate a second image of the fastener (12) from a second viewing point, a first support structure (26) to which the X-ray imaging system (18), the first camera device (20), and the second camera device (22) are coupled, a first support structure (26) configured to support the first camera device (20) and the second camera device (22) such that a three-dimensional image of the fastener (12) can be created from the first image and the second image, and to position the first camera device (20) and the second camera device (22) with respect to the component (16) and the fastener (12), at least one processing unit (24) configured to generate the three-dimensional image of the fastener (12) from the first image and the second image, at least one processing unit (24) further configured to inspect the fastener (12) based on the X-ray image (70) and the three-dimensional image, the system (10) being configured to measure the concentricity of the fastener (12), A system (10) comprising the above.

2. The system (10) according to claim 1, wherein the system (10) is configured to measure the flatness of the fastener (12) with respect to the component (16) using the three-dimensional image of the fastener (12).

3. The first support structure (26) is coupled to a first robotic arm (30) of a command robot (28), The first robotic arm (30) is configured to move the first support structure (26) with respect to the component (16), thereby adjusting the positions of the X-ray imaging system (18), the first camera device (20), and the second camera device (22) with respect to the fastener (12). The system (10) according to claim 1 or 2.

4. The system (10) is configured to inspect a plurality of fasteners (12) mounted within the component (16), a command robot (28) is, by determining the contour of the component (16), determines the movement of the first robotic arm (30) and, for each of the fasteners (12) of the plurality of fasteners (12) being inspected, positions the X-ray imaging system (18), the first camera device (20), and the second camera device (22), configured to scan the component (16) while the plurality of fasteners (12) are being inspected, the system (10) according to claim 3, wherein the first support structure (26) is configured to be moved relative to the component (16) between subsequent inspections of each fastener (12) via the movement of the first robotic arm (30).

5. the first robotic arm (30) is configured to attach the fastener (12) within the component (16), the system (10) according to claim 3, wherein the system (10) is configured to inspect the fastener (12) in real time.

6. further comprising an X-ray detector (32), wherein the X-ray imaging system (18) is disposed on a first side of the component (16) and the X-ray detector (32) is disposed on a second side of the component (16), whereby the X-ray imaging system (18) and the X-ray detector (32) are on opposite sides of each other with the component (16) therebetween, the X-ray detector (32) is coupled to a second support structure (34), the second support structure (34) is coupled to a second robotic arm (38), and the second robotic arm (38) is configured to move the second support structure (34) relative to the component (16), thereby adjusting the position of the X-ray detector (32) relative to the fastener (12), the system (10) according to any one of claims 1 to 5.

7. the system (10) according to any one of claims 1 to 6, wherein the system (10) is configured to inspect the fastener (12) non-destructively.

8. the system (10) according to any one of claims 1 to 7, further configured to detect fragments of foreign matter generated during attachment of the fastener (12).

9. The X-ray imaging system (18) includes a portable X-ray imaging system, the first camera device (20) includes a first pinhole camera, and the second camera device (22) includes a second pinhole camera. The system (10) is configured such that a first angle (52) and a first position of the X-ray imaging system (18) with respect to the fastener (12) can be selectively adjusted via the connection of the X-ray imaging system (18) to the first support structure (26). The system (10) according to any one of claims 1 to 8. **Claim 10** The first support structure (26) includes a C-beam rail (78). The C-beam rail (78) includes a first plurality of holes (14) spaced along a first lip (84) of the first support structure (26), and a second plurality of holes (14) spaced along a second lip (86) of the first support structure (26). Set screws inserted through one or more of the first plurality of holes (14) and one or more of the second plurality of holes (14) can be configured to limit or suppress movement of the X-ray imaging system (18), the first camera device (20), and the second camera device (22). The system (10) according to any one of claims 1 to 9, wherein the first plurality of holes (14) are spaced from the second plurality of holes (14).

Images