Method and system for tracking the position of a scanner

Abstract

Methods and systems for tracking the position of a scanner are disclosed. In particular, systems and methods for tracking the position of a non-destructive inspection (NDI) scanner using images of a target object acquired by the NDI scanner. The system includes a frame, an NDI scanner supported by the frame, a system configured to enable motorized movement of the frame, and a computer system communicatively coupled to receive sensor data from the NDI scanner and track the position of the NDI scanner. The NDI scanner includes a two-dimensional sensor array. Depth sensor data under a surface is repeatedly acquired by the two-dimensional sensor array and output from the two-dimensional sensor array as the surface is at different locations on a target object. The resulting sequence of two-dimensional scan images is fed to an image processing and feature point comparison module configured to track the position of the scanner relative to the target object using virtual features visible in the acquired scan images.

Description

Technical Field

[0001] This disclosure generally relates to systems and methods for tracking the position of a scanner as it moves over a target area (e.g., the surface of a target object). Specifically, this disclosure relates to systems and methods for tracking the position of a non-destructive testing scanner (hereinafter referred to as a "NDI scanner"). As used herein, the term "position" includes location in a coordinate system and orientation relative to that coordinate system. Background Technology

[0002] Various types of imagers can be used to perform non-destructive testing (NDI) on target objects. One or more imagers can move over the portion of the structure to be inspected and acquire scanning image data representing the characteristics or features of the structure, such as the boundaries of an object or surface. For example, ultrasonic data can be obtained using pulse-echo, transmission-through, or shear-wave sensors, such as for thickness measurement, detection of layered defects and porosity, and / or crack detection in structures. Resonant, pitch / catch, or mechanical impedance sensors can be used to provide indications of voids or porosity, such as in adhesive layers of a structure. Furthermore, single-eddy current sensors and dual-eddy current sensors impart and detect eddy currents within a structure to provide data for detecting cracks and / or corrosion, particularly in metallic and other conductive structures.

[0003] As used herein, the term "sensor data" refers to analog data acquired by a two-dimensional (2-D) sensor array, which may be part of an NDI scanner (also referred to herein as a "2-D NDI scanner") that further includes a digital signal processor. As used herein, the term "scanned image data" refers to digital data in the form of a 2-D matrix of pixel values ​​(hereinafter referred to as "pixels") derived from the sensor data. For example, a 2-D sensor array may acquire corresponding analog sensor data, which is converted into a corresponding 2-D pixel matrix representing an image of the subsurface structure of a portion of a target object facing the 2-D sensor array.

[0004] Some existing solutions for inspecting structures include a motion platform (e.g., a robotic tracked vehicle or an end effector mounted to a robot's manipulator arm) with a frame supporting an NDI scanner. This frame can move above the outer contour line of the structure. In alternative embodiments, the motion platform can be designed for manual movement.

[0005] The effective use of such motion platforms typically depends on their precise positioning within a moving environment. Numerous positioning solutions have been developed for this purpose. Some existing positioning systems require separate position measurement components, such as rotary incremental encoders. It may be desirable to provide systems and methods for NDI that avoid the use of rotary incremental encoders dedicated to position tracking functions. Summary of the Invention

[0006] The subject matter disclosed herein is dedicated to systems and methods for tracking the position (hereinafter “localization”) of an NDI scanner using images of a target object acquired by a nondestructive testing (NDI) scanner. The target object has features representing geometric elements, such as discontinuities in object boundaries, depth, and / or material type, and surface boundaries (hereinafter “structural features”). According to some embodiments, the system includes a frame, an NDI scanner supported by the frame, a robotic system configured to enable the frame to move maneuverably, and a computer system communicatively coupled to receive sensor data from the NDI scanner and track the position of the NDI scanner. The NDI scanner includes a two-dimensional sensor array (hereinafter “2-D sensor array”).

[0007] According to some embodiments, when positioned at different locations on the surface of a target object, a 2-D sensor array repeatedly (recursively, continuously) acquires and outputs depth sensor data beneath the surface. The resulting sequence of 2-D scanned images is fed to an image processing and feature point comparison module configured to use virtual features (hereinafter referred to as "features") visible in the acquired scanned images to track the scanner's position relative to the target object. Optionally, a synthetic scanned image construction module is configured to construct a synthetic scanned image comprising (virtual) features corresponding to structural features in the target object by stitching together sequential scanned images.

[0008] The image processing and feature point comparison module is configured to track the position of the NDI scanner relative to a target object using scanned image data obtained from sensor data. The module receives consecutive partially overlapping scanned images and then compares them to find common features. The module is further configured to calculate the scanner's current position on the surface of the target object based on a previous position and the positional change relative to that previous position, where the positional change is determined based on the positional changes of one or more common feature points in the corresponding partially overlapping scanned images.

[0009] As used herein, the term "feature point" means a feature point present in a scanned image. For example, a feature point may be the centroid of a feature. As used herein, the term "common feature" means a feature present in two consecutive scanned images. As used herein, the term "common feature point" means a point of a common feature. For example, a common feature point may include a point of a common feature in a first scanned image and an identical point of the same common feature in a second scanned image. If consecutive scanned images include at least two common feature points, the methods disclosed herein can be used to determine the offset (including position and orientation) between consecutive scanned images. In the case of translation without rotation of the NDI scanner, a common feature point can be used to determine the positional offset between consecutive scanned images.

[0010] According to some embodiments, the synthetic scan image construction module starts with an initial image and then continuously constructs and updates a synthetic image by stitching new image data to the current synthetic image while omitting redundant information. This process is repeated until the entire surface of the structure has been scanned. The synthetic scan image may consist of dozens or even hundreds of scan images, each of which partially overlaps with the immediately preceding and following (temporally) scan images.

[0011] The methods disclosed herein can be used for positioning in both manual and automated NDI applications without requiring a rotary incremental encoder. Optionally, the positioning process may include a calibration step for reorienting the sensor array, such as occasional manual checks and manual calibration of position and orientation. Mapping of system calibration values ​​can also be used to indicate manufacturing errors, such as incorrect positioning or omissions of substructures (e.g., subsurface features).

[0012] While various embodiments of systems and methods for tracking the position of an NDI scanner using scanned images acquired from a target object will be described in detail later herein, one or more of these embodiments can be characterized by one or more of the following aspects.

[0013] One aspect of the subject matter disclosed in detail below is a method for tracking the position of a scanner, the method comprising: (a) placing a sensor array of the scanner adjacent to a surface of a target object such that the sensor array covers a first structural feature of the target object when a reference point on the sensor array is at a first position on the surface of the target object, the first position having known first X-position coordinates and first Y-position coordinates on the surface of the target object; (b) acquiring a first set of sensor data from a first portion of the target object facing the sensor array when the reference point on the sensor array is at the first position; (c) converting the first set of sensor data into first scan image data of a first scan image; and (d) translating the scanner on the surface of the target object until the reference point is at a distance from the first position. (e) At a second position at a certain distance, while the sensor array again covers the first structural feature; (f) when the reference point is at the second position, acquire a second set of sensor data from a second portion of the sensor array facing the target object; (g) convert the second set of sensor data into second scan image data of a second scan image; (h) find feature points in the first and second scan images; (i) determine which feature points found in step (g) are common feature points in the first and second scan images; (j) calculate the pixel position difference between the common feature points; and (h) calculate the second Y position coordinates of the reference point at the second position on the surface of the target object, at least in part based on the first Y position coordinates and the pixel position difference calculated in step (i). Steps (h) and (i) can be performed individually or simultaneously. As used herein, the phrase “find feature points” means searching for pixels or groups of pixels in a scan image that meet the feature selection criteria.

[0014] According to some embodiments of the method described in the preceding paragraph, step (j) includes: calculating a first pixel offset equal to the number of pixel rows of the sensor array by which the position of the common feature point in the second scanned image deviates from the position of the common feature point in the first scanned image; multiplying the first pixel offset by the distance between adjacent sensor rows to form a product equal to a first distance translated by the scanner in a direction perpendicular to the sensor rows in step (d); and adding the first distance to a first Y position coordinate to obtain a second Y position coordinate of the reference point.

[0015] For cases where the sensor translation is not parallel to the Y-axis, the method according to one proposed embodiment further includes: (k) calculating the second X-position coordinates of a reference point on the sensor array on the surface of the target object based at least in part on the second difference between the first X-position coordinates and the corresponding positions of common feature points in the first and second scan images; and calculating the distance between a first point having the first X-position coordinates and the first Y-position coordinates and a second point having the second X-position coordinates and the second Y-position coordinates. In this case, step (j) includes: calculating a first pixel offset equal to the number of pixel rows by which the position of the common feature point in the second scanned image is offset relative to the position of the common feature point in the first scanned image; multiplying the first pixel offset by the distance between adjacent sensor rows to form a product equal to a first distance translated by the scanner in a direction perpendicular to the sensor rows during step (d); and adding the first distance to a first Y position coordinate to obtain a second Y position coordinate. Step (k) includes: calculating a second pixel offset equal to the number of pixel columns by which the position of the common feature point in the second scanned image is offset relative to the position of the common feature point in the first scanned image; multiplying the second pixel offset by the distance between adjacent sensor columns to form a product equal to a second distance translated by the scanner in a direction parallel to the sensor rows during step (k); and adding the second distance to the first X position coordinate to obtain a second X position coordinate.

[0016] According to some embodiments, the method further includes: processing the first scan image and the second scan image by aligning the second scan image data in the second scan image with the first scan image data in the first scan image using common feature points in the first scan image and the second scan image to form a composite scan image; finding feature points in the composite scan image that are associated with a second structural feature in the target object; calculating the third Y position coordinates of the feature points based at least in part on the first Y position coordinates of the reference point; and calculating the third X position coordinates of the feature points based at least in part on the first X position coordinates of the reference point.

[0017] Another aspect of the subject matter disclosed below is a method for tracking the position of a scanner, the method comprising: (a) placing a sensor array of the scanner adjacent to a surface of a target object such that the sensor array covers a first structural feature and a second structural feature of the target object and is oriented at a known first orientation angle; (b) acquiring a first set of sensor data from a first portion of the target object facing the sensor array while the sensor array is oriented at the first orientation angle; (c) converting the first set of sensor data into first scan image data of a first scan image; (d) orienting the scanner such that the sensor array has a second orientation angle different from the first orientation angle and again covers the first and second structural features; (e) acquiring, while the sensor array is oriented at the second orientation angle, acquiring, from a first portion of the target object facing the sensor array. (f) Acquire a second set of sensor data facing the second part of the sensor array; (g) Convert the second set of sensor data into second scan image data of a second scan image; (h) Find feature points in the first scan image and the second scan image; (h) Determine which feature points found in step (g) are common feature points in the first scan image and the second scan image; (i) Calculate the pixel position difference between the first common feature point and the second common feature point; and (j) Calculate a second orientation angle based at least in part on the difference between a first orientation angle and a first angle of a first line connecting the first common feature point and the second common feature point at the corresponding positions in the first scan image and a second angle of a second line connecting the first common feature point and the second common feature point at the corresponding positions in the second scan image.

[0018] According to some embodiments of the method described in the preceding paragraph, step (i) includes: calculating a first pixel offset equal to the number of pixel rows in which the position of the second common feature point in the first scanned image is offset relative to the position of the first common feature point in the first scanned image; calculating a second pixel offset equal to the number of pixel columns in which the position of the second common feature point in the first scanned image is offset relative to the position of the first common feature point in the first scanned image; calculating a third pixel offset equal to the number of pixel rows in which the position of the second common feature point in the second scanned image is offset relative to the position of the first common feature point in the second scanned image; and calculating a fourth pixel offset equal to the number of pixel columns in which the position of the second common feature point in the second scanned image is offset relative to the position of the first common feature point in the second scanned image, wherein the tangent of a first angle is equal to the ratio of the first pixel offset to the second pixel offset, and the tangent of a second angle is equal to the ratio of the third pixel offset to the fourth pixel offset.

[0019] Another aspect of the subject matter disclosed in detail below is a system comprising a frame, a scanner supported by the frame, a robotic system configured to enable motorized movement of the frame, and a computer system communicatively coupled to receive sensor data from the sensors and send control signals to the robotic system for controlling the motorized movement of the frame, wherein the scanner includes a sensor array, and the computer system is configured to perform operations including: (a) activating a sensor at a first position on the surface of a target object where the sensor array covers a first structural feature of the target object at a reference point on the sensor array, the first position having known first X-coordinates and first Y-coordinates on the surface of the target object; (b) receiving a first set of sensor data acquired from a first portion of the target object facing the sensor array when the reference point is at the first position; and (c) converting the first set of sensor data into a first scanned image. (d) Activating the sensor when the sensor array covers a first structural feature of the target object at a second position where the reference point is located at a certain distance from the first position; (e) Receiving a second set of sensor data from a second portion of the sensor array facing the target object when the reference point is at the second position; (f) Converting the second set of sensor data into second scan image data of a second scan image; (g) Finding feature points in the first scan image and the second scan image; (h) Determining which feature points found in step (g) are common feature points in the first scan image and the second scan image; (i) Calculating the pixel position difference between the common feature points in the first scan image and the second scan image; and (j) Calculating the second Y position coordinates of the reference point at the second position on the surface of the target object, at least in part, based on the first Y position coordinates and the pixel position difference calculated in step (i).

[0020] Further aspects of systems and methods for tracking the position of an NDI scanner using scanned images obtained from a target object are disclosed below. Attached Figure Description

[0021] The features, functions, and advantages discussed in the preceding sections can be implemented independently in various embodiments or combined in other embodiments. To illustrate the above and other aspects, various embodiments will now be described with reference to the accompanying drawings. The figures briefly described in this section are not drawn to scale.

[0022] Figure 1 This is a block diagram illustrating some components of a system for tracking the position of an NDI scanner including a two-dimensional sensor array, according to one embodiment.

[0023] Figure 2A This is a diagram showing the full scan area of ​​a target object with subsurface features.

[0024] Figure 2B This indicates that it was obtained from an NDI scanner. Figure 2A This is an illustration of sequentially scanned images captured from the overlapping portions of the full scan area.

[0025] Figure 2C It means including, or in Figure 2A An illustration of sequentially scanned images of pixels with common features obtained in partially overlapping regions.

[0026] Figure 3 This is a flowchart illustrating the steps of a method for tracking a motion platform using images acquired by an NDI scanner including a two-dimensional sensor array, according to one embodiment.

[0027] Figure 4 This is a diagram illustrating the location of common features in individual scan images acquired sequentially at various vertical alignment positions (without rotation) using a two-dimensional sensor array with sensors arranged in rows and columns.

[0028] Figures 5A to 5J This is an illustration showing various three-dimensional views of a two-dimensional array sensor at various locations during scanning of a reinforced body section according to one of the proposed embodiments.

[0029] Figures 6A to 6J It means that they correspond to respectively Figures 5A to 5J A diagram illustrating the stages in constructing a synthetic scan image at the scan position described in the diagram.

[0030] Figures 7A to 7J This is an illustration of a corresponding two-dimensional CAD model image representing a region of the reinforced fuselage, which has features corresponding to... Figures 5A to 5J The corresponding rectangles are superimposed at the scan positions shown.

[0031] Figures 8A to 8D This is a diagram illustrating the various paths used to scan the surface region of an exemplary target object.

[0032] Figure 9 This is a diagram illustrating the location of two common features in individual scan images acquired sequentially at various oriented positions using a two-dimensional sensor array with sensors arranged in rows and columns.

[0033] Figure 10 This is an illustration of some components of a system for tracking and navigating an automated motion platform equipped with an NDI scanner including a two-dimensional sensor array, according to another embodiment.

[0034] Figure 11 This is an explanation of what to use. Figure 10The flowchart in the middle part describes the steps of a method for constructing a synthetic scanned image using an automated motion platform of the type described above. The automated motion platform includes a two-dimensional sensor array and a computer system configured to track the position of the automated motion platform during scanning based on non-visual (e.g., NDI) data acquired by the sensor array.

[0035] Figure 12 This is a flowchart illustrating the steps of a method for acquiring a synthetic scan image of a target object using a movable two-dimensional sensor array according to one embodiment.

[0036] Figure 13 This is a top plan view illustration showing some components of a complete motion device according to one embodiment. The device can be connected to an NDI scanner (not shown).

[0037] Figure 14A and Figure 14B These are illustrations showing a top plan view and a front view of a handheld scanning device, which includes a connection to... Figure 13 The NDI scanner is a complete motion device of the type shown in the diagram.

[0038] Figure 15 This is a block diagram illustrating some components of a system, including a 2D sensor array mounted on a robotic tracked vehicle and a computer system configured to control the scanning position of the 2D sensor array based on images acquired by the 2D sensor array.

[0039] The following reference will be made to the accompanying drawings, in which similar elements in different drawings have the same reference numerals. Detailed Implementation

[0040] For illustrative purposes, a system and method for tracking the position of an NDI scanner using scanned images obtained from a target object will now be described in detail. However, not all features of an actual implementation are described in this specification. Those skilled in the art will understand that in the development of any such embodiment, many implementation-specific decisions must be made to achieve the developer's specific objectives, such as compliance with system-related and business-related constraints, which will vary in different implementations. Furthermore, it should be understood that such development work can be complex and time-consuming, but remains a routine task for those of ordinary skill in the art who benefit from this disclosure.

[0041] According to some embodiments, the system includes an image processing and feature point comparison module configured to track the position of a scanner relative to a target object using scanned image data obtained from sensor data. The image processing and feature point comparison module receives and processes consecutive partially overlapping scanned images, then compares the partially overlapping scanned images to find common feature points. The image processing and feature point comparison module is further configured to calculate the scanner's current position (position and orientation) on the surface of the target object based on a previous position and a position change relative to that previous position. This position change is determined based on the position changes of pixels representing common feature points in the respective partially overlapping scanned images.

[0042] The examples given below are for illustrative purposes only and are non-limiting. The target object may be an aircraft part, such as a barrel-shaped section of an aircraft fuselage. However, it should be understood that the systems and methods described below with reference to fuselage sections can also be applied to other types of workpieces that are part of some other type of vehicle or structure.

[0043] Furthermore, workpieces can be made from any material, depending on the specific application requirements. It should be understood that the type of material used for the workpiece can partially determine which type of non-destructive testing technique will be used to inspect it. By way of non-limiting examples, workpieces can be made of composite materials, such as composite laminates made of fiber-reinforced plastics or metals (such as aluminum or titanium). It should be understood that this is not intended to limit the materials used to manufacture the workpiece to be inspected in any way.

[0044] Depending on the type of material being inspected, any of a variety of NDI sensors can be used. The method presented herein can be applied to any 2-D NDI imager, including those in which a 2-D array of sensors (such as ultrasonic transducers or eddy current coils) is in contact with the surface being inspected. In alternative embodiments, infrared thermal imaging flash systems, terahertz cameras, microwave imagers, or laser Doppler vibration measurement systems can generate non-contact 2-D images digitized / pixelated in XY format, which can be overlaid, aligned, and used for tracking purposes.

[0045] In the context of a specific application involving the inspection of fuselage sections, a scanning system may include means for scanning the skin of the fuselage section from advantageous locations on the exterior of the fuselage section. In the embodiments disclosed below, the scanning means is an NDI scanner in the form of a 2-D sensor array that collects sensor data from the facing portion of the fuselage section. In one proposed embodiment, the NDI scanner scans the mold lines of the fuselage section in a serpentine pattern. For example, the NDI scanner moves vertically upwards parallel to the X-axis.

[0046] As used herein, the terms "X-axis" and "Y-axis" refer to the respective axes that intersect at a right angle at the origin on the surface of the target object and follow the contour of the surface as they extend away from the origin. When the surface is planar (flat), the X-axis and Y-axis are straight, coplanar, and perpendicular to each other. When the surface is curved in the Y direction and straight in the X direction, the Y-axis is locally tangent to the surface. When the surface is curved in both the X and Y directions, both the X and Y axes are locally tangent to the surface. In each case, the Y-coordinate of a point on the surface is measured along the Y-axis and equal to a first distance from the origin, while the X-coordinate of a point on the surface is measured along the X-axis and equal to a second distance from the origin. On the one hand, if the axis is straight, the position coordinate is equal to the distance of the point from the origin; on the other hand, if the axis is curved (because the surface of the target object is curved), the position coordinate is equal to the arc length (not the chord length) from the origin.

[0047] Figure 1 This is a block diagram illustrating some components of a system 10 for tracking the position of a 2-D NDI scanner 14, including a two-dimensional sensor array, according to one embodiment. In this embodiment, the 2-D NDI scanner 14 is mounted on a motion platform 12 (e.g., a robotic tracked vehicle). The motion platform 12 can be of the type that is manually moved by a human operator or of the type that is motorized for automated movement. During scanning operations, the 2-D NDI scanner 14 is positioned at successive locations on the surface of the target object due to the continuous movement of the motion platform 12. At each scanning location, the 2-D NDI scanner 14 is activated to acquire sensor data from the corresponding facing portion of the target object's surface.

[0048] According to some embodiments, the 2-D NDI scanner 14 is rigid, so the sensor array will not directly conform to the surface of the target object. However, in most cases, the 2-D NDI scanner 14 will be mounted in a biaxial universal joint, which will allow the sensor array to be roughly aligned with the surface. (This is a reasonable trade-off, as the curvature of the sensor array is typically small compared to the surface.) Furthermore, if the sensor is an ultrasonic transducer, an acoustic coupling agent (e.g., water or some type of gel-like substance or dry acoustic coupling agent elastomer material) can be used between the sensor array and the surface. The presence of the acoustic coupling agent provides some ability to compensate for slight curvature mismatches.

[0049] exist Figure 1The system 10, partially depicted, also includes an NDI sensor data processor 20 communicatively coupled (via cable or wireless) to the 2-D NDI scanner 14. The NDI sensor data processor 20 is configured to convert sensor data output from the 2-D NDI scanner 14 into 2-D scanned image data 22. Additionally, the system 10 includes an image processing and feature point comparison module 24 communicatively coupled to receive the 2-D scanned image data 22 from the NDI sensor data processor 20. The image processing and feature point comparison module 24 may be a processor or computer configured (e.g., programmed) to track the position (also referred to herein as “localization”) of the 2-D NDI scanner 14 relative to the surface of a target object using the 2-D scanned image data 22. The localization algorithm includes relative motion update processing based on a series of following scan features from one captured NDI scanned image to the next captured NDI scanned image to determine the motion of the 2-D sensor array.

[0050] The NDI sensor data processor 20 does not calculate position data. Instead, it processes sensor data acquired from the 2-D sensor array to create a 2-D scan image that will be used by the image processing and feature point comparison module 24 to determine relative positions. After the image processing and feature point comparison module 24 has calculated the relative position data 26, it can be fed back to the NDI sensor data processor 20 to create a synthetic scan image. If a synthetic scan image is not needed (e.g., if the system is only used for positioning and not for NDI synthetic image creation), the relative position data 22 will not need to be sent back to the NDI sensor data processor 20.

[0051] In some cases, the NDI sensor data processor 20 may only be able to accept relative position data 22 for the creation of synthetic scan images, which is the most commonly used method at present. However, in the future, more advanced techniques may be available for the generation of synthetic scan images, in which case absolute position information may be required. Figure 1 Assuming that the NDI sensor data processor 20 uses relative position data to generate synthetic scan images, however, in other embodiments, the generation of synthetic scan images may be handled by a separate image processing application.

[0052] Figure 2AThe full scan region 46 of the target object with random structural features 11 (represented by dots in this example) is shown. A dashed rectangle superimposed on the full scan region 46 encloses a first portion of the full scan region 46, which is captured in the first scan image 42a when the 2-D NDI scanner 14 is in a first position; when the 2-D NDI scanner 14 is in a second position at a distance less than the length of the 2-D sensor array from the first position, a dotted rectangle superimposed on the full scan region 46 encloses a second portion of the full scan region 46 captured in the second scan image 42b. The first and second portions of the full scan region are shared. Figure 2A The common region 46a is shown. In Figure 2B Scanned images 42a and 42b are shown separately.

[0053] like Figure 2A As shown, the first and second portions of the full-scan regions appearing in scanned images 42a and 42b, respectively, share a common scan region 46a with common feature 11a. Common feature points 48a appear in corresponding regions 58a and 58b of scanned images 42a and 42b, as shown... Figure 2C As shown. Since regions 58a and 58b of scanned images 42a and 42b are images of the same rectangular scanned region 46a, regions 58a and 58b will be identical (hereinafter referred to as "overlapping regions of the scanned images"). These overlapping regions, which have common characteristics in sequential (continuous) scanned images, are used to track a second position of the 2-DNDI scanner 14 relative to the first position. Furthermore, when scanned images 42a and 42b are stitched together to form a composite scanned image, redundant information in one of the scanned images 42a or 42b can be omitted.

[0054] The relative motion update process proposed in this paper is based on the concept that partially overlapping sequential scan images will share some common features (representing structural features in the target object) within the image pair. This paper assumes that the second scan image will share some common features with the first scan image (hereinafter referred to as "common features"). The pixel distance differences between the various locations of the common feature points in the first and second scan images are counted and scaled, and then the relative physical distance is added to the previous position estimate to obtain the new absolute position. In this setting, common features are defined by significant local changes in intensity (high contrast) appearing within the scan images. The system does not need to know which structural features within the physical object are represented by these virtual features; the system only needs to detect the same pattern of pixels in the successive scan images. The entire concept is sometimes referred to as "solving the camera pose problem"—in this case, the "camera" is an NDI scanner. The system tracks a set of points in successive scan images and determines their 2-D positions from one scan image to the next to derive the relative displacement of the common feature points in the scan images. This information is then used to calculate the relative physical motion (position and orientation) of the motion platform 12 on which the 2-D NDI scanner 14 is mounted during the time interval between a scanned image and a subsequent scan. For this to work for both position and orientation, a sufficient number of common features are required. Theoretically, a minimum of 2 is ideal, but more common feature points are preferred to improve the estimation.

[0055] The use of feature point comparisons during localization has been disclosed elsewhere in this paper. For example, the Simultaneous Localization and Mapping (SLAM) method uses data from one or more optical cameras or laser scanners and an extended Kalman filter to: (1) update the current state (position) estimate using dead reckoning data; (2) update the estimated position based on re-observed features (landmarks); and (3) add new features (landmarks) to the current state. In SLAM, the relative displacement of common feature points shared by two images is used to provide the offset estimate. For this, relatively small changes in position and orientation, as well as substantial overlap between the images, are required for registration. Additionally, a known reference size of the 2-D sensor array is needed to determine the magnitude (scale, range) of the displacement.

[0056] Various algorithms based on common feature points can be used to determine the distance the scanner moves during the time interval between two moments when capturing two images. These algorithms can be used to determine the positional and directional offset between the two images. The process involves aligning two sets of common feature points obtained from the two images and determining the amount by which one set of points must be translated and rotated to achieve optional alignment between the two sets of points. Such algorithms are configured to solve this point-to-point matching problem.

[0057] One method that can be used to determine the positional and directional offset between common feature points in two images (or more generally, between two sets of points) is to use the so-called Iterative Closest Point (ICP) algorithm, sometimes referred to as the "Iterative Corresponding Point" algorithm. In this case, the offset is determined based on the corresponding x and y pixel positions of the common feature points in the two images.

[0058] The basic form of the ICP algorithm is described in the technical paper "A Method for Registration of 3-D Shapes" by Besl and McKay in 1992 (hereinafter referred to as the "1992 paper"). Multiple SLAM methods use variations of the ICP algorithm to align a set of points (this type of alignment is also called "registration"). Several speed improvements exist for concepts that allow SLAM to run faster than the basic form of the ICP method, but the core idea remains the same. The 1992 paper describes solutions for points (x, y, z) in 3D space and points (x, y) in 2D space, in addition to other types of geometric data. The system disclosed in this paper uses the form of the ICP algorithm involving a set of points. This method determines how much a first set of data must be translated or rotated from its starting position and orientation toward another position and orientation to minimize the total distance between the first set of points and the second set of points.

[0059] The basic form of the ICP algorithm is as follows: (1) For each point in a given set of data points, use a distance metric to calculate the nearest point in another set; (2) Estimate the amount of translation and rotation required to align the points in the set; (3) Transform the points in a set by the amount determined in the translation and rotation estimates; and (4) Iterate (i.e., return to step (1) to calculate the nearest point again); and (5) Stop iterating when a predetermined distance metric value is reached (e.g., the distance metric value is equal to a specified threshold).

[0060] First, the distance to each point in each group is determined using a distance measurement or "distance metric" (here, the mean square distance metric is used); then, one of the groups is moved (offset) to reduce the mean square distance. The ICP method requires initial estimates of position and orientation offsets. In this application, a coarse approximation of the offset is made using the expected direction of travel of the motion platform and an estimate of the current velocity (this approximation does not need to be very precise). The distance measurement is then recalculated, followed by iterative optimization methods (such as gradient descent) to calculate new estimates of position and orientation offsets. This iteration continues until a convergence criterion is met. Ideally, if every point in one group has exactly one corresponding point in another group and all points are accurately acquired, the total offset determined by the mean square distance metric will be zero. However, due to the possibility of outliers in one group that are misaligned with points in another group (and small acquisition accuracy errors), the optimal mean square distance will not be zero. In the practical case where there are some commonalities and some non-commonalities between the two groups, the method will not reach a zero mean square distance. Therefore, the entire method needs to determine when to stop the iterative search, which is typically when the convergence rate slows down to a certain amount.

[0061] Common feature points are points with the largest nearest-neighbor distance values ​​compared to points with the largest nearest-neighbor distance values. The basic ICP algorithm finds common feature points along the way, but the ICP algorithm does not need to know which points these are before processing the remainder. For working methods, a sufficient number of common feature points are still required, but they do not need to be explicitly found in a dedicated step separate from the location and orientation offset determination process. Common feature points are discovered during iteration, and in some variations of the ICP process, non-common feature points (e.g., outliers) are picked out from the early analysis of the process to speed up convergence (reduce the number of iterations required). In other methods or other variations of the ICP method, outliers may be removed first from common feature points to improve performance.

[0062] In summary, ICP (Intermediate Point Calculation) technology uses common feature points between two sets of points to determine the position and orientation offset of one set of points relative to another. Depending on the specific algorithm used, the step of finding common feature points from two sets of points may or may not be separated from using points to determine position and orientation offset. For example, some versions of the ICP algorithm determine common feature points simultaneously with determining position and orientation offset.

[0063] Provided that the 2D sensor array of the 2D NDI scanner 14 is large enough to always cover at least two structural features of the target object (e.g., structural features such as fasteners), the image processing and feature point comparison module 24 of the system 10 (see...) Figure 1The system can track the position of a sensor array by processing sequentially scanned images, which include common features representing structural characteristics. The 2-D sensor array is configured to capture all elements in a viewing area at once, and as long as some common features from a portion of the first scanned image are visible in the second scanned image, it can be determined how much the 2-D sensor array has moved during the interval between sequential image acquisitions. Knowing the physical dimensions of the array, more specifically, the number of elements per inch, the image processing and feature point comparison module 24 is configured to count the differences between the corresponding positions of common feature points in the two scanned images and convert these differences into relative position estimates. The image processing and feature point comparison module 24 outputs relative position data 26 that can be used for purposes such as position tracking. For example, the pixel differences between the individual representations of the common feature points are counted and converted into the physical distance traveled by the 2-D sensor array. This estimated distance traveled is added to a previous absolute position estimate (measured in the reference frame of the target object) to obtain a new estimate of the absolute position of the 2-D sensor array. Optionally (e.g.) Figure 1 As shown, the relative position data 26 is sent to the NDI sensor data processor 20, where the relative position data 26 can be used to construct a synthetic scan image (as will be described in more detail below).

[0064] To calculate the absolute displacement in a reference frame on the surface of the target object, the system proposed in this paper sums the relative displacements after processing each set of scanned images; this is called dead reckoning. However, as more and more discrete relative distance values ​​are summed, the absolute position estimate becomes less accurate. Therefore, to address this bias, features appearing in the scanned images are compared with landmarks / features with known coordinates. The absolute position estimate is updated based on these known coordinates, which serves to recalibrate the system. Image-to-image feature tracking occurs at a high update rate, while comparisons with known landmark / feature data occur at a lower update rate.

[0065] Figure 3This is a flowchart illustrating the steps of a method 130 for tracking a motion platform using images acquired by a 2D sensor array of a 2D NDI scanner 14 according to one embodiment. Method 130 is suitable for manual or automatic motion control. First, the motion platform 12 is positioned such that the 2D sensor array of the 2D NDI scanner 14 is located at a known physical position defined in a reference frame of the target object (hereinafter referred to as the "initial position") (step 132). At the initial position, the 2D sensor array is positioned at a first location on the surface of the target object with known first X-coordinates and known first Y-coordinates, such that a first portion of the surface of the target object is relative to a known reference point on the sensor array (e.g., the center of the sensor array). Then, the 2D NDI scanner 14 is activated to acquire a first scan consisting of a first set of sensor data from the first portion of the target object (step 134). The first set of sensor data acquired by the first scan is converted into a first scan image (step 136). For example, the NDI sensor data processor 20 converts the first set of sensor data output by the 2D NDI scanner 14 into first scan image data of the first scan image. Then, the image processing and feature point comparison module 24 uses image processing (e.g., edge detection) to find the centroids of the features (hereinafter referred to as “feature points”) in the first scanned image and saves the pixel positions of those feature points in a non-transitory tangible computer-readable storage medium (e.g., computer memory) (step 138).

[0066] Then, the motion platform 12 is moved such that the 2D sensor array of the 2D NDI scanner 14 is positioned at a new physical location (hereinafter referred to as the "current position") on the surface of the target object, where the current scan area partially overlaps with the scan area at the initial position (step 140). When the 2D sensor array translates without rotating along an axis parallel to the rows or columns of the sensor array, the overlapping area must include at least one common feature of the target object (e.g., a subsurface feature) to achieve continuous position tracking. When the 2D sensor array rotates and translates, the overlapping area must include at least two common features of the target object to achieve continuous position tracking.

[0067] At the current position, the 2-D sensor array aligns a second portion of the target object's surface with a reference point on the sensor array at a second position, which has unknown second X-coordinates and unknown second Y-coordinates on the target object's surface. Then, the 2-D NDI scanner 14 is activated to acquire a second scan consisting of a second set of sensor data from the second portion of the target object (step 142). The second set of sensor data acquired through the second scan is converted into a second scan image (step 144). For example, the NDI sensor data processor 20 converts the second set of sensor data output by the 2-D NDI scanner 14 into second scan image data of the second scan image. Then, the image processing and feature point comparison module 24 uses image processing to find feature points in the second scan image and stores the pixel locations of those common features in a non-transitory tangible computer-readable storage medium (step 146).

[0068] The image processing and feature point comparison module 24 is also configured to compare the pixel positions of feature points appearing in the previous and current scanned images (the first and second scanned images in this iteration) to determine common feature points appearing in the two images and calculate a 2-D pixel position difference (step 148). The image processing and feature point comparison module 24 then uses the 2-D pixel position difference of the common feature points and a calibration scaling factor to calculate the relative physical position change (position and orientation) between the current and previous positions of the 2-D NDI scanner 14 (step 150). The process described herein assumes that a calibration scaling factor is available for the 2-D NDI scanner 14, which correlates the individual scanning elements of the device (which are associated with pixels in the NDI scanned images) with physical distances. This type of data is available from the device manufacturer's specification sheet.

[0069] The image processing and feature point comparison module 24 (or a different computer or processor) can then use the previous absolute position and the current relative position to change the current absolute position of the 2-D NDI scanner 14 (step 152), which is defined in the coordinate system (reference system) of the target object (e.g., the X, Y coordinate system on the surface of the target object, as described above).

[0070] Then it is determined whether the positioning process should continue (step 158). On the one hand, if it is determined in step 158 that the positioning process should continue, method 130 returns to step 140. Then the next scan image is acquired, which will have some of the same features as the second scan image, and so on. On the other hand, if it is determined in step 158 that the positioning process should not continue, method 130 terminates.

[0071] Figure 4This is an illustration showing the position of common feature point 48a in corresponding scanned images 42a and 42b, which are continuously acquired at corresponding vertical alignment positions (without rotation) using a 2-D sensor array with sensors arranged in rows and columns and a reference point RP at the center of the 2-D sensor array. (The reference point can be at any known location on the 2-D sensor array.) This example assumes that the motion platform 12 is controlled or restricted to move the 2-D NDI scanner 14 parallel to the Y-axis on the surface of the target object, where the sensor rows are perpendicular to the Y-axis (without rotation). In this case, the reference point RP moves from a first position in a reference system on the surface of the target object with first X position coordinates and first Y position coordinates to a second position with first (same) X position coordinates and second Y position coordinates at a distance from the first Y position coordinates. The second Y position coordinates are calculated by adding the changes in pixel positions (e.g., pixel rows) in scanned image 42b relative to pixel positions (e.g., pixel rows) in scanned image 42a. This change in pixel position of pixel rows is expressed as... Figure 4 The value is denoted by ΔR. In this example, pixels in scanned image 42a that include common feature point 48a are located in rows 5 and columns 11, while pixels in scanned image 42b that include common feature point 48a are located in rows 12 and columns 11, meaning ΔR = 7 rows. Therefore, scanned images 42a and 42b indicate that the 2-D sensor array has moved a distance in the Y direction equal to ΔR multiplied by the calibration scaling factor f. cal The product of these factors, the calibration scaling factor, is equal to the distance between the center lines of adjacent sensor rows in a 2-D sensor array.

[0072] More specifically, the image processing and feature point comparison module 24 is configured to: (a) find the X and Y pixel position values ​​of feature points in the first scanned image and the second scanned image; (b) determine which feature points found in step (a) are common feature points appearing in the first scanned image and the second scanned image; (c) calculate the pixel position difference between the common feature points appearing in the first scanned image and the second scanned image; and (d) calculate the pixel position difference by multiplying the product ΔR×f cal Add the first Y-position coordinate to calculate the second Y-position coordinate of the reference point RP at the second position on the surface of the target object.

[0073] According to one proposed implementation, step (c) includes: calculating a first pixel offset equal to the number of pixel rows of the sensor array whose position of the common feature point 48a in the second scan image is offset from the position of the common feature point 48a in the first scan image; multiplying the first pixel offset by the distance between the corresponding center lines of adjacent sensor rows to form a product equal to a first distance translated by the scanner in a direction perpendicular to the sensor rows; and adding the first distance to a first Y position coordinate to obtain a second Y position coordinate of the reference point RP.

[0074] The same principle can be applied when the 2-D sensor array is translated without rotating in both the X and Y directions. More specifically, the image processing and feature point comparison module 24 is configured to: (a) find the X and Y pixel position values ​​of feature points in the first scan image and the second scan image; (b) determine which feature points found in step (a) are common feature points in the first scan image and the second scan image; (c) calculate the pixel position difference between the common feature points appearing in the first scan image and the second scan image; (d) calculate the difference by multiplying the product ΔR×f cal Add to the first Y position coordinate, calculate the second Y position coordinate of the reference point RP at the second position on the target object surface; (e) by multiplying the product ΔC×f cal Adding the first X-position coordinate to the second X-position coordinate of the reference point RP on the second position of the target object surface, we calculate the second X-position coordinate, where ΔC is the second pixel offset. The second pixel offset is equal to the number of pixel columns of the sensor array whose position in the second scanned image is offset relative to the position of the common feature point in the first scanned image. (For illustrative purposes, it is assumed that the calibration scaling factor is the same for both rows and columns of the 2-D sensor array.)

[0075] Figures 5A to 5J This is an illustration showing a corresponding three-dimensional view of the 2-D NDI scanner 14 at a corresponding location during the scanning of a region on the surface of the target object 3. According to... Figures 5A-5J The proposed application described herein refers to a target object 3 that is a reinforced fuselage portion comprising a fuselage skin 9 and a plurality of reinforcing ribs 5 attached to the fuselage skin 9 by rows of fasteners 1 spaced apart by corresponding intervals, the fasteners having corresponding structural features suitable for producing detectable edges in scanned images.

[0076] Figures 6A to 6J It means that they correspond to respectively Figures 5A to 5J The illustration shows various stages in the construction of the composite scan image 30 of the scanned position of the 2-D NDI scanner 14 shown. The composite scan image includes features 44 with edges representing structural features of the fastener 1.

[0077] Figures 7A to 7JThis is an illustration showing various CAD model representations 32 composed of 3D CAD model data flattened into two dimensions. CAD model representation 32 includes structural information from regions of the stiffened fuselage section, which have corresponding structural information. Figures 5A to 5J The dashed rectangles 28 are superimposed at the scanning positions of the 2-D NDI scanner 14 shown in the figure. The CAD model representation 32 includes pixels 34 representing the state of the body skin 9, pixels 36 representing the state of the reinforcing rib 5, and pixels 38 representing the state of the fastener 1. All representations of the structural components are positioned according to the CAD model specifications.

[0078] Figure 5A The 2-D NDI scanner 14 is shown in its initial position, wherein the position coordinates of the reference point (not shown) on the 2-D NDI scanner 14 (in the reference frame of the target object 3) are known. Figure 6A The initial scanned image 30 is shown, which was initially created solely by the 2-D NDI scanner 14 located in Figure 5A The position shown is a single scan image acquired at the location indicated. Two features 44 appear in the initial scan image 30, and these features correspond to the position shown in the image. Figure 5A The fasteners 1 are shown in the scan (coverage) area of ​​the 2-D NDI scanner 14 at the location shown. Figure 7A The dashed rectangle 28 is superimposed on the CAD model representation 32 to indicate the absolute position of the 2-D NDI scanner 14 in the reference frame of the target object 3, which is determined by... Figure 7A The X and Y axes intersect at the origin O.

[0079] For in Figures 5B to 5E The 2-D NDI scanner 14 is partially depicted at various positions along an exemplary scanning path, where it is moved vertically (parallel to the Y-axis) upwards multiple times by a predetermined distance, stopping after each movement to acquire a new scanned image. This predetermined distance is less than the height of the 2-D sensor array of the 2-D NDI scanner 14, thereby ensuring that the successive scanned images will overlap with common areas on the surface of the target object 3. The predetermined distance the 2-D NDI scanner 14 advances is selected such that the successive scanned images will include common features corresponding to the fastener 1. Figures 5A-5J In the example described, the height of the 2-D sensor array of the 2-DNDI scanner 14 is greater than the distance between adjacent rows of the fastener 1. Using the positioning process described above, the absolute positions of the 2-D NDI scanner 14 in the reference frame of the target object 3 (corresponding to...) can be estimated. Figures 5B to 5E (as shown in the image), and then optionally displayed on the CAD model representation 32, such as Figures 7B to 7E As shown. Additionally, when the 2-DNDI scanner 14 is as follows... Figures 5B to 5EThe individual scan images acquired during the positioning process are stitched together sequentially to progressively construct the image. Figures 6B to 6E The synthetic scan image 30 seen in the image.

[0080] like Figure 5F As shown, after completing the vertical translation sequence, the next movement is a horizontal translation, the set distance of which is less than the width of the 2-D sensor array. During this stage, the absolute position of the 2-D NDI scanner 14 in the reference frame of the target object 3 is determined by… Figure 7F The dashed rectangle 28 in the figure represents this. Figure 6F It shows in Figure 5F The scanned images acquired in the stage shown have been stitched together to form the composite scanned image 30.

[0081] For in Figure 5G to Figure 5J The 2-D NDI scanner 14 is repeatedly moved vertically downwards by a predetermined distance at various positions along the exemplary scanning path, stopping after each movement to acquire a new scanned image. Again, the successive scanned images overlap with common areas on the surface of the target object 3. Using the above positioning process, the absolute positions of the 2-D NDI scanner 14 in the reference frame of the target object 3 (corresponding to...) can be estimated. Figures 5G to 5J The location depicted in the image is then displayed on the CAD model representation 32, such as... Figures 7G to 7J As shown. Additionally, when the 2-D NDI scanner 14 is as... Figure 5G to Figure 5J The individual scan images acquired during localization are stitched together sequentially to progressively construct the image. Figures 6G to 6J The synthetic scan image 30 seen in the image.

[0082] An automated motion platform can be programmed to travel along a predetermined path on the surface of a structure. The position of this automated motion platform can be associated with the coordinate system of the target object (e.g., an aircraft) and can therefore be programmed to follow the internal structure for inspection purposes. After NDI scan data has been collected, features from the aircraft's CAD model are correlated with the NDI scan data for the most accurate depiction of the substructure. Since the location information of the inspection area (defined in the aircraft's coordinate system) is known, this location information can be used to locate and orient the 2-D image in the same visualization environment as the CAD model showing the aircraft features, which is defined in the same reference coordinate system. This can be achieved as an overlay or side-by-side display. In an overlay display scenario, the 2-D NDI scan image is represented as a semi-transparent texture map, which allows the CAD model features to be seen through the 2-D scan image. Based on the observation and analysis of the overlay display, the inspector determines whether repair is required. The process of correlating features from an aircraft's CAD model with NDI scan data is described in U.S. Patent No. 9,182,487. After features from the aircraft's CAD model have been correlated with NDI scan data to produce an accurate depiction of features hidden beneath the aircraft's skin, the locations of features to be marked for repair can be selected manually or automatically using 2D scans and CAD images.

[0083] Figures 8A to 8D This is an illustration showing the paths used to scan the surface area of ​​the exemplary target object 3'. Dashed arrows indicate the scan path segments along which a reference point (or another point with a known position relative to the reference point) of the NDI scanner's 2-D sensor array is located. Adjacent scan path segments overlap, and the resulting scan image data are stitched together to form a composite scan image.

[0084] Figure 8A An edge-to-edge scanning method with an edge start is illustrated, wherein an NDI scanner follows a scan path 7a indicated by a dashed arrow. According to one proposed embodiment, the edge-start edge-to-edge scanning method includes the following steps: (a) The NDI scanner begins scanning at the start of an edge (similar to the corner of two adjacent edges) and then moves along that edge until it reaches the traversing, far edge. (An edge may not be linear.) (b) The NDI scanner then moves away from a portion of the sensor array width from the previous scan line. (c) The NDI scanner then retraces in the opposite direction using a selected overlap as a positioning guide until it reaches the traversing edge. (d) Steps (b) and (c) are repeated until the entire area is covered (i.e., the far edge opposite the original (near) edge is reached). (e) An optional path may be followed around the entire edge to ensure complete coverage of that edge.

[0085] Figure 8B An edge-start center-to-edge scanning method is illustrated, wherein an NDI scanner follows a scanning path 7b indicated by a dashed arrow. According to one proposed embodiment, the edge-start center-to-edge scanning method includes the following steps: (a) Starting from an edge, moving the NDI scanner away from that edge across the entire surface, and the NDI scanner traveling along a generally linear path until reaching the traversed far edge. (b) Then, using a selected overlap as a guide for positioning the NDI scanner, the NDI scanner retraces in the opposite direction until reaching the traversed edge. (c) Repeating step (b) until the entire area on one side of the original scan line is covered. (d) Moving the NDI scanner to the other side of the original scan line, selecting a starting point at either edge. (e) Repeating step (b) on the other side of the original scan line until the entire area is covered. (f) An optional path may be followed around the entire edge to ensure complete coverage of that edge.

[0086] Figure 8C An outside-to-in scanning method is illustrated, wherein an NDI scanner follows a scanning path 7c indicated by a dashed arrow. According to one proposed embodiment, the outside-to-in scanning method includes the following steps: (a) starting from an edge, moving the NDI scanner along that edge in any direction until a complete circle has been traveled. (b) removing the NDI scanner from the edge scan band by a pre-selected proportion of the array width. (c) maintaining the scan band overlap while scanning a circle inside the edge scan band. (d) repeating steps (b) and (c) until the entire area of ​​the surface has been scanned.

[0087] Figure 8D An inside-out scanning method is illustrated, wherein an NDI scanner follows a scanning path 7d indicated by a dashed arrow. According to one proposed embodiment, the inside-out scanning method includes the following steps: (a) First, the NDI scanner is placed at a point in the middle of the surface to be scanned. (b) The NDI scanner is moved along a closed path (returning to the scanning start point) that is small enough to leave no gaps at the center (where the longest axis of the area will be shorter than the array width). This closed path can be circular, elliptical, square, or some other shape, including shapes near the outer edge of the area. (c) The NDI scanner is moved away from the scanning band by a preselected proportion of the array width. (d) While maintaining the overlap of the scanning bands, a full circle is scanned around the outside of the previous scanning band until the NDI scanner returns to the beginning of the path. (e) If an edge is reached, the edge is scanned in the same direction along the closed path until the NDI scanner is able to continue at a point where a path guided by the selected overlap is found. (f) Steps (c) to (e) are repeated until a complete path around the entire edge is a default scan.

[0088] The positioning method proposed in this paper is also effective when the NDI scanner is translated and rotated during the interval between consecutive scans. Figure 9 This is a diagram illustrating the positions of two common feature points 48a and 48b in consecutively acquired scan images 42a and 42b when the NDI scanner is in various different orientation positions. As used herein, the phrase "different orientations" means that the 2D sensor array of the 2D NDI scanner 14 has different orientations at various positions. For illustrative purposes, it is assumed that the NDI has a first orientation angle when capturing the first scan image 42a and a second orientation angle when capturing the second scan image 42b, the second orientation angle differing from the first orientation angle by an angle θ. In this example, the NDI scanner includes multiple sensors arranged in rows and columns and has a reference point RP at a known position relative to the 2D sensor array. This example assumes that the motion platform 12 is controlled such that the 2D sensor array rotates relative to the angular position of the captured first scan image 42a. The rotation angle θ can be calculated using SLAM or other positioning techniques.

[0089] According to one embodiment, a method for tracking the position of a scanner includes the following steps: (a) The 2D sensor array is positioned adjacent to a surface and covers a first feature and a second feature of a target object, and is oriented at a known first orientation angle. (b) When oriented at the first orientation angle, the scanner acquires a first set of sensor data from a first portion facing the target object. (c) The first set of sensor data is converted into first scan image data of a first scan image. (d) The scanner is then reoriented such that the sensor array has a second orientation angle different from the first orientation angle, but again covers the first and second structural features. (e) When oriented at the second orientation angle, the scanner acquires a second set of sensor data from a second portion facing the target object. (f) The second set of sensor data is converted into second scan image data of a second scan image. (g) An image processing and feature point comparison module 24 finds the X and Y pixel position values ​​of feature points in the first and second scan images. (h) The image processing and feature point comparison module 24 determines which feature points found in step (g) are common feature points in the first and second scan images. (i) The image processing and feature point comparison module 24 calculates the pixel position difference between common feature points appearing in the first and second scanned images. (j) Then, the image processing and feature point comparison module 24 calculates a second orientation angle based at least in part on the difference between a first orientation angle and a first angle of a first line connecting the first and second common feature points at their respective positions in the first scanned image, and a second angle of a second line connecting the first and second common feature points at their respective positions in the second scanned image. Figure 9 In the example shown, the difference between the first angle and the second angle is angle θ.

[0090] According to one proposed implementation, step (i) includes (1) calculating a first pixel offset equal to the number of pixel rows in which the position of the second common feature point in the first scanned image is offset relative to the position of the first common feature point in the first scanned image; (2) calculating a second pixel offset equal to the number of pixel columns in which the position of the second common feature point in the first scanned image is offset relative to the position of the first common feature point in the first scanned image; (3) calculating a third pixel offset equal to the number of pixel rows in which the position of the second common feature point in the second scanned image is offset relative to the position of the first common feature point in the second scanned image; and (4) calculating a fourth pixel offset equal to the number of pixel columns in which the position of the second common feature point in the second scanned image is offset relative to the position of the first common feature point in the second scanned image, wherein a first angle is equal to the ratio of the first pixel offset to the second pixel offset, and the tangent of the second angle is equal to the ratio of the third pixel offset to the fourth pixel offset. To calculate the absolute position of the NDI scanner in a reference frame on the target object's surface, the system combines relative position and orientation data with previous (old) absolute position and orientation data to calculate the current (new) absolute position and orientation of the NDI scanner relative to the target object's coordinate system. This positioning method is a dead reckoning process, meaning that the absolute position estimate becomes less accurate as more discrete relative distance values ​​are summed. An optional correction process can be used in conjunction with the basic relative positioning process to improve the position estimate based on knowledge of identified common features within the scanned image that have known position coordinates. This correction process operates at a lower update rate than the master feature tracking process and can be used to improve the position estimate to compensate for feature synthesis measurement errors.

[0091] According to some embodiments, the positioning method includes periodic correction steps to reposition the 2-D sensor array, such as occasional manual checks and manual adjustments to position and orientation. For example, the array scans the length of the part, moves over a width smaller than the array's width, and then... Figures 5A-5J The scan returns to the original edge as shown. The distance from the starting position can be checked and adjusted after each scan, or after several scans, as can the array angle.

[0092] In the case of a robotic tracked vehicle, wheel slippage along the path becomes apparent as the NDI scanner travels through the structural features of the target object. Position and orientation between scans can be checked by measuring the distance to the starting point and making simple laser line alignment adjustments to the starting position. If a random or non-repeating pattern exists, the position and orientation of the array can be checked with each overlapping pass, and the array's position and orientation can be manually corrected using known reference features scanned on the target object whenever significant deviations are found.

[0093] As previously referenced Figure 1 As described, the synthetic scan image 30 (in) Figures 6A-6J The various stages (shown in the diagram) are constructed by the NDI sensor data processor 20 based on relative position data 26 generated by the image processing and feature point comparison module 24. Alignment of the stitched images uses relative pixel offsets and does not involve absolute position coordinates; however, for physical tracking, absolute distances can be calculated. Knowing only the number of pixels is insufficient for the system to accurately determine the length of structural features. The proposed localization process is approximate and will gradually “drift” over time until the position estimate can be updated using known absolute coordinate information, at which point the error returns to zero and begins to build up again as more image alignments are processed.

[0094] For example, a composite scan image with 20 images stitched together can be generated without significant misalignment, but the total length of the images in pixels may not precisely correspond to the physical distance traveled in inches or centimeters. This is because the composite image may be slightly distorted due to small alignment errors from one image to the next. Pixels are a digital representation of the analog world, but rounding, truncation, or other measurement artifacts spread from analog-to-digital conversion into the composite. Even if the distances between adjacent columns of elements that make up a 2-D sensor array are known, small errors will accumulate in the total distance summation as more and more images are aligned, unless or until the estimated absolute position is corrected using a known absolute reference. The total image-to-image error may be small, and it may be possible to capture multiple images between known landmark updates, but at some point, the measured distance error may become too large, necessitating corrections as disclosed herein.

[0095] On the other hand, if a precise synthetic scan image is desired, the synthetic scan image can be scaled to an appropriate size if the actual total distance is known from other landmark data. It is also understood that it is not always necessary to store synthetic scan images in memory. This would only be the case if the synthetic scan image is useful for certain types of analysis processes, such as NDI analysis. In other cases, scan image data may be stored in memory solely for location tracking purposes, and the localization process will remove them from memory when the individual images are no longer needed for feature alignment.

[0096] The positioning process based on 2-D NDI sensors proposed in this paper is applicable to automated motion control systems using feedback control (such as robots and surface-crawling vehicles), as well as to manual motion control systems (such as handheld devices).

[0097] For manual movement of a 2-D NDI scanner, there are several options: (1) In order to slide the 2-D NDI scanner on a surface, the support housing for the 2-D NDI scanner can be made of some type of low-friction plastic (such as Delrin) to reduce sliding resistance and potential scratches on the surface; or (2) the support frame for the 2-D NDI scanner can be provided with three or more omnidirectional wheels (hereinafter referred to as "omnidirectional wheels").

[0098] For automated movement, the NDI scanner can be mounted on the frame of a tracked vehicle (e.g., a fully-motorized or non-fully-motorized tracked vehicle). For example, U.S. Patent No. 8,738,226 discloses a tethered tracked vehicle capable of scanning a two-dimensional sensor array on a fuselage surface. In an alternative embodiment, the two-dimensional sensor array can be mounted on a vacuum-attached tracked vehicle of the type disclosed in U.S. Patent Nos. 8,738,226 and 10,168,287. Alternatively, automated movement can be achieved by mounting the NDI scanner to a frame assembly that is coupled to an end effector at the distal end of a manipulator arm (articulated, telescopic, frame, etc.). (As used herein, the term "end effector" refers to the last link of an automated device including an arm at which the frame assembly supporting the NDI scanner is connected.) Suitable robots including articulated arms are disclosed in U.S. Patent No. 9,933,396. According to another automated system, the NDI scanner can be carried by an unmanned aerial vehicle (UAV) that flies to the target area and then drags a sensor array across the surface of the target area. For example, U.S. Patent Application No. 16 / 202,347 discloses a UAV that carries a one-dimensional sensor array, but it can be adapted to carry a two-dimensional sensor array.

[0099] Figure 10This illustration depicts some components of a system 10' for tracking and navigating an automated motion platform 12 according to one embodiment. The automated motion platform 12 is equipped with a 2-D NDI scanner 14, a motion controller 16, and drive motors 18 (for driving wheels, not shown). The motion controller 16 may include an onboard motion control processor or module and multiple motor controllers that receive commands from the motion control processor or module to control the operation of multiple drive motors 18 that drive the wheels (e.g., omnidirectional wheels or Mecanum wheels) to rotate. During scanning operations, as a result of the continuous movement of the motion platform 12, the 2-D NDI scanner 14 is positioned at successive locations on the surface of a target object. At each scanning location, the 2-D NDI scanner 14 is activated to acquire sensor data from the corresponding facing portion of the target object's surface.

[0100] exist Figure 10 The system 10', partially depicted, also includes an NDI sensor data processor 20 communicatively coupled (via cable or wireless) to the 2-D NDI scanner 14. The NDI sensor data processor 20 is configured to convert sensor data output from the 2-D NDI scanner 14 into 2-D scanned image data 22. Additionally, the system 102-D includes an image processing and feature point comparison module 24 communicatively coupled to receive the 2-D scanned image data 22 from the NDI sensor data processor 20. The image processing and feature point comparison module 24 may be a processor or computer configured (e.g., programmed) to track the position (also referred to herein as "localization") of the 2-D NDI scanner 14 relative to a target object surface using the 2-D scanned image data 22. This localization algorithm includes relative motion update processing based on successive scan features from one captured NDI scanned image to the next captured NDI scanned image to determine the motion of the 2-D sensor array relative to its position during previous image acquisition.

[0101] The image processing and feature point comparison module 24 outputs relative position data 26 to a motion controller, which can be used to control the movement of the motion platform 12. For example, it counts the pixel differences between representations of common feature points and converts them into the physical distance traveled by the 2D sensor array. Optionally (e.g.) Figure 10 As shown, relative position data 26 is also sent to NDI sensor data processor 20, where relative position data 26 can be used to construct a synthetic scan image.

[0102] Figure 11This is a flowchart illustrating the steps of method 160, which is used to construct a synthetic scan image using an automated motion platform 12 equipped with a 2-D NDI scanner 14 and a computer system. The 2-D NDI scanner 14 includes a two-dimensional sensor array, and the computer system is configured to track the position of the motion platform during scanning based on non-visual (e.g., NDI) data acquired by the sensor array, and to construct the synthetic scan image by stitching together the acquired scan images. Method 160 is suitable for manual or automatic motion control. First, the motion platform 12 is positioned such that the 2-D sensor array of the 2-D NDI scanner 14 is located at a known physical position defined in the reference frame of the target object (hereinafter referred to as the "initial position") (step 132). Thereafter, as previously referenced... Figure 3 Steps 134, 136, 138, 140, 142, 144, and 146 are performed as described. The image processing and feature point comparison module 24 then compares the corresponding pixel positions of common feature points appearing in the previous and current scan images (the first and second scan images in this iteration) to determine the 2-D pixel position difference (step 148). The image processing and feature point comparison module 24 then uses the 2-D pixel position difference of the common feature points and a calibration scaling factor to calculate the relative physical position change (position and orientation) between the current and previous positions of the 2-D NDI scanner 14 (step 150). The image processing and feature point comparison module 24 (or a different computer or processor) can then use the previous absolute position and the current relative position change to calculate the current absolute position of the 2-D NDI scanner 14, which is defined in the target object's coordinate system (reference system) (e.g., the XY coordinate system associated with the surface of the target object as described above) (step 152). The updated absolute position data is then sent to the system motion controller (step 154). Furthermore, the synthesized scan image is updated using the current NDI scan image and the relative 2-D pixel position difference data calculated in step 156. Then, it is determined whether the localization process should continue (step 158). If it is determined in step 158 that the localization process should continue, method 130 returns to step 140. Then, the next scan image is acquired; this third scan image will have some common features similar to the second scan image, and so on. Otherwise, if it is determined in step 158 that the localization process should not continue, method 160 terminates.

[0103] Figure 11 The algorithm described in [the document] extends [the algorithm]. Figure 3The general localization process described herein addresses a specific type of NDI use case: generating a composite image from a series of individual images using relative position (position and orientation) offsets in pixel space. Furthermore, the position data is used by a motion controller during this process to enable the system to move automatically (e.g., by a robot). However, it should be understood that the aspects of composite image creation and automatic motion control can be used independently and do not necessarily need to be combined.

[0104] Figure 12This is a flowchart illustrating the steps of a method 100 for constructing (creating) a synthetic scan image of a target object using a movable two-dimensional sensor array according to one embodiment. Each scan image is acquired in the manner described above, i.e., by acquiring sensor data, generating a scan image from the sensor data, finding feature points, comparing the positions of the feature points in successive scan images to determine which feature points are common between the two images, and then determining the offset. Initially, the 2-D sensor array is placed at a first position with known position coordinates and a known orientation angle (step 102). Then, the NDI scanner is activated to acquire an initial scan image (hereinafter referred to as the "first scan image") from a first portion of the target object facing the sensor array (step 104). After the initial scan image is acquired, a system operator (e.g., a technician or inspector) verifies that the first scan image includes at least two distinct features. If at least two features are distinct, the first scan image is loaded into a synthetic scan image file for storage (step 106). The 2-D sensor array is then moved to the next position (step 108). The next position is selected such that the 2-D sensor array covers at least two distinct features seen in the previous (in this iteration, the first) scan image. At the next position, the NDI scanner is activated to acquire sensor data, and then the NDI sensor data processor generates the next (in this iteration, the second) scan image (step 110). The next scan image is then stitched to the composite scan image (step 112). In the iteration of stitching the second scan image to the first scan image, the composite scan image is constructed by aligning the second scan image data in the second scan image with the first scan image data in the first scan image using common features from both the first and second scan images. More generally, in subsequent iterations, the next scan image is aligned with the composite scan image using common features. According to one option, the composite scan image includes pixel data from the first scan image and pixel data from the portion of the second scan image that does not overlap with the first scan image. According to another option, the composite scan image includes: pixel data from a first portion of the first scan image that does not overlap with the second scan image; pixel data from the first portion of the second scan image that does not overlap with the first scan image; and pixel data that is a mixture of pixel data from a second portion of the second scan image that overlaps with the first scan image and pixel data from the second portion of the first scan image that overlaps with the second scan image.

[0105] After each successively acquired scan image is merged with the composite scan image, the technician performs a manual inspection to determine whether the entire surface of the component being inspected has been covered (step 114). If it is determined in step 114 that the entire surface of the component has not been covered, the technician uses an interface to notify the controller (e.g., Figure 15The computer system 40 in the middle requires more scanning (step 116). Then, Figure 12 The algorithm shown returns to step 108, where the 2-D sensor array is moved to the next position facing the uninspected portion of the part. If it is determined in step 114 that the entire surface of the part has been covered, the technician uses the interface to notify the controller to terminate the scan (step 118). The composite scan image file is then saved in the NDI results directory on a non-transitory tangible computer-readable storage medium for subsequent analysis (step 120).

[0106] As previously disclosed, the motion platform can be designed for manual movement. Figure 13 A top plan view of some components of a complete handheld motion tracking device 12a configured with four omnidirectional wheels is shown. This tracking device can be connected to an NDI scanner (…). Figure 13 Not shown in the image, but see [link / reference]. Figure 14A 2-D NDI scanner 14). Figure 13 The complete motion handheld tracking device 12a shown includes a rectangular frame 4 and four double-row omnidirectional wheels 4a-4d rotatably mounted to the frame 4 via respective shafts 6a-6d and bearings (not shown). In this particular embodiment, shafts 6a and 6c are coaxial; shafts 6b and 6d are coaxial; and shafts 6a and 6c are perpendicular to shafts 6b and 6d. This embodiment illustrates an optional double-row omnidirectional wheel (which can improve rolling smoothness under certain conditions), but a single-row omnidirectional wheel can also be used. In an alternative embodiment, the tracking device does not have wheels but slides on a surface.

[0107] Figure 14A and Figure 14B The top plan view and front view of the handheld scanning device are shown respectively. The handheld scanning device includes a rigid connection to Figure 13 The graphic depicts a 2-D NDI scanner 14 of a complete motion handheld tracking device 12a. The 2-D NDI scanner 14 is carried by the complete motion handheld tracking device 12a. The 2-D NDI scanner 14 has a handle 8 attached thereto, through which the operator can manually rotate or not rotate the scanning device in any direction. One or more omnidirectional wheels can be mounted in a manner that allows for conformability between the support frame and the wheel assembly (e.g., using linear guides and springs). This allows the wheels to maintain contact with the surface even when the surface is not perfectly flat.

[0108] One possible use case for a manual movement system that does not use NDI sensor data to create synthetic scan images is for general positioning, where the current position and orientation of the NDI scanner are displayed to the user. In some use cases, the user may be viewing a display of a single NDI scan image and wants to extract the location coordinates of a single point of interest. Another use case, which does not involve inspection, is using the manual movement device as a digital measuring tape (with location data displayed on the screen).

[0109] Figure 15 This is a block diagram illustrating some components of a system 50 according to one embodiment, which includes a 2-D sensor array 60 (e.g., an ultrasonic transducer array) mounted to a Mecanum wheeled robot tracked vehicle 52 (hereinafter referred to as "tracked vehicle 52"). In alternative embodiments, omnidirectional wheels or other types of wheels may be used. The system 50 further includes a computer system 40 configured to control the scanning position of the 2-D sensor array 60 and the acquisition of sensor data by the 2-D sensor array 60. The scanning position is controlled based on common features in the scanned images derived from the sensor data obtained by the 2-D sensor array 60 from the part being inspected, as previously described herein.

[0110] The tracked vehicle 52 includes a motion controller 16 and a plurality of Mecanum wheels 7 operably coupled to respective drive motors 18. The motion controller 16 includes a motion control processor 54 and a plurality of motor controllers 56 for independently controlling the drive motors 18 based on control signals received from the motion control processor 54. The motion control processor 54 then receives commands from a computer system 40. The computer system 40 can be connected via cable (…). Figure 15 The computer system 40 is wirelessly coupled to the motion control processor 54 (not shown in the diagram) or via a transceiver. The computer system 40 uses relative position information to track the relative position of the tracked vehicle 52 (e.g., relative to an initial absolute position obtained using a local positioning system).

[0111] More specifically, the computer system 40 is programmed using NDI scanning application software 64 and motion control application software 68. The computer system 40 may include a general-purpose computer. The NDI scanning application software 64 is configured to control the pulse generator / receiver 62. Figure 15 In the configuration shown, a pulse generator / receiver 62 is coupled to a 2-D sensor array 60 carried by the tracked vehicle 52. The pulse generator / receiver 62 sends pulses to the 2-D sensor array 60 and receives return signals from it. NDI scanning application software 64 running on the computer system 40 controls all details of the image scanning data and its display.

[0112] Additionally, the computer system 40 includes an image processing and feature point comparison module 24 that outputs relative position data 26. A motion control application 68 is configured to control the movement of the tracked vehicle 52 to continue along the original predefined (planned) scan path based on positioning updates received from the image processing and feature point comparison module 24. According to one embodiment, the motion control application software 68 is configured to estimate the position of the tracked vehicle 52 in absolute coordinates after tracking motion using the relative position data 26 is complete. The current position of the stopped device can be periodically checked to determine the extent to which it may have deviated from the desired position. According to the teachings herein, relative motion measurements can be corrected by obtaining accurate absolute measurements at a lower update rate. This absolute measurement processing (performed when the target object is stationary) can be integrated into a relative motion measurement system operating at a higher update rate, which acquires relative motion measurements as the target object moves. According to one embodiment disclosed below, a process based on a lower update rate local positioning system provides correction for a higher update rate positioning process.

[0113] In addition to mapping subsurface features, absolute position measurements can be used to map the location of surface and subsurface anomalies in a target object. Mapping defect size, shape, and depth to the CAD model of the target object allows for finite element analysis of defects in the structure, analysis of their impact on structural performance, and repair analysis and planning.

[0114] The processor that creates the composite scan image needs access to both the NDI scan image and the relative position data. According to some embodiments, the task of constructing the composite scan image can be handled by the NDI sensor data processor 20. According to other embodiments, the task of creating the composite scan image is handled by a separate process (or processor). In typical applications, the same processor that creates individual images from the raw NDI sensor data handles combining the individual scan images into a larger composite scan image (because the hardware is packaged by the vendor for easy integration into a single unit). However, this image processing need not be performed in this way. Alternatively, the NDI sensor data processor 20 can output the individual scan images and send them to a separate processor to stitch the scan images together to form a single large composite scan image.

[0115] Image processing methods analyze common features between pairs of NDI scan images to find positional differences between identical common features in each scan image. This allows the synthetic scan image stitching method to know how much these images are offset from each other. By understanding this information (common feature positions and orientation differences), the current scan image can be correctly aligned with the synthetic scan image. If the common features in the images are aligned, the images are aligned. This alignment is independent of the generation of position and orientation tracking data, meaning that even if errors in the tracking data increase, the scan images will still be aligned.

[0116] Positioning and tracking correction can be performed manually (human-assisted), where a person identifies common known landmarks in the CAD model data and NDI scan image data and performs estimations to align with known coordinates. Alternatively, the correction process can be automated by using appropriately prepared reference images. These reference images can come from previous NDI scans where the coordinates of the landmarks have been identified, or the data can come from CAD model data with known coordinates for the landmarks.

[0117] According to one proposed embodiment, the 2-D sensor array takes the form of an array of ultrasonic transducer elements configured to generate and display C-scans over small areas. Many different ultrasonic transducer element configurations can be employed. For example, the ultrasonic transducer array may include an array of transmitting / receiving electrodes arranged in rows and columns in a pixel-like configuration. In an optional configuration, the ultrasonic transducer array includes a set of elongated, parallel transmitting electrodes that overlap and intersect with a set of elongated, parallel receiving electrodes at a non-zero angle. In various industries requiring the detection of defects or anomalies in structures, such as in the aircraft, automotive, marine, or construction industries, ultrasonic transducer arrays can be used to inspect any number of structures. This ultrasonic transducer array is capable of detecting any number of defects or anomalies within or on the surface of the structure, such as impact damage (e.g., delamination and matrix cracking), debonding (e.g., fuselage / reinforcing members or cellular composite materials), discontinuities, voids, or porosity, which may adversely affect the performance of the structure.

[0118] Certain systems, devices, applications, or processes have been described herein as comprising multiple modules. A module can be a unit of different functions implemented in software, hardware, or a combination thereof, in addition to those modules preferably implemented as hardware or firmware to achieve streaming computing as disclosed herein. When the functionality of a module is performed by software in any part, the module may include a non-transitory tangible computer-readable storage medium.

[0119] While systems and methods for tracking the position of an NDI scanner using scanned images acquired from a target object have been described with reference to specific embodiments, those skilled in the art will understand that various changes can be made and elements can be substituted with equivalents without departing from the scope of the teachings herein. Furthermore, many modifications can be made to adapt particular situations to the teachings herein without departing from the spirit of the invention. Therefore, it is intended that the claims set forth below be limited to the disclosed embodiments.

[0120] As used herein, the term "computer system" should be interpreted broadly to include a system having at least one computer or processor, and may have multiple computers or processors communicatively coupled by means of a network or bus. As used in the preceding sentence, the terms "computer" and "processor" both refer to means including a processing unit (e.g., a central processing unit) and some form of memory (e.g., a non-transitory tangible computer-readable storage medium) for storing programs that can be read by the processing unit.

[0121] The methods described herein can be encoded as executable instructions included in a non-transitory tangible computer-readable storage medium, including but not limited to storage devices and / or memory devices. When executed by a processor or computer, such instructions cause the processor or computer to perform at least a portion of the methods described herein.

[0122] The method claims set forth herein should not be construed as requiring the steps described herein to be performed in alphabetical order (any alphabetical order in the claims is for the purpose of referencing previously described steps only) or in the order in which they are described, unless the language of the claims expressly specifies or states conditions indicating a particular order in which some or all of these steps are performed. Nor should the method claims be construed as excluding any portion of two or more steps that are performed simultaneously or alternately, unless the language of the claims expressly states conditions excluding such an interpretation. For example, the steps of determining which feature points are common feature points and calculating the pixel position differences between common feature points can be performed individually or simultaneously.

[0123] Furthermore, this disclosure includes embodiments based on the following:

[0124] Item 1. A method for tracking the position of a scanner, the method comprising:

[0125] (a) The sensor array (60) of the scanner (14) is placed near the surface (31) of the target object (30) such that the sensor array covers a first structural feature (11) of the target object, wherein a reference point on the sensor array is at a first position having known first X position coordinates and first Y position coordinates on the surface of the target object, wherein the sensor array is two-dimensional.

[0126] (b) When the reference point is in the first position, acquire the first set of sensor data from the first part of the sensor array facing the target object;

[0127] (c) Convert the first set of sensor data into first scan image data of the first scan image (42a);

[0128] (d) Move the scanner across the surface of the target object until the reference point is at a second position at a certain distance from the first position, while the sensor array covers the first structural feature again;

[0129] (e) When the reference point is in the second position, acquire a second set of sensor data from the second part of the sensor array facing the target object;

[0130] (f) Convert the second set of sensor data into second scan image data of the second scan image (42b);

[0131] (g) Locate the feature points in the first and second scan images (48);

[0132] (h) Determine which feature points found in step (g) are common feature points in the first scan image and the second scan image (48a);

[0133] (i) Calculate the pixel position difference between common feature points; and

[0134] (j) Calculate the second Y position coordinates of a reference point at a second position on the surface of the target object, based at least in part on the first Y position coordinates and the pixel position difference calculated in step (i).

[0135] 2. The method as described in item 1, wherein step (j) comprises:

[0136] Calculate the first pixel offset, which is equal to the number of pixel rows of the sensor array whose position in the second scanned image is offset relative to the position of the common feature point in the first scanned image.

[0137] The first pixel offset is multiplied by the distance between adjacent sensor rows to form a product equal to the first distance the scanner translates in a direction perpendicular to the sensor rows during step (d); and

[0138] Add the first distance to the first Y-position coordinate to obtain the second Y-position coordinate of the reference point.

[0139] 3. The method as described in any one of items 1-2, further comprising:

[0140] (k) Calculate the second X position coordinates of the reference point on the surface of the target object, based at least in part on the first X position coordinates and the pixel position difference calculated in step (i).

[0141] 4. The method of claim 3, further comprising calculating the distance between a first point having a first X-position coordinate and a first Y-position coordinate and a second point having a second X-position coordinate and a second Y-position coordinate.

[0142] 5. The method as described in any one of items 3-4, wherein step (j) comprises:

[0143] Calculate the first pixel offset, which is equal to the number of pixel rows by which the position of the common feature point in the second scanned image is offset relative to the position of the common feature point in the first scanned image;

[0144] The first pixel offset is multiplied by the distance between adjacent sensor rows to form a product equal to the first distance the scanner translates in a direction perpendicular to the sensor rows during step (d); and

[0145] The second Y-position coordinate is obtained by adding the first distance to the first Y-position coordinate, wherein step k includes:

[0146] Calculate the second pixel offset, which is equal to the number of pixel columns by which the position of the common feature point in the second scanned image is offset relative to the position of the common feature point in the first scanned image;

[0147] The second pixel offset is multiplied by the distance between adjacent sensor columns to form a product equal to the second distance the scanner translates in a direction parallel to the sensor rows during step (k); and

[0148] The second X-position coordinate is obtained by adding the second distance to the first X-position coordinate.

[0149] 6. The method of any one of items 3-5 further includes processing the first scan image and the second scan image to form a composite scan image (30) by aligning the second scan image data in the second scan image with the first scan image data in the first scan image using common feature points in the first scan image and the second scan image.

[0150] 7. The method of claim 6, further comprising:

[0151] Find feature points representing the second structural features of the target object in the synthetic scan image;

[0152] The third Y-position coordinates of the feature points are calculated at least in part based on the first Y-position coordinates of the reference point; and

[0153] The third X-position coordinates of the feature point are calculated at least in part based on the first X-position coordinates of the reference point.

[0154] 8. A method for tracking the position of a scanner, the method comprising:

[0155] (a) The sensor array (60) of the scanner (14) is placed adjacent to the surface (31) of the target object (30) such that the sensor array covers the first structural feature and the second structural feature (11) of the target object and is oriented at a known first orientation angle;

[0156] (b) When the sensor array is oriented at a first orientation angle, a first set of sensor data is acquired from the first portion of the target object facing the sensor array;

[0157] (c) Convert the first set of sensor data into first scan image data of the first scan image (42a);

[0158] (d) An orientation scanner, which gives the sensor array a second orientation angle different from the first orientation angle, and again covers the first and second structural features;

[0159] (e) When the sensor array is oriented at the second orientation angle, a second set of sensor data is acquired from the second portion of the target object facing the sensor array;

[0160] (f) Convert the second set of sensor data into second scan image data of the second scan image (42b);

[0161] (g) Locate the feature points in the first and second scan images (48);

[0162] (h) Determine which feature points found in step (g) are common feature points in the first scan image and the second scan image (48a);

[0163] (i) Calculate the pixel position difference between the first common feature point and the second common feature point; and (j) Calculate the second orientation angle based at least in part on the difference between the first orientation angle and the first angle of the first line connecting the first common feature point and the second common feature point at their respective positions in the first scanned image and the second angle of the second line connecting the first common feature point and the second common feature point at their respective positions in the second scanned image.

[0164] 9. The method as described in item 8, wherein step (i) comprises:

[0165] Calculate the first pixel offset, which is equal to the number of pixel rows by which the position of the second common feature point in the first scanned image is offset relative to the position of the first common feature point in the first scanned image;

[0166] Calculate the second pixel offset, which is equal to the number of pixel columns by which the position of the second common feature point in the first scanned image is offset relative to the position of the first common feature point in the first scanned image;

[0167] Calculate the third pixel offset, which is equal to the number of pixel rows by which the position of the second common feature point in the second scanned image is offset relative to the position of the first common feature point in the second scanned image; and

[0168] Calculate the fourth pixel offset, which is equal to the number of pixel columns by which the position of the second common feature point in the second scanned image is offset relative to the position of the first common feature point in the second scanned image;

[0169] Wherein, the tangent of the first angle is equal to the ratio of the first pixel offset to the second pixel offset, and the tangent of the second angle is equal to the ratio of the third pixel offset to the fourth pixel offset.

[0170] 10. The method of any one of items 8-9 further comprises processing the first scan image and the second scan image to form a composite scan image (30) by aligning the second scan image data in the second scan image with the first scan image data in the first scan image using common feature points in the first scan image and the second scan image.

[0171] 11. The method of claim 10, wherein the synthesized scan image includes pixel data from a first scan image and pixel data from a portion of a second scan image that does not overlap with the first scan image.

[0172] 12. The method of any one of items 10-11, wherein the synthesized scanned image comprises:

[0173] Pixel data from the first portion of the first scanned image that does not overlap with the second scanned image;

[0174] Pixel data from the first portion of the second scan image that does not overlap with the first scan image; and

[0175] The pixel data is a mixture of pixel data from the second portion of the second scanned image that overlaps with the first scanned image and pixel data from the second portion of the first scanned image that overlaps with the second scanned image.

[0176] 13. The method of any one of items 10-12, further comprising:

[0177] Find feature points in the synthetic scanned image that are associated with the second structural features in the target object;

[0178] The third Y-position coordinates of the feature points are calculated at least in part based on the first Y-position coordinates of the reference point; and

[0179] The third X-position coordinates of the feature point are calculated at least in part based on the first X-position coordinates of the reference point.

[0180] 14. A system comprising a frame (2), a scanner (14) supported by the frame and including a two-dimensional array of sensors (60), a robot system (50), and a computer system (40), the robot system (50) being configured to enable the frame to move motorized, the computer system (40) being communicatively coupled to receive sensor data from the sensors and to send control signals to the robot system for controlling the motorized movement of the frame, and being configured to perform the following operations:

[0181] (a) When the sensor array covers the first structural feature (11) of the target object (30), the sensor is activated, wherein the reference point on the sensor array is located on the surface of the target object at a first position with known first X position coordinates and first Y position coordinates.

[0182] (b) When the reference point is in the first position, receive the first set of sensor data acquired from the first part of the sensor array facing the target object;

[0183] (c) Convert the first set of sensor data into first scan image data of the first scan image (42a);

[0184] (d) Activate the sensor while the sensor array covers the first structural feature of the target object, wherein the reference point is located at a second position at a certain distance from the first position;

[0185] (e) When the reference point is in the second position, receive a second set of sensor data acquired from the second part of the sensor array facing the target object;

[0186] (f) Convert the second set of sensor data into second scan image data of the second scan image (42b);

[0187] (g) Locate the feature points in the first and second scan images (48);

[0188] (h) Determine which feature points found in step (g) are common feature points in the first scan image and the second scan image (48a);

[0189] (i) Calculate the pixel position difference between common feature points; and

[0190] (j) Calculate the second Y position coordinates of a reference point at a second position on the surface of the target object, based at least in part on the first Y position coordinates and the pixel position difference calculated in step (i).

[0191] 15. The system as described in item 14, wherein operation (j) includes:

[0192] Calculate the first pixel offset, which is equal to the number of pixel rows by which the position of the common feature point in the second scanned image is offset relative to the position of the common feature point in the first scanned image;

[0193] The first pixel offset is multiplied by the distance between adjacent sensor rows to form a product equal to the first distance the scanner translates in a direction perpendicular to the sensor rows during step d; and

[0194] Add the first distance to the first Y-position coordinate to obtain the second Y-position coordinate of the reference point.

[0195] 16. The system as described in any one of items 14-15, wherein the computer system is further configured to:

[0196] (k) Calculate the second X position coordinates of the reference point on the surface of the target object, at least in part, based on the first X position coordinates and the pixel position difference calculated in step (i).

[0197] 17. The system of claim 16, wherein the computer system is further configured to calculate the distance between a first point having first X position coordinates and first Y position coordinates on the surface of the target object and a second point having second X position coordinates and second Y position coordinates on the surface of the target object.

[0198] 18. The system of any one of items 15-17, wherein operation (j) comprises:

[0199] Calculate the first pixel offset, which is equal to the number of pixel rows by which the position of the common feature point in the second scanned image is offset relative to the position of the common feature point in the first scanned image;

[0200] The first pixel offset is multiplied by the distance between adjacent sensor rows to form a product equal to the first distance the scanner translates in a direction perpendicular to the sensor rows during step d; and

[0201] Add the first distance to the first Y-position coordinate to obtain the second Y-position coordinate, where operation k includes:

[0202] Calculate the second pixel offset, which is equal to the number of pixel columns by which the position of the common feature point in the second scanned image is offset relative to the position of the common feature point in the first scanned image;

[0203] The second pixel offset is multiplied by the distance between adjacent sensor columns to form a product, which is equal to the second distance the scanner translates in a direction parallel to the sensor rows during step (k); and

[0204] Add the second distance to the first X-position coordinate to obtain the second X-position coordinate.

[0205] 19. The system of any one of claims 15-18, wherein the computer system is further configured to process the first scan image and the second scan image to form a composite scan image (30) by aligning the second scan image data in the second scan image with the first scan image data in the first scan image using common feature points in the first scan image and the second scan image.

[0206] 20. The system of claim 19, wherein the computer system is further configured to perform a plurality of operations, including:

[0207] Find feature points (48) in the synthetic scan image that are associated with the second structural feature (11) in the target object;

[0208] The third Y-position coordinates of the feature points are calculated at least in part based on the first Y-position coordinates of the reference point; and

[0209] The third X-position coordinates of the feature point are calculated at least in part based on the first X-position coordinates of the reference point.

Claims

1. A method for tracking the position of a scanner, the method comprising: (a) The sensor array (60) of the scanner (14) is placed adjacent to the surface of the target object such that the sensor array covers a first structural feature of the target object at a first position on the surface of the target object with reference points on the sensor array, wherein the first position has known first X position coordinates and first Y position coordinates on the surface of the target object, wherein the sensor array is two-dimensional. (b) When the reference point is at the first position, a first set of sensor data is acquired from the first portion of the sensor array facing the target object; (c) Convert the first set of sensor data into first scan image data of the first scan image (42a); (d) The scanner is translated on the surface of the target object until the reference point is located at a second position at a certain distance from the first position, while the sensor array covers the first structural feature again; (e) When the reference point is in the second position, a second set of sensor data is acquired from the second portion of the sensor array facing the target object; (f) Convert the second set of sensor data into second scan image data of the second scan image (42b); (g) Locate feature points (48) in the first scanned image and the second scanned image; (h) Determine which feature points found in step (g) are common feature points in the first scan image and the second scan image (48a); (i) Calculate the pixel position difference between the common feature points; and (j) Calculate the second Y-position coordinates of the reference point at the second position on the surface of the target object, based at least in part on the first Y-position coordinates and the pixel position difference calculated in step (i). Step (j) includes: Calculate a first pixel offset, which is equal to the number of pixel rows of the sensor array whose position of the common feature point in the second scanned image is offset relative to the position of the common feature point in the first scanned image. The first pixel offset is multiplied by the distance between adjacent sensor rows to form a product, the product being equal to a first distance the scanner has translated in a direction perpendicular to the sensor rows during step (d); and Add the first distance to the first Y position coordinate to obtain the second Y position coordinate of the reference point.

2. The method according to claim 1, further comprising: (k) Calculate the second X position coordinates of the reference point on the surface of the target object, based at least in part on the first X position coordinates and the pixel position difference calculated in step (i).

3. The method according to claim 2, further comprising calculating the distance between a first point having the first X position coordinates and the first Y position coordinates and a second point having the second X position coordinates and the second Y position coordinates.

4. A method for tracking the position of a scanner, the method comprising: (a) The sensor array (60) of the scanner (14) is placed adjacent to the surface of the target object such that the sensor array covers a first structural feature of the target object at a first position on the surface of the target object with reference points on the sensor array, wherein the first position has known first X position coordinates and first Y position coordinates on the surface of the target object, wherein the sensor array is two-dimensional. (b) When the reference point is at the first position, a first set of sensor data is acquired from the first portion of the sensor array facing the target object; (c) Convert the first set of sensor data into first scan image data of the first scan image (42a); (d) The scanner is translated on the surface of the target object until the reference point is located at a second position at a certain distance from the first position, while the sensor array covers the first structural feature again; (e) When the reference point is in the second position, a second set of sensor data is acquired from the second portion of the sensor array facing the target object; (f) Convert the second set of sensor data into second scan image data of the second scan image (42b); (g) Locate feature points (48) in the first scanned image and the second scanned image; (h) Determine which feature points found in step (g) are common feature points in the first scan image and the second scan image (48a); (i) Calculate the pixel position difference between the common feature points; and (j) Calculate the second Y-position coordinates of the reference point at the second position on the surface of the target object, based at least in part on the first Y-position coordinates and the pixel position difference calculated in step (i). (k) Calculate the second X-position coordinates of the reference point on the surface of the target object, based at least in part on the first X-position coordinates and the pixel position difference calculated in step (i). Step (j) includes: Calculate the first pixel offset, which is equal to the number of pixel rows by which the position of the common feature point in the second scanned image is offset relative to the position of the common feature point in the first scanned image; Multiply the first pixel offset by the distance between adjacent sensor rows to form a product equal to the first distance the scanner has translated in a direction perpendicular to the sensor rows during step (d); and Add the first distance to the first Y-position coordinate to obtain the second Y-position coordinate. Step (k) includes: Calculate the second pixel offset, which is equal to the number of pixel columns by which the position of the common feature point in the second scanned image is offset relative to the position of the common feature point in the first scanned image; Multiply the second pixel offset by the distance between adjacent sensor columns to form a product equal to the second distance the scanner has translated in a direction parallel to the sensor rows during step (d); and Add the second distance to the first X position coordinate to obtain the second X position coordinate.

5. The method of claim 4, further comprising processing the first scan image and the second scan image to form a composite scan image by aligning the second scan image data in the second scan image with the first scan image data in the first scan image using the common feature points in the first scan image and the second scan image.

6. The method of claim 5, further comprising: In the synthetic scan image, feature points representing the second structural features of the target object are found; The third Y-position coordinate of the feature point is calculated at least in part based on the first Y-position coordinate of the reference point; and The third X-position coordinate of the feature point is calculated at least in part based on the first X-position coordinate of the reference point.

7. A method for tracking the position of a scanner, the method comprising: (a) The sensor array (60) of the scanner (14) is placed adjacent to the surface of the target object such that the sensor array covers the first and second structural features of the target object and is oriented at a known first orientation angle; (b) When the sensor array is oriented at the first orientation angle, a first set of sensor data is acquired from a first portion of the target object facing the sensor array; (c) Convert the first set of sensor data into first scan image data of the first scan image (42a); (d) Orient the scanner such that the sensor array has a second orientation angle different from the first orientation angle, and again covers the first structural feature and the second structural feature; (e) When the sensor array is oriented at the second orientation angle, a second set of sensor data is acquired from the second portion of the target object facing the sensor array; (f) Convert the second set of sensor data into second scan image data of the second scan image (42b); (g) Locate feature points (48) in the first scanned image and the second scanned image; (h) Determine which feature points found in step (g) are common feature points in the first scan image and the second scan image (48a); (i) Calculate the pixel position difference between the first common feature point and the second common feature point; and (j) The second orientation angle is calculated at least in part based on the difference between the first orientation angle and the first angle of a first line connecting the first common feature point and the second common feature point at their respective positions in the first scanned image, and the second angle of a second line connecting the first common feature point and the second common feature point at their respective positions in the second scanned image. Step (i) includes: Calculate the first pixel offset, which is equal to the number of pixel rows by which the position of the second common feature point in the first scanned image is offset relative to the position of the first common feature point in the first scanned image; Calculate the second pixel offset, which is equal to the number of pixel columns by which the position of the second common feature point in the first scanned image is offset relative to the position of the first common feature point in the first scanned image; Calculate the third pixel offset, which is equal to the number of pixel rows by which the position of the second common feature point in the second scanned image is offset relative to the position of the first common feature point in the second scanned image; and Calculate the fourth pixel offset, which is equal to the number of pixel columns by which the position of the second common feature point in the second scanned image is offset relative to the position of the first common feature point in the second scanned image; Wherein, the tangent of the first angle is equal to the ratio of the first pixel offset to the second pixel offset, and the tangent of the second angle is equal to the ratio of the third pixel offset to the fourth pixel offset.

8. A system for tracking the position of a scanner, comprising a frame (2), a scanner (14) supported by the frame and including a two-dimensional sensor array (60), a robot system (50), and a computer system (40), the robot system (50) being configured to enable the frame to move motorized, the computer system (40) being communicatively coupled to receive sensor data from sensors in the sensor array and to send control signals to the robot system for controlling the motorized movement of the frame and being configured to perform the following operations: (a) When the sensor array covers a first structural feature of the target object with a reference point on the sensor array located at a first position on the surface of the target object, the sensor is activated, wherein the first position has known first X position coordinates and first Y position coordinates on the surface of the target object. (b) When the reference point is at the first position, receive a first set of sensor data acquired from a first portion of the sensor array facing the target object; (c) Convert the first set of sensor data into first scan image data of the first scan image (42a); (d) When the sensor array covers the first structural feature of the target object at a second position where the reference point is at a certain distance from the first position, the sensor is activated; (e) When the reference point is in the second position, receive a second set of sensor data acquired from the second portion of the sensor array facing the target object; (f) Convert the second set of sensor data into second scan image data of the second scan image (42b); (g) Locate feature points (48) in the first scanned image and the second scanned image; (h) Determine which feature points found in step (g) are common feature points in the first scan image and the second scan image (48a); (i) Calculate the pixel position difference between the common feature points; and (j) Calculate the second Y-position coordinates of the reference point at the second position on the surface of the target object, based at least in part on the first Y-position coordinates and the pixel position difference calculated in step (i). Step (j) includes: Calculate a first pixel offset, which is equal to the number of pixel rows of the sensor array whose position of the common feature point in the second scanned image is offset relative to the position of the common feature point in the first scanned image. The first pixel offset is multiplied by the distance between adjacent sensor rows to form a product, the product being equal to a first distance the scanner has translated in a direction perpendicular to the sensor rows during step (d); and Add the first distance to the first Y position coordinate to obtain the second Y position coordinate of the reference point.

Images