Aligning ultrasonic sensors using robotic systems and methods

Abstract

Data characterizing an initial position of an ultrasonic sensor is received. Information about internal reflections, which are reflected from within a component to be inspected, obtained by the ultrasonic sensor are received. A target position of the ultrasonic sensor relative to the initial position of the ultrasonic sensor is determined, based, at least in part, on the received data. A difference between the initial position and the target position is then provided.

Description

Atty. Dkt. No: 43UT-509996-WO-2 / 815001WOALIGNING ULTRASONIC SENSORS USING ROBOTIC SYSTEMS AND METHODSCROSS-CITATION

[0001] This application claims priority to United States Provisional Patent Application 63 / 760,254, Filed on 19 Februaiy 2025, entitled “ALIGNING ULTRASONIC SENSORS USING ROBOTIC SYSTEMS AND METHODS” the entirety of which is hereby incorporated by reference.TECHNICAL FIELD

[0002] The subject matter described herein relates to a robotic alignment system for ultrasonic sensors.BACKGROUND

[0003] Ultrasonic sensors can be used to nondestructively inspect components to detect flaws prior to operation. For this reason, ultrasonic non-destructive testing can be used in a number of industries such as automotive, aerospace, power generation or equipment used in oil and gas processing plants, to detect and correct flaws in the equipment prior to operation. To obtain the clearest readings, ultrasonic sensors should be properly aligned with the surface of the component to be inspected such that the ultrasonic signals received from the internal reflections of the component are improved.SUMMARY

[0004] This disclosure relates to technologies relating to aligning ultrasonic sensors using robotic systems and methods.

[0005] An implementation described within this disclosure is a method with the following features. Data characterizing an initial position of an ultrasonic sensor is received. Information about internal reflections, which are reflected from within a component to be inspected, obtainedAtty. Dkt. No: 43UT-509996-WO-2 / 815001WOby the ultrasonic sensor are received. A target position of the ultrasonic sensor relative to the initial position of the ultrasonic sensor is determined, based, at least in part, on the received data. A difference between the initial position and the target position is then provided.

[0006] Aspects of the example method, which can be combined with the example method alone or in combination with other methods, can include the following. The difference between the initial position and the target position is provided to a robotic arm. The robotic arm is adjusted to move the ultrasonic sensor to the target position based on the provided difference, adjusting the robotic arm involves adjusting an inclination of the ultrasonic sensor. Data characterizing the target position of the ultrasonic sensor is received in the form of information about a second set of internal reflections, which can be reflected from within the component to be inspected, once the robotic arm has been adjusted to move the ultrasonic sensor to the target position.

[0007] Aspects of the example method, which can be combined with the example method alone or in combination with other methods, can include the following. Data characterizing images of the component to be inspected are received. The images can be obtained by a camera at an end of the robotic arm. The ultrasonic sensor is at the end of the robotic arm. The initial position of the ultrasonic sensor is determined based upon the received data characterizing the images. Movements of the robotic arm to bring the ultrasonic sensor in contact with the initial position are determined. A signal configured to cause the robotic arm to move based on the determined movements is provided to the robotic arm.

[0008] Aspects of the example method, which can be combined with the example method alone or in combination with other methods, can include the following. Data characterizing the contact force and / or momentum applied to the probe as it makes contact with the component is received from force and / or momentum sensors. The force and or momentum sensors are at the distal end of the robotic arm. the contact force or momentum of the probe is controlled as it makes contact with the component. A signal to cause the robotic arm to move is provided based on the determined contact force and momentum applied to the ultrasonic sensor.

[0009] The disclosed method can be implemented in a variety of ways. For example, the method can be implemented within a system that includes an actuable robotic arm, an ultrasonic sensor at a distal end of the robotic arm, and a controller coupled to the robotic arm and the ultrasonic sensor. In some implementations, the robotic arm includes force or momentum sensors.Atty. Dkt. No: 43UT-509996-WO-2 / 815001WO

[0010] The controller comprises at least one data processor and a non-transitory memory storing instructions for the processor to perform aspects of the method. Alternatively or in addition, the controller can include non-transitory computer readable memory storing the method as instructions which, when executed by at least one data processor forming part of at least one computing system, causes the at least one data processor to perform one or more operations of the method. Alternatively or in addition, the robotic arm can be communicatively coupled to the processor such that the robotic arm can move the ultrasonic sensor based on the instructions from the controller. Alternatively or in addition, the system can include a camera configured to provide the data characterizing images of the component to be inspected .BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings, which are incorporated in and constitute a part of this specification, show certain aspects of the subject matter disclosed herein and, together with the description, help explain some of the principles associated with the disclosed implementations. In the drawings,

[0012] FIG. 1 is a schematic diagram of an example system having a robotic arm for aligning an ultrasonic sensor;

[0013] FIG. 2 is a flowchart of an example method for robotically aligning an ultrasonic sensor involving machine vision;

[0014] FIGS. 3 A-3C illustrate example images of robotically aligning an ultrasonic sensor involving machine vision;

[0015] FIG. 4 is a flowchart of an example method for robotically aligning an ultrasonic sensor involving ultrasonic readings of a component to be inspected by the ultrasonic sensor;

[0016] FIGS. 5A-5B illustrate example images of estimating an inclination of the ultrasonic sensor relative to a surface of the component;

[0017] FIG. 6 illustrates an example image of adjusting the inclination of the ultrasonic sensor such that it is normal to the surface of the component;Atty. Dkt. No: 43UT-509996-WO-2 / 815001WO

[0018] FIGS. 7A-7B illustrate example images of centering the ultrasonic sensor relative to the component;

[0019] FIGS. 8A-8D illustrate example images of adjusting the inclination of the ultrasonic sensor relative to internal reflections; and

[0020] FIG. 9 is a block diagram of an example controller that can be used with aspects of this disclosure.DETAILED DESCRIPTION

[0021] Certain implementations will now be described to provide an overall understanding of the principles of the structure, function, manufacture, and use of the devices and methods disclosed herein. One or more examples of these implementations are illustrated in the accompanying drawings. Those skilled in the art will understand that the devices and methods specifically described herein and illustrated in the accompanying drawings are non-limiting implementations and that the scope of the present invention is defined solely by the claims. The features illustrated or described in connection with one implementation can be combined with the features of other implementations. Such modifications and variations are intended to be included within the scope of the present invention.

[0022] Further, in the present disclosure, like-named components of the implementations generally have similar features, and thus within a particular implementation each feature of each like-named component is not necessarily fully elaborated upon. Additionally, to the extent that linear or circular dimensions are used in the description of the disclosed systems, devices, and methods, such dimensions are not intended to limit the types of shapes that can be used in conjunction with such systems, devices, and methods. A person skilled in the art will recognize that an equivalent to such linear and circular dimensions can easily be determined for any geometric shape. Sizes and shapes of the systems and devices, and the components thereof, can depend at least on the anatomy of the subject in which the systems and devices will be used, the size and shape of components with which the systems and devices will be used, and the methods and procedures in which the systems and devices will be used.Atty. Dkt. No: 43UT-509996-WO-2 / 815001WO

[0023] As discussed previously, to obtain clear readings, ultrasonic sensors should be properly aligned with the surface of the component to be inspected such that the ultrasonic signals received from the internal reflections of the component are maximized. However, users may find it difficult to manually align the ultrasonic sensors relative to the surface of the component to receive clear readings, especially when the surface is flexible, uneven, and / or contoured. Thus, manually inspecting a component using such ultrasonic sensors may prove time-consuming and imprecise. Accordingly, there remains a need for improved systems and methods for aligning ultrasonic sensors for the inspection of components.

[0024] To address this need, systems and methods of robotically aligning ultrasonic sensors for the inspection of objects are disclosed herein. Tn the context of this disclosure, an ultrasonic sensor (also referred to herein as a “probe”) is positioned on an actuable robotic mechanism (also referred to herein as a “robotic arm”). The robotic arm is configured to move the probe along myriad directions, taking the probe from an initial position (e.g., in which the probe is in the vicinity of a component to be inspected) to a target position (e.g., in which the probe is properly aligned relative to a surface of the component to be inspected). To move the probe into the initial position, machine vision methods can be used to process visual data e.g., in the form of images from a camera) and determine how to position the robotic arm such that the probe is roughly in the vicinity of the component to be inspected. Once the probe is in the initial position, adjustments can be made to move the robotic arm into the target position. The adjustments can fine-tune the alignment of the probe relative to the surface of the component to be inspected based on ultrasonic signals received by the probe from the internal reflections of the component. These adjustments can include adjustments of the probe’s position, inclination, force of contact against the surface, etc. Data from additional sensors, such as force and / or momentum sensors, can be used to confirm that the probe is properly aligned against the component.

[0025] Correctly aligning the probe relative to the component, as can be achieved using the systems and methods described herein, is crucial for generating accurate and reproducible readings. Further, controlling the contact force and / or momentum of the probe as it makes contact with the component is crucial for preventing damage to the probe and / or other components of the robotic arm. The systems and methods described herein allow for faster and more accurate probe positioning that can be automatically performed by the robotic arm without user input, which, in turn, allows for faster and more autonomous inspection of components. Thus, the automated,Atty. Dkt. No: 43UT-509996-WO-2 / 815001WOrobotic inspection of components increases reproducibility and reduces the cost of each inspection while increasing the number of inspections which can be performed, saving money while producing more accurate ultrasonic readings compared to manual methods.

[0026] In some implementations, the robotic arm includes an ultrasonic sensor at a distal end of the robotic arm. The ultrasonic sensor is configured to move from an initial position to a target position based on internal reflections from within a component to be inspected by the robotic arm and / or based on camera images of the component to be inspected. A controller coupled to the robotic arm and the ultrasonic sensor performs a variety of operations in order to move the ultrasonic sensor. In some implementations, the controller receives data characterizing the initial position of the ultrasonic sensor. Additionally, the controller receives information about the internal reflections from within the component to be inspected. Based on the data characterizing the initial position and / or the internal reflection information, the controller determines the target position of the ultrasonic sensor. The controller can then provide a difference between the initial position and the target position (e.g., to the robotic arm) and adjust the robotic arm to move the ultrasonic sensor to the target position based on the provided difference.

[0027] FIG. 1 is a schematic diagram of an example system 100 compatible with the automated, robotic inspection methods described herein. According to some implementations, the system 100 has a robotic arm 102 for aligning an ultrasonic sensor 104 relative to a component 199. The robotic arm 102 is electrically coupled to and is configured to support the ultrasonic sensor 104, as well as several other sensors such as an optical sensor 106 and a force / momentum sensor 108. The controller 110 is configured to control the robotic arm 102. The controller 110 is communicatively coupled to one or more motors and / or actuators of the robotic arm 102, an ultrasonic transmitter module 104a of the ultrasonic sensor 104, an optical transmitter module 106a of the optical sensor 106, and / or a position transmitter module 108a of the force / momentum sensor 108. More details on the controller 110 are described later within this disclosure.

[0028] As shown in FIG. 1, the robotic arm 102 of the system 100 includes a mechanism with a base, one or more actuable segments extending from the base, and one or more motors and / or actuators configured to maneuver the actuable segments as instructed by the controller 110. In some implementations, adjusting the robotic arm 102 involves changing an angle between actuable segments, a linear position of an actuable segment, and / or a rotational position of anAtty. Dkt. No: 43UT-509996-WO-2 / 815001WOactuable segment in order to change a position and / or inclination of one or more sensors attached to the robotic arm 102 (e.g., ultrasonic sensor 104, optical sensor 106). The adjustment of the robotic arm 102 can be driven by the controller 110, which instructs the motors, and / or actuators to move in the appropriate parameters to achieve the desired adjustments. In some implementations, the robotic arm 102 can be powered (e.g., via a battery or an electrical coupling to a power source) and be electrically coupled to one or more components that require power (e.g., motors, sensors, etc.). In some implementations, the robotic arm 102 can be driven by hydraulic and / or pneumatic actuators.

[0029] With respect to sensor placement, in some implementations, at a distal end of the robotic arm 102 (e.g., a distal end of an actuable segment), the ultrasonic sensor 104 is attached to the robotic arm 102. Other sensors such as the optical sensor 106 and / or the force / momentum sensor 108 can also be attached to the robotic arm 102 such that they are proximate to the end of the robotic arm 102. However, it is to be understood that the illustrated robotic arm 102 of FIG. 1 is merely exemplary and that the sensors 104, 106, 108 do not necessarily need to be positioned at the distal end of the robotic arm 102; indeed, the ultrasonic sensor 104, optical sensor 106, and / or the force / momentum sensor 108 can be positioned at any point along the robotic arm 102 so long as the ultrasonic sensor 104 can make contact with a component 199 to be inspected without obstruction by the other components of the robotic arm 102.

[0030] The ultrasonic sensor 104, the optical sensor 106, and the force / momentum sensor 108 can take on any form so long as they are compact and lightweight enough to attach to the robotic arm 102 without interfering with the actuation of the robotic arm 102. In some implementations, the ultrasonic sensor 104 is a probe and / or transducer configured to make contact with the component 199 and to produce ultrasonic waves that reflect internally within the component 199. The ultrasonic sensor 104 can detect, collect, and / or transmit data pertaining to the internal reflections that are informative of the structure of the component 199. The ultrasonic sensor 104 is communicatively coupled to the controller 110 and can transmit internal reflection data to the controller 110 (e.g., via wired transmission, wireless transmission, Bluetooth, radio, etc.). In some implementations, the optical sensor 106 is a camera and / or imager configured to collect and / or transmit visual data (e.g., images) of the component 199. The optical sensor 106 is communicatively coupled to the controller 110 and can transmit visual data to the controller 110 (e.g., via wired transmission, wireless transmission, Bluetooth, radio, etc.) In someAtty. Dkt. No: 43UT-509996-WO-2 / 815001WOimplementations, the force / m omentum sensor 108 is a force probe and / or transducer that detects the force of contact between a component of the robotic arm 102 (e.g., the ultrasonic sensor 104) and the component 199 to be inspected. In some implementations, the force / momentum sensor 108 is an accelerometer and / or gyroscope that detects the linear and / or rotational movement of the robotic arm as it moves toward the component 199 to be inspected. The force / momentum sensor 108 is communicatively coupled to the controller 110 and can transmit force and / or momentum data to the controller 110 (e.g., via wired transmission, wireless transmission, Bluetooth, radio, etc.) Used in conjunction with the controller 110, data collected from the force and / or momentum sensor 108 can be used to control the force and momentum applied between the ultrasonic sensor 104 and the component 199 to avoid damage during contact. In some implementations, the force applied between the ultrasonic sensor 104 and the component 199 is limited to less than or equal to 1 N, and the momentum applied to the ultrasonic sensor 104 is limited to less than or equal to 0.02 Nm.

[0031] In some implementations, the controller 110 includes at least one data processor and non-transitory memory storing instructions, that, when executed by the at least one data processor causes the at least one data processor to perform a variety of operations related to the adjustment and alignment of the robotic arm 102. More details on the operations related to the alignment of the robotic arm 102 are described later within this disclosure, such as at FIG. 2 below.

[0032] FIG. 2 is a flowchart of an example method 200 for robotically aligning an ultrasonic sensor involving machine vision, according to some implementations. Aspects of the method 200 can be performed by the controller 110, all or in part, using any components of the system 100 as described above. At 202, the controller 110 receives, from a camera, data (e.g., visual data) characterizing images of the component to be analyzed. The camera can be attached to an end of a robotic arm that contains an ultrasonic probe configured to make contact with the component to be analyzed. The camera is configured to capture visual data in any variety of wavelengths, including but not limited to visual light wavelengths. The controller 110 can use machine vision methods to interpret features within the visual data and determine how to align the ultrasonic sensor. At 204, the controller 110 can determine an initial position of the ultrasonic sensor based on the received visual data from 202. As used herein, the initial position of the ultrasonic sensor can be a desired position to which the ultrasonic sensor should be moved prior to the initiation of any ultrasonic methods (e.g., method 400 of FIG. 4). In some implementations,Atty. Dkt. No: 43UT-509996-WO-2 / 815001WOthe initial position of the ultrasonic sensor is any position in which the sensor is in the vicinity of a component to be inspected, regardless of whether contact is made between the sensor and the component. In some implementations, the initial position of the ultrasonic sensor is a predetermined position in which the sensor makes contact with a feature of interest on the component to be inspected. In order to move the ultrasonic sensor into contact with the component, the controller 110 first determines the movements of the robotic arm necessary to bring the ultrasonic sensor into the initial position at 206. At 208, the controller 110 then provides a signal configured to cause the robotic arm to move the ultrasonic sensor into the initial position based on the determined movements. In some implementations, the controller 110 continues receiving data from the camera throughout the movement of the robotic arm to monitor whether the ultrasonic sensor has been successfully moved into the initial position. Upon completion of the movement, the ultrasonic sensor has been moved into the initial position relative to the component to be inspected, and the ultrasonic methods (e.g., method 400 of FIG. 4) can then be initiated.

[0033] FIGS. 3A-3C illustrate example images 300a, 300b, 300c, respectively, of robotically aligning an ultrasonic sensor into the initial position involving machine vision, as received by the controller 110 during the execution of method 200 of FIG. 2. As shown in image 300a of FIG. 3 A, at 202, prior to any movement of the robotic arm, a camera captures an image 300a of the component to be inspected. At 204, the controller 110 uses machine vision to interpret the image 300a, identifying a shape / position of a feature of interest 304 as well as an initial position 302 to which the feature of interest should be moved in order to obtain optimal ultrasonic imaging results. For example, the initial position 302 can be a position in which the feature of interest 304 is centered relative to the borders of an ultrasonic imaging window. To achieve the initial position 302, the robotic arm should be moved such that the feature of interest 304 is visually centered and / or aligned with the crosshairs of the initial position 302 (as illustrated in image 300c of FIG.3C). At 206, the controller 110 calculates movements for the robotic arm based, at least in part, on the image 300a, and, at 208, provides a signal (e.g. instructions) to the robotic arm 102 to move accordingly. Image 300b of FIG. 3B illustrates an intermediary view from the camera in which the robotic arm has been moved to begin visually aligning the feature of interest 304 with the crosshairs denoting the initial position 302. Image 300c of FIG. 3C illustrates a view from the camera after the robotic arm has completed its movement to align the feature of interest 304 with the crosshairs denoting the initial position 302.Atty. Dkt. No: 43UT-509996-WO-2 / 815001WO

[0034] Once the robotic arm, and by extension, the ultrasonic sensor, is properly aligned (e. ., the ultrasonic sensor has been moved into the initial position and has made contact with the component to be inspected), the camera may no longer be used to provide information on the movement of the robotic arm. Instead of using machine vision, the controller 110 can instead rely on ultrasonic readings from the ultrasonic sensor itself to make any further adjustments to the position and / or inclination of the ultrasonic sensor.

[0035] FIG. 4 is a flowchart of an example method 400 for robotically aligning an ultrasonic sensor involving ultrasonic readings of a component to be inspected by the ultrasonic sensor, according to some implementations. Aspects of the method 400 can be performed by the controller 110, all or in part, using any components of the system 100 as described above. The controller 110 can perform the method 400 on its own, or, optionally, method 400 can be performed in sequence after the method 200 of FIG. 2. At 402, the controller 110 receives data characterizing the initial position of the ultrasonic sensor. The data is obtained from any source communicatively coupled to the controller 110, such as from an input provided via an input / output interface, from a sensor (e.g., visual data collected by the camera during execution of the method 200), and / or from an external database, etc. The data characterizing the initial position of the ultrasonic sensor includes information such as coordinates of the ultrasonic sensor (e.g., relative to a feature of interest on the component to be inspected), inclination of the ultrasonic sensor, rotational position of the ultrasonic sensor, force of contact against the component, etc. At 404, the controller 110 receives information about internal reflections received by the ultrasonic sensor. The internal reflections are the result of ultrasonic signals reflected from within the component and yield information about the composition, integrity, structure, etc. of the component. The ultrasonic signals used to induce the internal reflections can be emitted by the same ultrasonic sensor that detects the internal reflections, or in some implementations, the ultrasonic signals can be emitted by a separate ultrasonic transducer. Based on the initial set of internal reflection data collected at the initial position at 404, the controller 110 determines if any additional adjustments are needed to yield clearer internal reflection data (e.g., adjusting an inclination of the ultrasonic sensor relative to a surface of the component). At 406, the controller 110 determines a target position for the ultrasonic sensor. In some implementations, the target position can be a position to which the ultrasonic sensor can move in order to obtain clearer internal reflection data, relative to the initial set of internal reflection data obtained at the initial position of the ultrasonic sensor.Atty. Dkt. No: 43UT-509996-WO-2 / 815001WOAt 408, the controller 110 provides a difference between the initial position and the target position. The difference can be provided to the robotic arm (e.g., in order to prime the motors and / or actuators of the robotic arm for subsequent movement). Optionally, upon providing the difference between the positions to the robotic arm, the controller 110 can adjust the robotic arm to move the ultrasonic sensor to the target position based on the provided difference. Adjusting the robotic arm can involve issuing one or more instructions to the motors and / or actuators of the robotic arm to adjust the inclination, position, and / or rotation of the robotic arm relative to the component to be imaged. Then, to confirm that the internal reflection data is clearer at the target position, the ultrasonic sensor can collect a second set of internal reflection data that characterizes the target position of the ultrasonic sensor. If necessary, the controller 110 can determine an updated target position for the ultrasonic sensor based on the second set of internal reflection data, and the steps at 406-onward can be repeated in order to fine-tune the final position of the ultrasonic sensor. In some implementations, a third, fourth, fifth, sixth, etc. set of internal reflection data is collected, and any of the steps of the method 400 are iterated to refine the final position of the ultrasonic sensor. Upon the completion of the method 400, the ultrasonic sensor is in the final position and is ready to obtain its final set of internal reflection data as part of its inspection of the component.

[0036] FIGS. 5A-5B illustrate example images 500a, 500b of estimating an inclination of the ultrasonic sensor relative to a surface of the component, as can be performed during the execution of method 400 of FIG. 4. Using the ultrasonic signals received by the ultrasonic sensor (e.g., the sets of internal reflection data collected during method 400), the inclination of the ultrasonic sensor relative to the surface of the component can be estimated. In some implementations, as shown in image 500a of FIG. 5A, a plane corresponding to the initial inclination of the ultrasonic sensor relative to the surface is calculated based on factors such as the known alignment of the ultrasonic sensor relative to the surface, the depth information provided by the internal reflection data, and / or a reconstructed ultrasound image of the surface. As shown in image 500b of FIG. 5B, the difference between the normal vector n of the plane (corresponding to the initial inclination of the ultrasonic sensor relative to the surface) and the vertical axis (representing the central axis of the plane, wherein the axis is drawn as normal to a horizontal plane centered along the same axis) is represented by the inclination angle a and the azimuthal angle y. The inclination angle a and the azimuthal angle y are used to determine the robotic arm movements needed to properly align the ultrasonic sensor to the surface (e.g., to reduce theAtty. Dkt. No: 43UT-509996-WO-2 / 815001WOinclination angle a). For example, FIG. 6 illustrates an example image 600a of adjusting the inclination of the ultrasonic sensor such that it is normal to the surface of the component, as can be performed during the execution of method 400 of FIG. 4. In the image 600a of FIG. 6, the ultrasonic sensor has been adjusted to be in the target position such that the ultrasonic sensor is correctly aligned to receive clear internal reflection data. As shown in image 600a, the plane corresponding to the target inclination of the ultrasonic sensor relative to the surface is substantially horizontal and level.

[0037] In addition to adjusting the inclination of the ultrasonic sensor relative to the surface, it may also be necessary for the component to be inspected to be approximately centered within the imaging window of the ultrasonic sensor. FIGS. 7A-7B illustrate example images of centering the ultrasonic sensor relative to the component, as can be performed during the execution of method 200 of FIG. 2 and / or method 400 of FIG. 4. As shown in image 700a of FIG. 7A, a feature of interest 704 on the component to be inspected is initially off-center relative to the image frame. In the initial position, the ultrasonic sensor is at a position and / or inclination relative to the component that is not ideal for detecting internal reflections within the component. To correct for this, the ultrasonic sensor can be centered and / or properly inclined, as shown in image 700b of FIG. 7B. The distance between the feature of interest 704 and the center of the image frame, as represented by the crosshairs 702, can be used to determine how much the ultrasonic sensor should be moved. In some implementations, the controller 110 instructs the robotic arm to move the ultrasonic sensor until the feature of interest 704 is centered near the crosshairs 702 of the image frame, allowing for the ultrasonic sensor to be correctly centered with respect to the internal reflections in the component to be inspected. In some implementations, the images 700a, 700b are ultrasonic images generated based on the internal reflections received by the ultrasonic sensor. In some implementations, the images 700a, 700b are visual images collected by the camera and processed by the controller 110 using the machine vision methods.

[0038] In addition to adjusting the inclination of the ultrasonic sensor relative to the surface and the imaging window of the ultrasonic sensor, in some implementations, the inclination of the ultrasonic sensor is adjusted relative to internal reflections. FIGS 8A-8D illustrated example images of adjusting the inclination of the ultrasonic sensor relative to internal reflections, as can be performed during the execution of method 200 of FIG. 2 and / or method 400 of FIG. 4. Using the ultrasonic signals received by the ultrasonic sensor (e.g., the sets of internal reflection dataAtty. Dkt. No: 43UT-509996-WO-2 / 815001WOcollected during method 400), the inclination of the ultrasonic sensor relative to internal reflections of the component can be estimated. As shown in image 800a of FIG. 8 A, the initial inclination of the ultrasonic sensor relative to internal reflections is calculated based on factors such as the depth information provided by the internal reflection data and / or a reconstructed ultrasound image of internal reflections. The initial inclination is used to determine the robotic arm movements needed to properly align the ultrasonic sensor to internal reflections (e.g., to reduce the inclination). For example, in image 800b of FIG. 8B the ultrasonic sensor has been adjusted such that the ultrasonic sensor is correctly aligned to receive clear internal reflection data, as can be performed during the execution of method 400 of FIG. 4. As shown in image 800c of FIG 8C a feature of interest 804 of the component to be inspected is initially inclined within the image frame. In the initial position, the ultrasonic sensor is at an inclination relative to the internal reflection that is not ideal for detecting internal reflections within the component. To correct for this, the ultrasonic sensor can be properly inclined, as shown in image 800d of FIG 8D. The inclination of the feature of interest can be used to determine how movements of the robotic arm that are needed to align the ultrasonic sensor 104. In some implementations, the controller 110 instructs the robotic arm to align the ultrasonic sensor until the feature of interest 804 is substantially horizontal and level in the image frame. This allows the ultrasonic sensor to be correctly aligned with respect to the internal reflections in the component to be inspected. In some implementations, the images 800c, 800d are ultrasonic images generated based on the internal reflections received by the ultrasonic sensor.

[0039] FIG. 9 illustrates a block diagram of an example controller 110 that can be used with some aspects of the current subject matter. The controller 110 can, among other things, monitor parameters of the system 100 and send signals to actuate and / or adjust various operating parameters of such systems. As shown in FIG. 9, the controller 110 includes one or more processors 950 and non-transitory computer readable memory storage (e.g., memory 952) containing instructions that cause the processors 950 to perform operations. The processors 950 are coupled to an input / output (VO) interface 954 for sending and receiving communications with components in the system, including, for example, the ultrasonic sensor 104, the optical sensor 106, the force / momentum sensor 108, the motor(s) and / or actuator(s) of the robotic arm, and / or any other electronic components of the system. In certain instances, the controller 110 can additionally communicate status with and send actuation and / or control signals to one or more ofAtty. Dkt. No: 43UT-509996-WO-2 / 815001WOthe various system components of the system 100, as well as other sensors and components that provide signals to the system 100.

[0040] The controller 110 can be implemented with various levels of autonomy. In some implementations, the controller 110 alerts an operator that a parameter is out of specification, for example, that no ultrasonic readings are being detected, and the operator then adjusts the position, sensitivity, on / off status, etc. of the ultrasonic sensor to detect ultrasonic readings. In some implementations, the controller 110 alerts the operator that a parameter is out of specification and provides recommendations to the operator to move the parameter within specification. The operator then selects an option, and the controller adjusts operations accordingly. In some instances, the controller 110 determines that a parameter is out of specification and changes or otherwise adjusts operations to move the parameter within specification with no input from the operator.

[0041] It is to be understood that the controller 110, as well as any other components of the system 100, can be implemented in alternate configurations that enable the alignment of the ultrasonic sensor relative to the component. In some implementations, the ultrasonic waves emitted by the ultrasonic sensor are steered electronically using ultrasonic arrays to improve the reflection signals based on the surface geometry and position of internal reflections (e.g., dynamic electronic steering). In some implementations, the received ultrasonic signals are processed by ultrasonic image reconstruction algorithms. In some implementations, rather than directly moving the robotic arm to adjust the position and / or inclination of the ultrasonic sensor, additional motion controllers, such as motors attached directly to the ultrasonic sensor, are used to control the position and / or inclination of the ultrasonic sensor. In some implementations, rather than relying on active force / momentum sensors to implement motion control of the robotic arm, the robotic arm includes passive control elements (e.g., springs and / or distance spacers) for controlling the forces / momentum acting on the ultrasonic probe.

[0042] While this disclosure contains many specific implementation details, these should not be construed as limitations on the scope of what may be claimed, but rather as descriptions of features specific to particular implementations. Certain features that are described in this disclosure in the context of separate implementations can also be implemented in combination in a single implementation. Conversely, various features that are described in the context of a singleAtty. Dkt. No: 43UT-509996-WO-2 / 815001WOimplementation can also be implemented in multiple implementations separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

[0043] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Moreover, the separation of various system components in the implementations described above should not be understood as requiring such separation in all implementations, and it should be understood that the described components and systems can generally be integrated together in a single product or packaged into multiple products.

[0044] Thus, particular implementations of the subject matter have been described. Other implementations are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results.

[0045] Other implementations can be within the scope of the following claims.

Claims

Atty. Dkt. No: 43UT-509996-WO-2 / 815001WOCLAIMSWhat is claimed:

1. A method compri sin :receiving data characterizing an initial position of an ultrasonic sensor; receiving information about internal reflections received by the ultrasonic sensor, the internal reflections being reflected from within a component to be inspected;determining a target position of the ultrasonic sensor, based at least in part on the received data, relative to the initial position of the ultrasonic sensor; andproviding a difference between the initial position and the target position.

2. The method of claim 1, wherein providing comprises providing the difference to a robotic arm, the method further comprising adjusting the robotic arm to move the ultrasonic sensor to the target position based on the provided difference.

3. The method of claim 2, wherein adjusting the robotic arm comprises adjusting an inclination of the ultrasonic sensor.

4. The method of claim 3, further comprising receiving data characterizing the target position of the ultrasonic sensor, the data comprising a second set of internal reflections received by the ultrasonic sensor, the second set of internal reflections being reflected from within the component to be inspected.

5. The method of any one of claims 2-4, further comprising:receiving data characterizing images of the component to be inspected, the images received from a camera at an end of the robotic arm, the ultrasonic sensor being at the end of the robotic arm;determining the initial position of the ultrasonic sensor based upon the received data characterizing the images;determining movements of the robotic arm to bring the ultrasonic sensor in contact with the initial position; andproviding a signal configured to cause the robotic arm to move based on the determined movements.Atty. Dkt. No: 43UT-509996-WO-2 / 815001WO6. A system comprising:an actuable robotic arm;an ultrasonic sensor at a distal end of the robotic arm; anda controller coupled to the robotic arm and the ultrasonic sensor, the controller comprising:at least one data processor; andnon-transitory memory storing instructions, which, when executed by the at least one data processor causes the at least one data processor to perform operations comprising:receiving data characterizing an initial position of the ultrasonic sensor; receiving information about internal reflections received by the ultrasonic sensor, the internal reflections being reflected from within a component to be inspected;determining a target position of the ultrasonic sensor, based at least in part on the data, relative to the initial position of the ultrasonic sensor; andproviding a difference between the initial position and the target position.

7. The system of claim 6, wherein providing comprises providing the difference to the robotic arm, the operations further comprising adjusting the robotic arm to move the ultrasonic sensor to the target position based on the provided difference.

8. The system of claim 7, wherein adjusting the robotic arm comprises adjusting an inclination of the ultrasonic sensor.

9. The system of any one of claims 7-8, further comprising receiving data characterizing the target position of the ultrasonic sensor, the data comprising a second set of internal reflections received by the ultrasonic sensor, the second set of internal reflections being reflected from within the component to be inspected.

10. The system of any one of claims 7-9, wherein the system further comprises a camera.

11. The system of claim 10, wherein the operations further comprise:receiving data characterizing images of a component to be inspected, the images received from the camera, the camera being at the distal end of the robotic arm;determining the initial position of the ultrasonic sensor based upon the received data;Atty. Dkt. No: 43UT-509996-WO-2 / 815001WOdetermining movements of the robotic arm to bring the ultrasonic sensor in contact with the initial position; andproviding a signal configured to cause the robotic arm to move based on the determined movements.

12. The system of any one of claims 7-11, wherein the system further compromises force or momentum sensors.

13. The system of claim 12, wherein the operations further compromise:receiving data characterizing the contact force and / or momentum applied to the probe as it makes contact with the component, the data being received from force or momentum sensors, the force and or momentum sensors being at the distal end of the robotic arm;controlling the contact force or momentum of the probe as it makes contact with the component; andproviding a signal to cause the robotic arm to move based on the determined contact force and momentum applied to the ultrasonic sensor.

14. A system comprising:one or more processors; andnon-transitory memory storing instructions, which, when executed by the one or more processors, causes one or more processors to perform operations comprising:receiving data characterizing an initial position of an ultrasonic sensor; receiving information about internal reflections received by the ultrasonic sensor, the internal reflections being reflected from within a component to be inspected;determining a target position of the ultrasonic sensor, based at least in part on the data, relative to the initial position of the ultrasonic sensor; andproviding a difference between the initial position and the target position.

15. The system of claim 14, wherein the system further comprises a robotic arm commutatively coupled to the one or more processors.

16. The system of claim 15, wherein providing comprises providing the difference to the robotic arm, the operations further comprising adjusting the robotic arm to move the ultrasonic sensor to the target position based on the provided difference.Atty. Dkt. No: 43UT-509996-WO-2 / 815001WO17. The system of claim 16, wherein adjusting the robotic arm comprises adjusting an inclination of the ultrasonic sensor.

18. The system of any one of claims 16-17, further comprising receiving data characterizing the target position of the ultrasonic sensor, the data comprising a second set of internal reflections received by the ultrasonic sensor, the second set of internal reflections being reflected from within the component to be inspected.

19. The system of any one of claims 14-18, wherein the system further comprises a camera.

20. The system of claim 19, wherein the operations further comprise:receiving data characterizing images of a component to be inspected, the images received from a camera at an end of the robotic arm, the ultrasonic sensor being at the end of the robotic arm;determining the initial position of the ultrasonic sensor based upon the received data;determining movements of the robotic arm to bring the ultrasonic sensor in contact with the initial position; andproviding a signal configured to cause the robotic arm to move based on the determined movements.

21. The system of any one of claims 14-20, wherein the system further compromises force or momentum sensors.

22. The system of claim 21, wherein the operations further compromise:receiving data characterizing the contact force and / or momentum applied to the probe as it makes contact with the component, the data received from force or momentum sensors, the force and or momentum sensors being at the distal end of the robotic arm;controlling the contact force or momentum of the probe as it makes contact with the component; andproviding a signal to cause the robotic arm to move based on the determined contact force and momentum applied to the ultrasonic sensor.

Images