Contact-based infrastructure inspection system

Abstract

An unmanned aerial vehicle (UAV)-based system for contact inspection of civil infrastructure includes a UAV equipped with a modular payload comprising an actuated boom, contact-based end effector, and multi-modal sensor suite. The end effector uses spring-loaded strikers or toothed rollers to excite the structure surface, generating diagnostic acoustic vibrations. Onboard sensors measure the acoustic response, which is analyzed in real-time by a machine learning model to detect and quantify subsurface defects like delamination, voids, and cracks. Auxiliary sensors enable intelligent navigation relative to the structure surface and precise defect localization. The system allows remote, automated inspection of hard-to-reach areas, overcoming the safety and accuracy limitations of manual methods. By enabling efficient, comprehensive, and quantitative structural health evaluation, this invention will facilitate proactive infrastructure maintenance, extending service life and ensuring public safety.

Description

TITLECONTACT-BASED INFRASTRUCTURE INSPECTION SYSTEMCROSS-REFERENCE TO RELATED APPLICATIONS]

[0001] The present application claims priority to US Provisional Application No. 63 / 512,627, filed July 8, 2023, “Contact-Based Infrastructure Inspection System”; and US Provisional Application No. 63 / 529,105, filed July 26, 2023, “Toothed Roller for Contact-Based Infrastructure Inspection System,” each of which is hereby incorporated by reference in its entirety.TECHNICAL FIELD

[0002] The present invention relates generally to inspection systems for civil infrastructure, and more particularly to a contact-based inspection system using unmanned aerial vehicles.BACKGROUND

[0003] Inspection of civil infrastructure such as bridges, dams, and buildings is critical for ensuring public safety and optimal performance of these structures over their service life. However, current manual inspection techniques have several limitations.

[0004] Inspectors must physically access every part of a structure, which can be dangerous, time-consuming and costly, especially for large or hard-to-reach areas. This leads to infrequent and inadequate inspections.

[0005] Even with close-up access, many defects like subsurface delamination are difficult to detect visually. Inspectors typically use hammers to acoustically detect delamination, but this method is subjective, imprecise, and unable to quantify defect severity. As a result, critical defects can go unnoticed until they become severe, jeopardizing structural integrity.

[0006] There is a clear need for an improved infrastructure inspection system that overcomes the limitations of manual methods. Such a system should enable remote inspection of all areas, provide quantitative defect detection, and integrate seamlessly into existing workflows to improve safety and efficiency.SUMMARY

[0007] The present invention addresses this need through an unmanned aerial vehicle (UAV)-based system that uses contact sensors and real-time data analysis to automate infrastructure inspection while enhancing defect detection capabilities. This could include non-destructive testing for delamination, voids, crack depth measurement, compressive strength measurement, etc. The illustrative embodiments include features enabling the contact inspection to occur with market available UAV’s that have the inventive payload attached.

[0008] A primary embodiment of the present invention is a contact-based inspection system that utilizes unmanned aerial vehicles (UAVs) to remotely evaluate the structural health of civil infrastructure. It consists of a UAV equipped with a specialized payload that includes a boom, a modular end effector, and a sensor suite for real-time data collection and defect detection.

[0009] The end effector is a core component of the system, housing the contact-based inspection tools. In one embodiment, it uses a spring-loaded striker to impact the structure surface, exciting acoustic waves. In another implementation, a toothed roller is used, where teeth encounter surface irregularities as the UAV moves, causing spring- loaded impacts. The sensor suite, including accelerometers and microphones, captures the acoustic response which contains valuable information about the subsurface material properties.

[0010] The captured sensor data is analyzed in real-time by an onboard machine learning model trained to detect and localize defects like delamination, voids, and cracks. The model learns to differentiate between the acoustic signatures of damaged and intact areas, considering features like amplitude, frequency content, and duration.

[0011] The illustrative embodiment also includes auxiliary sensors like cameras and distance sensors that enable autonomous UAV navigation and high-precision spatial mapping of detected defects. Intuitive operation is enabled by assistive features like surface-relative navigation, where the end effector automatically maintains contact with the structure surface as the UAV moves.

[0012] This combination of UAV mobility, contact-based sensing, and onboard data analytics enables the invention to achieve several key objectives. It eliminates the needfor direct human access, greatly improving inspection safety and efficiency. The automated defect detection is highly sensitive and provides quantitative severity assessments, a major leap over current manual methods. And integration with existing UAV platforms and workflows allows easy adoption.

[0013] In summary, the present invention may be embodied in a system that provides a comprehensive solution to the challenges of civil infrastructure inspection. Its innovative design allows for safe, efficient, and precise evaluation of structural health, enabling proactive maintenance to extend infrastructure life and ensure public safety.Key benefits include:• Improved safety by replacing manual inspections that are difficult and dangerous to access.• Higher inspection coverage and frequency since UAV-based inspections are faster and easier.• Potential to swap tools based on inspection requirements (e.g. delamination, cracks, voids).

[0014] The disclosed embodiments provide a practical means of conducting contactbased inspection of civil infrastructure using unmanned aerial vehicles. The payload system addresses challenges like maintaining contact with surfaces at different angles, reducing reaction forces and automatically detecting defects.

[0015] Moreover, adding computer vision to this inspection system would provide improvements like higher precision, more proactive detection capabilities, automated defect marking and reporting, and adaptive inspection path planning. The combination of contact and vision-based approaches also increases the robustness and reliability of the overall system.BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 depicts an embodiment of a UAV configured in accordance with the present invention.

[0017] Figure 2 is a high-level flowchart of a method of operation.

[0018] Figure 3 is a more detailed flowchart of the method of operation.

[0019] Figure 4 depicts an embodiment of a toothed roller of the kind available on the market for the use of detecting delamination manually.

[0020] Figure 5 is a spectrogram of rolling striking on control and delamination surfaces.DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

[0021] Overview

[0022] The following detailed description presents a novel system for contact-based inspection of civil infrastructure using unmanned aerial vehicles (UAVs). This system significantly improves conventional manual inspection methods in terms of safety, efficiency, coverage, and precision.

[0023] The core of the system is a specialized payload that attaches to a UAV and includes an actuated end effector equipped with various inspection tools. This allows the UAV to perform contact-based assessments of structures at different angles and orientations, greatly expanding the range of surfaces that can be inspected.

[0024] One key aspect of the system is its ability to detect defects like delamination in concrete structures. This is achieved through innovative excitation methods such as an automated hammer or a rolling toothed tool that generates diagnostic acoustic signals. Onboard sensors capture these signals, and real-time machine learning algorithms analyze them to identify and localize areas of damage with high precision.

[0025] Another important component is the system's advanced navigation capabilities. Proximity sensors and control algorithms allow the end effector to maintain consistent contact with the structure surface, even as it changes orientation. This enables detailed inspections of walls, ceilings, and other angled surfaces that would be difficult or dangerous to access with manual methods.

[0026] The flexibility to replace inspection tools on the end effector is another key advantage. This allows the system to adapt to different structure types and inspection needs, detecting a range of defects like cracks, voids, and variations in material strength.

[0027] Throughout the following sections, we will describe the technical details of how each component of the system works. This includes the design of the UAV payload and end effector, the principles behind the defect detection tools and methods, the signalprocessing and machine learning techniques used to analyze inspection data, and the novel navigation and control approaches that enable autonomous contact-based inspection.

[0028] While the illustrative embodiments focus on specific configurations and techniques, the overall system is highly adaptable. The modular architecture allows for the integration of complementary technologies like computer vision and advanced sensor fusion, enabling even more comprehensive and autonomous infrastructure inspection solutions in the future.

[0029] Ultimately, this UAV-based contact inspection system represents significant progress in our ability to maintain and ensure the safety of critical civil infrastructure. By automating dangerous and time-consuming manual tasks, improving defect detection precision, and enabling more frequent and thorough inspections, this technology has the potential to significantly enhance public safety and optimize infrastructure maintenance operations.

[0030] System Architecture

[0031] Referring to Figure 1 , a payload for a UAV (I) which mounts to the UAV and has an end effector (E) with contact inspection technology on a boom (C) which can be positioned at various angles through a rotating boom system (F). This end effector also contains a wide platform and wheels (A). The end effector is also shock mounted(B). The inspection technology itself is comprised of an automated hammer system (D), as well as various sensors including acoustic and acceleration sensors (H) to allow detection of the defects. On the end effector there is also a surface marking system (K). There are time of flight (ToF) and visual sensors (J) as well as a GNSS system (G) to feedback into the UAV control system to allow for wall following and position tracking.

[0032] The illustrative embodiment allows a pilot to fly a UAV with contact inspection technology, and easily inspect walls, ceilings, and various other angles by attaching a custom payload to off the shelf drones.

[0033] The illustrative embodiment can take the form of a payload for a UAV, which mounts to the UAV and has an end effector with contact inspection technology on a boom which can be positioned at various angles to match the angles of different surfaces required to inspect. This end effector also contains a wide platform to allow forstable contact, and wheels to allow for ease of movement around the surface while maintaining consistent contact with the surface, which is important for various inspection devices. The wheels also ensure there is not a rigid connection to the drone for reaction force purposes. The end effector is also shock mounted to reduce reaction force purposes. The inspection technology itself is comprised of an automated hammer system, which excites the surface to detect delaminated concrete, as well as various sensors including acoustic and acceleration sensors to allow detection of the defects. There is a real-time on-board machine learning inference model that takes the live signals and infers whether a defect is present on the structure. On the end effector there is also a surface marking system, which can mark the location where a defect was last found. To assist in navigating around structures that may be at different angles, there is an automated wall-following UAV navigation system that use sensors such as time of flight and visual sensors to feedback into the UAV control system to automatically adjust the angle of the UAV and end effector to match the angle of the surface, and to allow to turn the 3d surface to be controlled with only 2 dimensions of control from the operator.

[0034] For the on-board machine learning (ML) model to detect delaminated concrete, thousands of acoustic sensor recordings of both delaminated and non-delaminated concrete were collected and labeled by an inspector manually collecting the data using conventional approaches such as a hammer. Different signal processes and calculated features that are indicative of different types of delamination were used to train according to whether the value for the feature was representative of delamination or non-delamination. This feature-based model is, therefore, a very lightweight model that can be easily computed without heavy hardware such as GPUs.[00351 Sensors

[0036] As discussed above, the illustrative embodiments of the invention employ sensors of various types. Here below is a more detailed description of the function and potential hardware implementations for each type of sensor:

[0037] Acoustic Sensors

[0038] Function: Acoustic sensors are used to detect sound waves generated when the inspection tools (e.g., automated hammer, toothed roller) excite the infrastructuresurface. The characteristics of these sound waves can indicate the presence of defects like delamination, voids, or cracks.Hardware Implementation:• Piezoelectric microphones: These convert sound pressure into electrical signals. They are rugged, compact, and can detect a wide frequency range, making them suitable for this application.• MEMS microphones: Micro-Electro-Mechanical Systems (MEMS) microphones are miniature, low-power, and can be easily integrated into the end effector.

[0039] Accelerometers

[0040] Function: Accelerometers measure the vibrations and mechanical shock waves propagating through the infrastructure when excited by the inspection tools. Changes in these vibration patterns can indicate the presence of subsurface defects.Hardware Implementation:• MEMS accelerometers: These are compact, low-power, and can measure accelerations in multiple axes. They are widely used in vibration monitoring applications.• Piezoelectric accelerometers: These offer high sensitivity and wide frequency response. They are robust and suitable for harsh environments.

[0041] Time-of-FHght (ToF) Sensors

[0042] Function: ToF sensors measure the distance between the end effector and the infrastructure surface by emitting a light pulse and measuring the time taken for the reflected pulse to return. This helps in maintaining a consistent contact force and alignment between the end effector and the surface.Hardware Implementation:• Laser-based ToF sensors: These use a laser diode to emit a light pulse and a photosensor to detect the reflected pulse. They offer high accuracy and fast response times.• LED-based ToF sensors: These use an LED light source instead of a laser. They are generally cheaper and safer than laser-based sensors but may have lower range and accuracy.

[0043] Visual Sensors

[0044] Function: Visual sensors, i.e., cameras, capture images of the infrastructure surface. These images can be used for navigation, alignment, and potentially for visual defect detection using computer vision techniques.Hardware Implementation:• Monocular cameras: A single camera can capture 2D images of the surface. This is the simplest and most cost-effective option.• Stereo cameras: Two cameras separated by a known distance can capture 3D information about the surface, which could be useful for navigation and alignment.• Time-of-Flight cameras: These are specialized cameras that capture depth information for each pixel in the image. They could be used for precise 3D mapping of the surface.

[0045] GNSS Receiver

[0046] Function: The Global Navigation Satellite System (GNSS) receiver determines the geospatial position of the UAV. This information is used to map the location of detected defects and to navigate the UAV to specific inspection locations.Hardware Implementation:• GPS receiver: The Global Positioning System (GPS) is the most widely used GNSS. GPS receivers are readily available and can provide position accuracy of a few meters.• Multi-GNSS receiver: These receivers can use signals from multiple GNSS constellations (e.g., GPS, GLONASS, Galileo) to improve position accuracy and reliability.

[0047] The specific choice of sensor hardware will depend on factors like the required performance, operating environment, size and weight constraints, power consumption,and cost. The sensor data would typically be interfaced to the UAV's onboard computer via standard protocols like I2C, SPI, or UART.

[0048] Wall-following Navigation

[0049] In one exemplary embodiment, the wall-following navigation system works as follows:1. The system uses sensors like time-of-flight (ToF) sensors and cameras to detect the position and orientation of the end effector relative to the structure surface.2. The sensors provide feedback to the UAV's control system about how far away and at what angle the end effector is from the surface.3. The control system then automatically adjusts the UAV's position and orientation to align the end effector more closely with the surface. This keeps the end effector in contact with the surface during inspection.4. As the UAV moves along the surface, the sensors continuously track the end effector's position and orientation relative to the surface.5. The control system makes corresponding adjustments to the UAV's position and orientation, causing it to "follow" the contours of the surface.6. The operator only needs to provide 2D controls to move the UAV along the surface. The system handles automatically orienting the end effector to match the changing 3D shape of the surface.7. For vertical or angled surfaces, the system controls the boom to position the end effector at the proper angle while also adjusting the UAV's orientation to match.

[0050] Obstacles and Surface Texture Changes

[0051] The system may be configured to handle obstacles and surface texture changes in the following ways:

[0052] Obstacles:• The sensors like cameras and ToF sensors would detect obstacles on the surface ahead of the end effector.• The control system would calculate a trajectory to avoid the obstacle while maintaining contact with the surface. This could involve pitching or yawing the UAV to move around the obstacle.• If the obstacle is too large to completely avoid, the system may prompt the operator to manually maneuver the UAV until the obstacle has passed.

[0053] Surface Texture Changes:• Changes in surface texture would likely cause changes in the sensor readings as the sensors detect differences in reflection, range, etc.• The control system's algorithms would be trained to ignore minor changes in sensor readings due to surface texture. This would help avoid excessive adjustments in response to texture changes.• For more significant texture changes that do impact the sensor readings, the control system would make appropriate adjustments to the UAV's position and orientation to realign the end effector.• The system may use internal models of common types of surface textures to further distinguish "true" changes in surface alignment from differences due to texture.• If the texture change causes sensor readings to become unreliable, the system may fall back to semi-autonomous or manual mode until a consistent surface is redetected.

[0054] In summary, the system would use its sensors, control algorithms, and internal models to distinguish obstacles, true changes in surface alignment, and simple texture differences. It would then make the appropriate adjustments or fallback to less autonomous operation to ensure the end effector remains in contact with the surface during inspection.

[0055] Machine Learning Models

[0056] We will now discuss in greater detail the ML models that could be used in the contact-based infrastructure inspection system.

[0057] The main machine learning model would be used to detect defects based on the acoustic sensor data collected during inspection. This could be implemented as a supervised learning model that is initially trained using data collected from conventional inspection methods. The sensor data would be labeled as "defect" or "no defect" based on what the inspector found.

[0058] One option is a feature-based model where relevant features are first extracted from the raw sensor data. These could include things like:• Amplitude of high-frequency components• Amplitude of low-frequency components• Root mean squared value• Spectral centroid• Mel frequency cepstral coefficients (Mel frequency cepstral coefficients are a type of feature that can be extracted from audio data. They are designed to match the way humans perceive sound and are commonly used for speech recognition and musical instrument identification.)

[0059] Only a small number of relevant features would likely be needed to detect major types of defects. The features would then be used to train a relatively simple classifier like logistic regression or a decision tree. Since only a few features are used as input, the model would be lightweight and fast enough to run on the UAV's onboard computer in real-time.

[0060] An alternative is to use a model like a convolutional neural network that can learn relevant features directly from the raw sensor data. However, this would require more training data and computational resources.

[0061] Once deployed, the model could be updated continuously using data collected during UAV inspections. This would allow the model to adapt to new contexts and refine what it considers a defect.

[0062] Reinforcement learning could also be used to further optimize aspects of the inspection procedure like surface excitation, end effector positioning and speed ofmovement. Data from successful defect detections could reward the UAV's control system, training it to perform the inspection more effectively over time.

[0063] It is possible for the ML model to also consider location (and signals from locations nearby), or a specific set of data collected in an immediately recent period, when classifying. This effectively mimics what a delamination inspector does by allowing local comparison to determine defects, not just a global one. The algorithm would be more accurate on a wide variety of concrete structures that can sound different due to varying vibrational modes based on their shape and composition.

[0064] In summary, a range of ML models, from simple feature-based classifiers to more complex deep learning approaches, could be suitable for defect detection in this system. The key is to match the computational complexity of the model to the available resources while still achieving adequate performance. Continuous updating and reinforcement learning further improve the system after initial deployment.

[0065] Benefits of Real-time Detection

[0066] There are several benefits to being able to detect defects in real-time during infrastructure inspection using UAVs:1. Increased Coverage - Real-time detection allows the UAV to immediately mark locations with defects and adjust its path to fully inspect areas of interest. This improves the coverage and completeness of the inspection.2. Interactive Inspection - The inspector can see defects as they are detected and respond accordingly. The inspector can direct the UAV to reinspect areas, collect additional data, or apply other inspection tools based on the real-time results.3. Increased Safety - By detecting and marking defects as it progresses, the UAV system reduces the need for follow-up inspections. This decreases the total time workers have to spend accessing dangerous infrastructure.4. Higher Precision - Real-time defect detection using machine learning can achieve higher precision than traditional inspection tools by considering a variety of data like images, acceleration signals and acoustic readings.5. Improved Decision Making - Inspectors can make more informed maintenance and repair decisions based on a complete and accurate real-time assessment of defects and their locations.6. Continual Improvement of Models - The machine learning models used for detection can be continually updated and improved during inspection based on new data and confirmed or corrected results from the inspector. The inspector is skilled in the conventional "hammer test", thus can provide the ground truth by simply listening to the acoustic sound at remote location in real-time.7. Detection of Early Warning Signs - Real-time detection allows the system to potentially identify subtle signs of defects or degradation as they initially develop. This enables earlier intervention before defects worsen.

[0067] Alternative Embodiments

[0068] The system that allows for the angle adjustment of the end effector could be different depending on the type of UAV and surface angle requirements. Various examples are below:The stator and rotor rings that encircle the UAV with a gimbal motor that allows for angle adjustment. end effectors that are spring mounted that allow for limited different conforming angles.

[0069] The illustrative embodiment currently has delamination detection tools on it; however, these could be swapped out for other tools such as: compressive strength detection tools; crack depth detection tools; void detection tools.

[0070] The UAV end effector currently uses castor wheels and a shock, but alternatives could be: all direction rolling ball-style castor wheels, flexible springy rods instead of a shock.

[0071] Contact Inspection Plus Computer Vision

[0072] Combining this contact-based inspection system with computer vision techniques on the UAV could provide several improvements:1. Increased Precision - Computer vision models could detect surface defects based on image features with high precision. Used in conjunction with contact inspection tools, this could provide additional confirmation of defects and rule out false positives.2. Detection of Early Signs - Computer vision has the potential to detect very early signs of deterioration, like cracking, discoloration and surface unevenness, even before they are detectable using contact methods. This enables truly proactive inspections.3. Automated Marking - An automated computer vision system could precisely mark the locations of defects on the surface without requiring human input. This simplifies and speeds up the inspection process.4. Complete Mapping - Images captured by the UAV could be used to generate a complete 3D textured map of the inspected surface. This mapped model could then be analyzed on the ground for a more thorough defect assessment.5. Automated Report Generation - Automatic detection of defects in images combined with location data could generate a report of defect locations, sizes, severity classifications, etc. with little to no human input required.6. Adaptive Path Planning - By detecting defects in real time from images, the UAV could automatically adjust its flight path and end effector position to focus on areas of interest and avoid missing any defects.7. Redundancy and Robustness - Combining contact and computer vision approaches provides redundancy, helping to ensure defects are not missed if one method fails or produces unreliable results for a period of time.

[0073] Toothed Roller

[0074] It is possible for the concrete to be excited in various ways that produce signals indicative of delaminated or non-delaminated concrete. One is a way in which the concrete exciting system is dragged along a surface by the UAV, and the kinetic energy of the UAV movement provides the energy for the excitation. This could mean chain links, metal spheres, or other metal shapes getting dragged or pushed along thesurface. In this case the UAV induced velocity plus the surface irregularities cause the metal to lift from the surface and either gravity or a spring, or a combination of both, would return them to and excite the surface.

[0075] Referring to Figure 4, this could also mean a rolling system where the pulling or pushing causes a torsion or other spring to charge, and at certain instances the spring will be released, and a striker attached to the spring excites the surface. This could also mean a rolling system where the pulling or pushing of irregular shaped wheels are rolled over the surface, and the irregularities strike the surface as it rolls and excites the surface. It could include multiple of or a combination of the above embodiments.

[0076] The toothed roller depicted in Figure 4 is not novel. It is currently used for manual inspection of structures. However, this particular tool is an example of what could be pushed / pulled by the UAV to strike the surface. A novel way of exciting the concrete surface to detect delamination could work as follows.

[0077] The toothed roller includes a wheel with teeth or protrusions around its circumference (see Figure 4). As the roller rotates and moves along the concrete surface due to traction from the UAV, the teeth encounter surface irregularities. This causes the teeth to temporarily lose contact with the surface as they roll over the irregularity. The teeth are mounted on springs that compress as they lose contact.

[0078] Once the tooth has passed the irregularity, the spring releases and drives the tooth back into contact with the surface with force. This impact between the tooth and the concrete surface generates acoustic signals and vibrations that the onboard sensors can detect.

[0079] The signals generated by impacts on delaminated concrete will differ from those of intact concrete. This allows the machine learning model to differentiate between delaminated and non-delaminated areas based on the toothed roller excitation.

[0080] The rotation speed of the toothed roller and the distance between teeth can be optimized based on the size of expected delamination and the acoustic response desired. A higher rotation speed and smaller tooth spacing would generate more impacts per unit of time, exciting the surface more. In summary, the toothed roller provides a robust and simple method of exciting the concrete surface as the end effectormoves. The rolling motion and spring-loaded teeth generate impulsive impacts that produce audible and vibration-based signals, allowing delamination to be detected.

[0081] Figure 5 shows spectrograms of the acoustic signals generated by rolling a toothed roller over control (non-delaminated) and delaminated concrete surfaces. The x- axis represents time, the y-axis represents frequency, and the color intensity represents the amplitude of the acoustic signal at each time-frequency point.

[0082] On the control surface (left), the spectrogram shows a relatively consistent pattern of brief, broad-spectrum pulses. These correspond to the individual impacts of the roller's teeth on the intact concrete. The pulses are well-separated in time, indicating distinct impacts as each tooth strikes the surface.

[0083] In contrast, the spectrogram for the delaminated surface (right) shows more prolonged and overlapping broad-spectrum signals. This suggests that the impacts from the teeth are exciting resonant vibrations in the delaminated layer, leading to sustained acoustic emissions. The higher amplitudes and "smeared" time-frequency pattern are indicative of the altered acoustic response of delaminated concrete.

[0084] These pronounced differences in the time-frequency characteristics of the acoustic signals allow the ML model to reliably distinguish between intact and delaminated areas during rolling inspections with the toothed tool.

[0085] Example Use Case: Dam Inspection

[0086] Consider the inspection of a large concrete dam using the disclosed UAV-based contact inspection system. The dam has expansive vertical and angled surfaces that need to be checked for defects like delamination, cracks, and degradation that could compromise its structural integrity.

[0087] An inspector arrives at the dam site with a UAV equipped with the specialized contact inspection payload. After conducting pre-flight checks and setting up the control station, the inspector launches the UAV.

[0088] Using the onboard cameras and navigation sensors, the UAV flies to the face of the dam and positions the contact inspection end effector against the concrete surface. The wall-following algorithms engage, ensuring the end effector maintains proper contact as the UAV begins to move along the dam face.

[0089] As the UAV progresses, the toothed roller on the end effector excites the concrete surface. The onboard accelerometers and acoustic sensors capture the response vibrations and sounds. This data is instantly analyzed by the machine learning model, which compares the signals to its trained understanding of how intact and defective concrete should respond.

[0090] At one point, the model detects a signal anomaly indicative of subsurface delamination. The UAV immediately marks the suspect location with spray paint and logs the GPS coordinates. The inspector, monitoring the real-time data feed, can see the defect detection and choose to have the UAV perform additional inspection passes in that area to confirm and map the extent of the damage.

[0091] The UAV continues its inspection, navigating over various angled surfaces and around obstacles protruding from the dam face. It adapts its path to concentrate on areas where the ML model is detecting signs of potential defects. All the while, it is building a comprehensive 3D map of the dam surface, with suspected defect locations flagged.

[0092] After completing the programmed inspection route, the UAV returns to the launch point. The inspector can then review the mapped defect data, crosscheck it against the visual feed, and generate a report for the dam maintenance team.

[0093] What might have taken a team of inspectors weeks to examine with manual methods, the UAV completed in a matter of hours. The inspection was conducted without putting any human workers at risk and yielded a highly detailed defect map based on objective, quantitative data. Armed with this information, the dam managers can now prioritize repairs and be confident in the integrity of this critical infrastructure.

[0094] This use case illustrates how the UAV-based contact inspection system can significantly improve the safety, efficiency, and effectiveness of structural inspections for large, complex infrastructure assets like dams. The integration of UAV mobility, contactbased sensing, real-time data analysis, and autonomous defect detection capabilities into one system represents a major advance over traditional inspection methods.

[0095] The UAV-based contact inspection system can be adapted for various types of infrastructure inspections beyond dams. The modular nature of the system, especiallythe interchangeable end effector tools, allows it to be configured for different structures and inspection needs. Here are a few examples:1. Bridge InspectionBridges, especially large ones, have many structural elements that require regular inspection, such as decks, girders, cables, and towers. The UAV system could be used to detect defects like delamination, corrosion, and cracking in these elements. The ability to access high and difficult-to-reach areas would be particularly valuable.2. Building InspectionThe system could inspect the walls, roofs, and foundations of large buildings for signs of degradation, water damage, or structural issues. This could be especially useful for tall structures where access is challenging or for rapid post-disaster assessments to determine if buildings are safe to occupy.3. Nuclear Power Plant InspectionNuclear power plants require frequent, thorough inspections to ensure safety, but many areas are hazardous for human inspectors to access. The UAV system could be adapted with radiation-hardened components to inspect reactor containment structures, cooling towers, and other critical infrastructure.4. Wind Turbine InspectionWind turbine blades and towers can develop defects like delamination, cracking, and erosion over time. The UAV system could efficiently inspect these tall structures, detecting and mapping damage to guide repairs and preventative maintenance.5. Tunnel InspectionTunnels, whether for transportation, water, or utilities, require regular inspection to ensure structural integrity and safety. The UAV system could navigate through tunnels, using its contact sensors to detect defects in the walls and ceiling that could indicate larger structural issues.6. Pipeline InspectionAbove-ground pipelines could be inspected for corrosion, leaks, and structural damageusing the UAV system. The ability to cover long distances and navigate various terrains would make this an efficient alternative to manual inspections.

[0096] In each of these cases, the core capabilities of the system - UAV mobility, contact-based sensing, real-time defect detection, and autonomous navigation - would provide significant benefits over traditional inspection methods. The specific sensors, end effector tools, and machine learning models could be optimized for each application to detect the most relevant types of defects. Moreover, the data collected by the system could be integrated with other asset management and predictive maintenance systems to provide a comprehensive view of the infrastructure's health and to guide data-driven maintenance decisions.

[0097] Therefore, while the dam inspection use case illustrates the core capabilities of the system, the potential applications extend to a wide range of critical infrastructure sectors. As UAV and sensing technologies continue to advance, the adaptability and impact of this system will only grow.

[0098] Conclusion

[0099] The disclosed contact-based inspection system provides a novel and advantageous approach for using UAVs to perform detailed infrastructure assessments. By equipping a UAV with a specialized payload featuring an actuated end effector, a variety of contact-based inspection tools can be deployed on structures at various angles.

[0100] The illustrative embodiments enable precise defect detection using techniques such as automated hammer sounding, rolling impact excitation, and collection of acoustic and vibration data. Onboard machine learning models analyze this sensor data in real-time to identify and localize areas of damage like delamination, cracks and voids with high precision.

[0101] Importantly, the system includes innovative navigation capabilities that allow the end effector to maintain contact with the structure surface, even as it changes orientation. This, combined with the ability to carry interchangeable tools, allows detailed inspections to be performed on walls, ceilings, and other angled surfaces that are difficult or dangerous to access manually.

[0102] Overall, this UAV-based approach greatly improves the safety, coverage, frequency and precision of infrastructure inspections compared to conventional methods. The system can be further enhanced through the addition of complementary technologies such as computer vision, allowing for truly comprehensive structural assessments.

[0103] While specific embodiments have been described, the disclosed invention is not limited to these exact configurations. Various modifications to the payload design, inspection tools, excitation methods, navigation approach and data analysis techniques are possible without departing from the scope of the invention as claimed.

Claims

CLAIMSWe claim:1 . A contact-based inspection system for evaluating the structural health of civil infrastructure, comprising: an unmanned aerial vehicle (UAV); a payload attached to the UAV, the payload comprising: a boom; a modular end effector housing one or more inspection tools and a sensor suite, the end effector further including: a spring-loaded striker or toothed roller for generating diagnostic acoustic signals; accelerometers and microphones for capturing the acoustic response; and a surface marking system; a real-time data analysis system, including an onboard machine learning model trained to detect and localize defects in the structure based on the captured acoustic response; and auxiliary sensors for UAV navigation and spatial mapping; wherein the system is configured to trigger the surface marking system automatically, in real-time, when the machine learning model and sensing systems detect a defect in the structure being inspected.

2. The system of claim 1 , wherein the one or more inspection tools include an automated hammer (D) for exciting the surface to generate acoustic signals indicative of infrastructure defects.

3. The system of claim 1 , wherein the one or more inspection tools include a toothed roller for exciting the surface to generate acoustic signals indicative of infrastructure defects.

4. The system of claim 1 , wherein the one or more inspection tools include one or more acoustic sensors (H) for detecting acoustic signals generated by exciting the surface.

5. The system of claim 1 , wherein the one or more inspection tools include one or more accelerometers (H) for detecting vibrations generated by exciting the surface.

6. The system of claim 1 , wherein the one or more sensors include one or more time-of-flight (ToF) sensors (J).

7. The system of claim 1 , wherein the one or more sensors include one or more visual sensors (J).

8. The system of claim 1 , wherein the real-time data analysis system comprises an onboard processor configured to analyze sensor data from the one or more inspection tools to detect and localize infrastructure defects in real-time.

9. The system of claim 8, wherein the onboard processor is configured to implement a machine learning model trained to classify sensor data as indicating either presence or absence of infrastructure defects.

10. The system of claim 1 , further comprising a transceiver for transmitting sensor data or defect detection results to a remote system.11 . The system of claim 1 , further comprising a global navigation satellite system (GNSS) receiver (G) for determining a geospatial position of the UAV.

12. A method for inspecting civil infrastructure using a contact-based inspection system, the method comprising: equipping an unmanned aerial vehicle (UAV) with a payload comprising a boom and a modular end effector housing one or more inspection tools and a sensor suite; flying the UAV to a target inspection area of the civil infrastructure; performing contact-based inspection including using the end effector to generate diagnostic acoustic signals on the structure surface and capturing the acoustic response; analyzing the captured acoustic response in real time using an onboard machine learning model trained to detect and localize defects in the structure; navigating the UAV and maintaining contact with the structure surface using auxiliary sensors; and triggering a surface marking system on the end effector automatically, in real time, when the machine learning model and sensing systems detect a defect in the13. The method of claim 12, wherein the one or more inspection tools include an automated hammer, and wherein said contact-based inspection comprises actuating the automated hammer to impact the surface.

14. The method of claim 12, wherein the one or more inspection tools include a toothed roller, and wherein said contact-based inspection comprises rolling the toothed roller along the surface.

15. The method of claim 12, wherein capturing the acoustic response comprises collecting acoustic signals using one or more acoustic sensors.

16. The method of claim 12, wherein capturing the acoustic response comprises collecting vibration signals using one or more accelerometers.

17. The method of claim 12, wherein analyzing the captured acoustic response comprises detecting and localizing infrastructure defects using the machine learning model implemented by an onboard processor.

18. The method of claim 17, further comprising transmitting the collected sensor data to a remote system for analysis.

19. The method of claim 12, further comprising physically marking detected infrastructure defects using a marking system on the end effector.

20. The method of claim 12, wherein navigating the UAV to move the end effector along the surface comprises: determining a position and orientation of the end effector relative to the surface using one or more sensors; and adjusting a position and orientation of the UAV based on the determined position and orientation of the end effector to maintain a consistent contact force between the end effector and the surface.21 . The method of claim 20, wherein the one or more sensors include one or more time-of-flight (ToF) sensors.

22. The method of claim 20, wherein the one or more sensors include one or more visual sensors.

23. The method of claim 12, further comprising generating a map of the infrastructure surface indicating locations of detected defects.

24. The method of claim 12, further comprising adjusting one or more operating parameters of the UAV or the one or more inspection tools based on the detected and localized infrastructure defects.

Images