Unmanned aerial vehicle intelligent inspection and digital management system for road and bridge facilities

Abstract

The invention provides an unmanned aerial vehicle intelligent inspection and digital management system for road and bridge facilities. The unmanned aerial vehicle intelligent inspection and digital management system comprises a cloud digital twin platform, distributed edge computing nodes and an unmanned aerial vehicle cluster capable of being dispatched and executed. The cloud digital twin platform is used for constructing and maintaining a high-precision three-dimensional semantic model corresponding to physical road and bridge facilities, and the model fuses geological and geographic information, historical operation and maintenance data and real-time sensor data. Through intelligent collaborative operation and edge real-time data processing of the unmanned aerial vehicle cluster, qualitative leap of inspection efficiency is achieved, labor cost and safety risks are greatly reduced, physical facilities are holographic digitalized, management decisions are made to be based on comprehensive, visual and dynamic visual data, and the inspection efficiency of the unmanned aerial vehicle cluster is improved. According to the method, the accuracy and scientificity of operation and maintenance management are remarkably improved, the evolution trend prediction of the health condition of the facility is realized through deep data mining and simulation analysis, a key basis is provided for accurately formulating a maintenance strategy, and the service life of the facility is effectively prolonged.

Description

Technical Field

[0001] This invention relates to the field of building construction, and in particular to an intelligent unmanned aerial vehicle (UAV) inspection and digital management system for road and bridge facilities. Background Technology

[0002] Traditional road and bridge facility inspection and maintenance work mainly relies on manual visual inspection or measurement with simple instruments. This method is not only inefficient, time-consuming, and costly, but also suffers from a series of problems such as strong subjectivity, high risk, and difficulty in discovering hidden defects. Although drone technology has been introduced into this field for data collection in recent years, most applications are still in the initial stage of replacing manual photography. The massive amount of raw data generated by drones still requires manual interpretation in the background, resulting in low levels of intelligence, long information processing cycles, and disconnect between collection, processing, and analysis, forming data silos. In addition, existing management methods are mostly passive responses or preventive maintenance based on fixed cycles, lacking the ability to predict the development trend of facility status. Operation and maintenance decisions lack scientific data support, making it impossible to shift from regular maintenance to on-demand maintenance. This can lead to safety accidents due to untimely maintenance and waste of resources due to over-maintenance. At the same time, the data credibility and traceability of the entire operation and maintenance process are insufficient, making it difficult to meet the higher requirements of refined management and digital asset operation.

[0003] Therefore, in order to address the above issues, we are now developing an intelligent unmanned aerial vehicle (UAV) inspection and digital management system for road and bridge facilities. Summary of the Invention

[0004] To overcome the shortcomings of existing technologies, this invention provides an intelligent unmanned aerial vehicle (UAV) inspection and digital management system for road and bridge facilities.

[0005] The technical solution of this invention is: an intelligent inspection and digital management system for road and bridge facilities using unmanned aerial vehicles (UAVs), comprising: a cloud-based digital twin platform, distributed edge computing nodes, and a UAV cluster that can be scheduled and executed; the cloud-based digital twin platform is used to construct and maintain a high-precision three-dimensional semantic model corresponding to the physical road and bridge facilities, which integrates geological and geographical information, historical operation and maintenance data, and real-time sensor data; the distributed edge computing nodes are deployed around key facilities and have data caching and computing capabilities; the innovation of the system lies in the setting of a task intelligent parsing and resource dynamic orchestration module. This module responds to inspection task requests, performs task simulation on the digital twin platform, generates an optimized virtual formation scheme including UAV model, payload configuration, flight path, and data fusion strategy, and based on this scheme, schedules physical UAVs from the UAV resource pool associated with the edge computing nodes. Before task execution, a task-customized lightweight AI model and processing algorithm are injected into the UAV's onboard computing unit through the edge nodes, making the UAV an intelligent agent with specific task processing capabilities.

[0006] As a preferred embodiment of the present invention, the UAV onboard computing unit integrates a multimodal data streaming processing and real-time damage feature extraction module. This module is configured to perform online processing of raw data collected by onboard optical cameras, lidar, infrared thermal imagers, and multispectral sensors, performing operations including image pixel-level segmentation based on deep learning networks, point cloud 3D reconstruction, and hot spot anomaly detection, and outputting a structured damage feature data packet. This data packet at least includes damage type, geometric parameters, spatial coordinates, timestamp, and confidence level. The damage feature data packet is preferentially transmitted to nearby edge computing nodes for preliminary aggregation and verification, and only the aggregated summary information is synchronized to the cloud-based digital twin platform.

[0007] As a preferred embodiment of the present invention, the cloud-based digital twin platform further includes a facility status prediction and risk assessment engine; the engine couples a physical mechanism model and a data-driven model, receives the damage feature data package and external environmental data, and performs at least one of the following prediction simulations: crack propagation prediction based on fracture mechanics, structural bearing capacity impact analysis based on finite element method, material performance degradation trend prediction based on machine learning, and local scour evolution simulation of bridge piers based on computational fluid dynamics; the platform visualizes the prediction results and overlays them onto a three-dimensional model, and dynamically adjusts the priority and inspection strategy of subsequent inspection plans according to the predicted risk level.

[0008] As a preferred embodiment of the present invention, the system integrates a blockchain-based operation and maintenance data storage and audit traceability module; this module records the hash values, timestamps and responsible party identifiers of the key information of the damage feature data package, prediction simulation results, risk warning reports and maintenance work orders in a distributed ledger, which is jointly maintained by multiple authorized nodes to form an immutable operation and maintenance full life cycle audit chain.

[0009] As a preferred embodiment of the present invention, the system further includes an augmented reality operation and maintenance auxiliary interactive terminal; the terminal is configured to locate physical facilities through image recognition when maintenance personnel are working on-site, and retrieve and overlay virtual information related to the work location from the cloud digital twin platform, including but not limited to damage markings, historical data, prediction results, maintenance guidance animations and material lists; it supports maintenance personnel to input on-site verification data and maintenance feedback through a human-computer interaction interface, and update the digital twin model in real time.

[0010] As a preferred embodiment of the present invention, the UAV swarm possesses self-organizing and collaborative capabilities based on distributed consensus; each UAV within the swarm shares status and perception information through an inter-UAV communication network and follows preset collective intelligence behavior rules to achieve autonomous obstacle avoidance, formation maintenance, dynamic task allocation, and focused collaborative reconnaissance of key areas in complex environments, without the need for full intervention from a central controller.

[0011] As a preferred embodiment of the present invention, the system is equipped with an inspection strategy optimization engine based on reinforcement learning. The engine takes minimizing long-term operation and maintenance costs and risks as the objective function, and the inspection strategy parameters as the action space. By continuously collecting feedback data on task execution effects, it autonomously learns and dynamically optimizes the drone scheduling, sensor configuration, flight path and AI model selection strategies, thereby realizing the autonomous evolution of the system's inspection efficiency.

[0012] As a preferred embodiment of the present invention, the system architecture extension includes a decentralized drone resource sharing platform; this platform manages compliant drone resources registered by third parties through smart contracts, automatically inquires about prices and schedules external resources to participate in inspections according to the contract terms when a task requirement arises, and realizes the confirmation of ownership of task data and automatic settlement of service fees based on blockchain technology.

[0013] By adopting the above technical solution, the present invention has the following advantages: 1. This invention achieves a qualitative leap in inspection efficiency through intelligent collaborative operation of drone swarms and real-time edge data processing, significantly reducing labor costs and safety risks. It holographically digitizes physical facilities, enabling management decisions to be based on comprehensive, intuitive, and dynamic visualized data, significantly improving the accuracy and scientific nature of operation and maintenance management. Through in-depth data mining and simulation analysis, it enables the prediction of the evolution trend of facility health status, providing a key basis for accurately formulating maintenance strategies, effectively extending the service life of facilities, and ensuring operational safety.

[0014] 2. By constructing a trusted data chain covering the entire process of inspection, analysis, decision-making, execution, and verification, this invention not only ensures the quality and traceability of the operation and maintenance process itself, but also provides a solid data foundation for the financialization, actuarial science, and sustainable development assessment of facility assets, thus creating a new paradigm for infrastructure operation and maintenance management. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the structure of the present invention. Detailed Implementation

[0016] References to embodiments herein mean that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0017] Intelligent inspection and digital management system for road and bridge facilities using drones, such as Figure 1As shown, the system includes a cloud-based digital twin platform, distributed edge computing nodes, and a swarm of drones that can be scheduled and executed. The cloud-based digital twin platform is used to build and maintain a high-precision 3D semantic model corresponding to the physical road and bridge facilities. This model integrates geological and geographical information, historical operation and maintenance data, and real-time sensor data. The distributed edge computing nodes are deployed around key facilities and have data caching and computing capabilities. The system's innovation lies in the inclusion of a task intelligent parsing and dynamic resource orchestration module. This module responds to inspection task requests, performs task simulation on the digital twin platform, and generates an optimized virtual formation scheme that includes drone models, payload configurations, flight paths, and data fusion strategies. Based on this scheme, the system integrates data with the edge computing nodes. Physical drones are scheduled from a drone resource pool associated with computing nodes. Before mission execution, a lightweight AI model and processing algorithm customized for the mission are injected into the drone's onboard computing unit via edge nodes, enabling the drone to become an intelligent agent with specific mission processing capabilities. The drone's onboard computing unit integrates a multimodal data streaming processing and real-time damage feature extraction module. This module is configured to perform online processing of raw data collected by onboard optical cameras, LiDAR, infrared thermal imagers, and multispectral sensors, performing operations including image pixel-level segmentation based on deep learning networks, point cloud 3D reconstruction, and hot spot anomaly detection, and outputting a structured damage feature data package, which at least includes damage type, geometric parameters, and spatial coordinates. The damage feature data packet is first transmitted to the nearest edge computing node for preliminary aggregation and verification. Only the aggregated summary information is synchronized to the cloud digital twin platform. The cloud digital twin platform further includes a facility status prediction and risk assessment engine. This engine couples the physical mechanism model and the data-driven model, receives the damage feature data packet and external environmental data, and performs at least one of the following prediction simulations: crack propagation prediction based on fracture mechanics, structural bearing capacity impact analysis based on finite element method, material property degradation trend prediction based on machine learning, and local scour evolution simulation of bridge piers based on computational fluid dynamics. The platform visualizes the prediction results and overlays them on the three-dimensional model, and dynamically adjusts the risk level according to the prediction. The system integrates a blockchain-based operation and maintenance data storage and audit traceability module to prioritize and implement subsequent inspection plans and strategies. This module records the hash values, timestamps, and responsible party identifiers of key information from damage feature data packages, prediction simulation results, risk warning reports, and maintenance work orders in a distributed ledger, which is jointly maintained by multiple authorized nodes to form an immutable operation and maintenance lifecycle audit chain. The system also includes an augmented reality operation and maintenance auxiliary interactive terminal. This terminal is configured to locate physical facilities through image recognition when maintenance personnel are working on-site, and retrieve and overlay virtual information related to the work location from the cloud digital twin platform, including but not limited to damage markings, historical data, prediction results, maintenance guidance animations, and material lists.The system supports maintenance personnel to input on-site inspection data and maintenance feedback through a human-machine interface, updating the digital twin model in real time. The drone swarm possesses self-organizing and collaborative capabilities based on distributed consensus. Within the swarm, drones share status and perception information through an inter-drone communication network and follow preset collective intelligence behavior rules, enabling autonomous obstacle avoidance, formation maintenance, dynamic task allocation, and focused collaborative reconnaissance of key areas in complex environments, without the need for full intervention from a central controller. The system includes a reinforcement learning-based inspection strategy optimization engine. This engine aims to minimize long-term operation and maintenance costs and risks, using inspection strategy parameters as its action space. By continuously collecting task execution feedback data, it autonomously learns and dynamically optimizes drone scheduling, sensor configuration, flight paths, and AI model selection strategies, achieving autonomous evolution of system inspection efficiency. The system architecture also includes a decentralized drone resource sharing platform. This platform manages compliant drone resources registered by third parties through smart contracts. When a task requirement arises, it automatically inquires about and schedules external resources to participate in the inspection according to contract terms, and uses blockchain technology to confirm the ownership of task data and automatically settle service fees.

[0018] Specifically, the work cycle begins at the cloud-based digital twin platform, the highest decision-making center. This platform is not a static 3D display model, but a dynamic simulation and deduction environment that deeply integrates physical laws, historical operational big data, real-time environmental information, and artificial intelligence algorithms. It constructs a high-precision 3D model with rich semantic information that completely corresponds to the physical bridge and road facilities. This model not only includes geometric shapes but also integrates material properties, design loads, historical stress states, and a cumulative damage database, forming a digital mirror. When a preset inspection plan is triggered, a manual instruction is issued, or a potential risk is predicted by the platform's embedded predictive risk assessment engine based on physical mechanisms and data-driven approaches, a specific work task is activated. The task intelligent analysis and resource dynamic orchestration module in the digital twin platform begins operation. It performs a full-process simulation and optimization of the upcoming task in virtual space. This simulation comprehensively considers the structural characteristics of the facility, known weaknesses, future weather forecasts, real-time traffic load models, and historical inspection data. Through complex optimization algorithms, it generates an optimal virtual execution plan. This plan precisely specifies the composition of the drone swarm required to execute the task, including the drone model and quantity, the specific payload combinations such as optical cameras, lidar, infrared thermal imagers, or multispectral sensors on each drone, the optimal flight path trajectory, flight attitude, and the communication topology and data fusion strategy between individuals within the swarm. This allows for precise analysis and optimization before the task begins. After a perfect rehearsal in the digital world, the optimized scheme, which has been virtually verified, is then distributed to the corresponding distributed edge computing nodes in the target area. These nodes act as regional nerve centers connecting cloud intelligence and physical execution terminals. Based on the received virtual formation scheme, the edge nodes perform real-time matching and scheduling from their managed drone resource pool. This resource pool can be the system's own drones or certified third-party drone resources accessed through a decentralized sharing economy platform. The scheduling process ensures a high degree of consistency between physical resources and the virtual scheme. The edge computing nodes transmit the task-customized, optimized lightweight AI recognition model, real-time path planning algorithm, and dedicated data stream via high-speed wireless communication. The processing logic is dynamically injected into the onboard computing unit of each drone. This process empowers standardized drone hardware before mission execution, transforming it into a highly customized intelligent agent with autonomous perception and edge computing capabilities, thus achieving a leap from general-purpose tools to dedicated intelligent agents. After the drone swarm takes off according to instructions, the system enters the collaborative perception and data acquisition phase. The swarm's working mode is not a rigid, preset flight path, but rather exhibits self-organization and adaptive capabilities based on swarm intelligence. Within the swarm, each drone shares its position, status, remaining battery power, and preliminary local environmental information in real time through self-organizing network communication links, and follows collaborative rules designed according to biomimetic principles.Dynamically maintaining formation, avoiding conflict, redistributing tasks, and conducting focused collaborative reconnaissance of unexpected situations or key suspicious areas are all possible. For example, when a drone identifies suspected major damage through its onboard sensors, it immediately signals other members of the swarm, attracting multiple drones to conduct a three-dimensional, multi-scale joint detailed survey of the area from different angles. The swarm as a whole adaptively adjusts its inspection plan to compensate for any coverage gaps that may result from resource allocation. This global intelligence emerging from local interactions ensures the robustness and efficiency of the inspection mission in complex and ever-changing real-world environments. During the perception process, the multimodal sensors on the drones simultaneously and continuously acquire high-resolution optical images, high-precision laser point clouds, temperature distribution thermal images, and multispectral data in specific bands. These raw data streams do not wait to be sent back to the backend for processing. Instead, they are immediately processed on the UAV's onboard computing unit by a built-in multimodal data stream processing and real-time damage feature extraction module. This module runs a lightweight AI model deployed from edge nodes, capable of pixel-level image segmentation to identify cracks and spalling, 3D reconstruction of point clouds to quantify deformation and defects, and thermal image temperature difference analysis to detect hidden internal defects. It also outputs in real-time structured, standardized damage event data packets with precise geographic coordinates, timestamps, damage types, geometric parameters (such as length, width, and area), and confidence indices. This ability to convert data into information at the data source fundamentally solves the bottleneck problems of high latency and high bandwidth pressure caused by the massive data transmission of raw data in traditional inspection methods.

[0019] These high-value "damage event" data packets, extracted in real time, are prioritized and sent to the nearest edge computing node via a low-latency communication link. This node is not merely a relay station, but possesses powerful data aggregation and preliminary fusion capabilities. It performs temporal and spatial alignment, cross-validation, and information complementarity on data from multiple drones and sensors within the cluster. For example, it fuses crack information identified by optical images with depth information measured by laser point clouds to generate a more accurate 3D damage model, thus forming a regional and more reliable preliminary comprehensive diagnostic report. After completing data aggregation, the edge node synchronously uploads the refined summary information, key evidence data, and fusion results to the cloud-based digital twin platform. Upon receiving this fresh field data, the digital twin platform immediately... Entering the deep cognitive and decision support stage, its built-in facility status prediction and risk assessment engine begins to operate at full capacity. This engine couples physical law-based mechanical simulation models (such as finite element analysis, fracture mechanics, and computational fluid dynamics) with machine learning-based data-driven models. It uses newly discovered damage as initial conditions to simulate the evolution trajectory of the damage under the coupled effects of multiple factors such as future traffic loads, changes in ambient temperature, and wind and rain erosion. It predicts the rate of expansion and the degree of impact on the overall structural safety, quantifying the risk level at different future points in time. All these prediction results are visually integrated into a high-precision 3D model in a four-dimensional (space + time) manner, allowing managers to intuitively see the facility's "current state" and its "future evolution trend." Based on the prediction results, the system automatically generates graded early warning reports and forward-looking maintenance decision recommendations (such as "recommend pressure grouting treatment for cracks within 90 days").

[0020] Meanwhile, to ensure the traceability and credibility of the entire operation and maintenance process, the blockchain-based operation and maintenance data storage and audit traceability module is automatically activated. It encrypts and records key data hash values, operation timestamps, and responsible parties from the entire decision-making chain, from damage identification and prediction simulation to decision recommendations, onto a distributed ledger maintained by multiple parties. This forms an immutable and traceable audit trail. When decisions need to be implemented in the physical world for maintenance intervention, the augmented reality operation and maintenance auxiliary interactive terminal acts as a bridge. On-site maintenance personnel scan actual facilities using AR devices, and the system uses spatial positioning technology to accurately overlay virtual information from the digital twin platform (such as precise damage location, 3D visualization models, maintenance step animations, and safety instructions) onto the real scene, guiding personnel to conduct efficient and accurate verification and construction, and transforming the on-site environment. Feedback data from the operation is transmitted back to the digital platform in real time, thus completing a precise closed loop from "virtual discovery, diagnosis, and decision-making" to "physical execution and verification." Finally, the system's reinforcement learning optimization engine collects all performance data throughout the entire task cycle, including damage detection rate, false alarm rate, task time, energy consumption cost, and prediction accuracy. With the goal of minimizing the total long-term operation and maintenance cost, the system autonomously adjusts future inspection strategy parameters through reinforcement learning algorithms, such as the scheduling logic of drones, sensor configuration preferences, and flight path planning algorithms. This enables the system to achieve self-optimization and continuous performance improvement without human intervention. At this point, the system completes a full work cycle. Through continuous iteration, the operation and maintenance management of road and bridge facilities becomes increasingly precise, efficient, and economical, ultimately achieving digital and intelligent management and control of the entire life cycle of the facilities.

[0021] The above embodiments are provided for those skilled in the art to implement or use the present invention. Those skilled in the art can make various modifications or changes to the above embodiments without departing from the spirit of the present invention. Therefore, the scope of protection of the present invention is not limited to the above embodiments, but should be the maximum scope that conforms to the innovative features mentioned in the claims.

Claims

1. An unmanned aerial vehicle intelligent inspection and digital management system for a bridge facility, characterized in that, The system comprises a cloud digital twin platform, distributed edge computing nodes, and a drone cluster that can be scheduled to execute; the cloud digital twin platform is used to build and maintain a high-precision three-dimensional semantic model corresponding to the physical bridge facility, which integrates geological and geographical information, historical operation and maintenance data, and real-time sensor data; the distributed edge computing nodes are deployed around the key facilities and have data caching and computing capabilities; the system sets up a task intelligent analysis and resource dynamic arrangement module, which responds to the inspection task request, simulates the task in the digital twin platform, generates an optimized virtual formation scheme containing drone model, payload configuration, flight path, and data fusion strategy, and based on this scheme, schedules physical drones from the drone resource pool associated with the edge computing nodes, and before the task execution, injects the task customized lightweight AI model and processing algorithm into the drone on-board computing unit through the edge node, so that the drone becomes an intelligent agent with specific task processing capability.

2. The bridge facility unmanned plane intelligent inspection and digital management system based on claim 1, characterized in that, The on-board computing unit of the drone is integrated with a multi-modal data stream processing and damage feature real-time extraction module; the module is configured to perform online processing on the raw data collected by the on-board optical camera, laser radar, infrared thermal imager, and multispectral sensor, perform operations including image pixel-level segmentation based on deep learning network, point cloud three-dimensional reconstruction, and hot spot anomaly detection, and output structured damage feature data packets, which at least contain damage type, geometric parameter, spatial coordinate, timestamp, and confidence; the damage feature data packet is preferentially transmitted to the adjacent edge computing node for preliminary aggregation and verification, and only the aggregated summary information is synchronized to the cloud digital twin platform. 3.The bridge facility unmanned plane intelligent inspection and digital management system based on claim 1, characterized in that, The cloud digital twin platform further comprises a facility state prediction and risk assessment engine; the engine is coupled with a physical mechanism model and a data-driven model, receives the damage feature data packet and external environment data, and performs at least one of the following prediction simulations: crack propagation prediction based on fracture mechanics, structure bearing capacity influence analysis based on finite element method, material performance degradation trend prediction based on machine learning, and bridge pier local scouring evolution simulation based on computational fluid dynamics; the platform visualizes the prediction results superimposed on the three-dimensional model, and dynamically adjusts the priority and inspection strategy of the subsequent inspection plan according to the predicted risk level.

4. The bridge facility unmanned plane intelligent inspection and digital management system based on claim 1, characterized in that, The system is integrated with a blockchain-based operation and maintenance data evidence and audit traceability module; the module records the hash value, timestamp, and responsible party identifier of the key information of the damage feature data packet, prediction simulation result, risk warning report, and maintenance work order in the distributed ledger, which is maintained by multiple authorized nodes, forming an unalterable operation and maintenance full-life-cycle audit chain.

5. The unmanned aerial vehicle intelligent inspection and digital management system for bridge facilities according to claim 1, characterized in that, The system also includes an augmented reality operation and maintenance auxiliary interaction terminal; the terminal is configured to locate the physical facility through image recognition when the maintenance personnel are on-site, and call and superimpose display of virtual information related to the work position from the cloud digital twin platform, including but not limited to damage annotation, historical data, prediction results, maintenance guidance animation and material list; support the maintenance personnel to input on-site verification data and maintenance feedback through the human-computer interaction interface, and update the digital twin model in real time.

6. The bridge facility unmanned plane intelligent inspection and digital management system based on claim 1, characterized in that, The UAV cluster has self-organizing and cooperative ability based on distributed consensus; each UAV in the cluster shares state and perception information through an inter-machine communication network, and follows preset group intelligence behavior rules, to realize autonomous obstacle avoidance, formation keeping, task dynamic allocation and focused cooperative exploration for key areas in complex environments, without the need for central controller intervention throughout.

7. The unmanned aerial vehicle intelligent inspection and digital management system for bridge facilities according to claim 1, characterized in that, The system is provided with a patrol strategy optimization engine based on reinforcement learning; the engine takes the minimization of long-term operation and maintenance cost and risk as the objective function, and takes the patrol strategy parameters as the action space, to autonomously learn and dynamically optimize the UAV scheduling, sensor configuration, flight path and AI model selection strategy through continuous collection of task execution effect feedback data, to realize autonomous evolution of system patrol efficiency. 8.The bridge facility unmanned plane intelligent inspection and digital management system based on claim 1, characterized in that, The system architecture expansion includes a decentralized UAV resource sharing platform; the platform manages compliant UAV resources registered by third parties through smart contracts, automatically inquires and schedules external resources to participate in the patrol when the task demand is generated according to the contract terms, and realizes the ownership confirmation and automatic settlement of service charges of task data based on blockchain technology.

Images