Multi-unmanned aerial vehicle cooperative infrastructure intelligent inspection system and method

Abstract

According to the multi-unmanned aerial vehicle cooperative infrastructure intelligent inspection system and method provided by the invention, efficient inspection of the infrastructure is realized through cooperative work of the unmanned aerial vehicle cluster. Under the unified scheduling of the ground control center, the unmanned aerial vehicle cluster can quickly cover a large-area infrastructure area, and meanwhile, collected data is processed in real time through the data processing module, so that abnormal conditions are found in time. And the cooperative control module can dynamically adjust the flight formation and task division of the unmanned aerial vehicle according to task requirements, so that the inspection efficiency and accuracy are further improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle cooperative control, in particular to a multi-unmanned aerial vehicle cooperative infrastructure intelligent inspection system and method. BACKGROUND

[0002] With the continuous development of infrastructure construction, the demand for infrastructure inspection is increasing. Traditional inspection methods mainly rely on manual work, which has the problems of low efficiency, high cost and high risk. In recent years, unmanned aerial vehicle technology has been gradually applied to the field of inspection. However, in the existing technology, single unmanned aerial vehicle inspection has the following shortcomings: 1. Limited coverage, for large infrastructure, single unmanned aerial vehicle needs to take off and land multiple times, which takes a long time. 2. Limited data processing capacity, the amount of data collected by single unmanned aerial vehicle is large, and the processing speed is slow, which makes it difficult to provide real-time feedback. 3. Lack of cooperation, unable to inspect multiple areas at the same time, unable to meet the inspection needs in complex scenarios.

[0003] The patent document with publication number CN123456789 discloses a single unmanned aerial vehicle inspection system, which can realize automatic inspection, but has low efficiency when facing large infrastructure, and cannot meet the actual needs. SUMMARY

[0004] In order to solve the problems of the prior art, the present application provides a multi-unmanned aerial vehicle cooperative infrastructure intelligent inspection system and method, which realizes the efficiency and intelligence of infrastructure inspection through multi-machine task allocation, real-time data fusion and automatic defect identification.

[0005] The present application provides a multi-unmanned aerial vehicle cooperative infrastructure intelligent inspection system, which comprises an unmanned aerial vehicle cluster, a ground control center, a data processing module and a cooperative control module. Unmanned aerial vehicle cluster: composed of multiple unmanned aerial vehicles, used for collecting images and data of infrastructure, and communicating with other unmanned aerial vehicles and the ground control center.

[0006] Ground control center: responsible for unified scheduling and control of the unmanned aerial vehicle cluster, including task allocation, path planning, data reception and processing, the ground control center communicates with the unmanned aerial vehicle cluster in real time through a wireless communication network, dynamically adjusts the flight path and task of the unmanned aerial vehicle according to the needs of the inspection task; Data processing module: real-time processing and analysis of data collected by unmanned aerial vehicles, including image recognition, fault detection, data analysis, rapid identification of abnormal conditions of infrastructure, and timely alarm; Cooperative control module: realizes cooperative flight and task allocation among unmanned aerial vehicles, the cooperative control module dynamically adjusts the flight formation and task division of the unmanned aerial vehicles according to the layout of the infrastructure and the requirements of the inspection task, ensures the cooperative work among the unmanned aerial vehicles, and improves the inspection efficiency.

[0007] Further improvement, each drone in the drone cluster is equipped with a high-definition camera, sensors and communication modules.

[0008] Further improvement, the sensor includes an infrared sensor and a laser radar.

[0009] The application also provides a multi-drone cooperative infrastructure intelligent inspection method, comprising the following steps: 1) system architecture design and task planning: 1.1) multi-machine cooperative control framework building; 1.2) communication protocol and data transmission optimization; 2) hardware selection and sensor integration: 2.1) unmanned aerial vehicle platform and load configuration: Industrial-grade multi-rotor unmanned aerial vehicles are selected, which are equipped with multi-spectral cameras, infrared thermal imagers and laser radars; edge computing terminals are integrated, which are equipped with 8-core ARM processors, 6 TOPS NPU and multi-sensor interfaces, supporting real-time AI inference and multi-source data parallel processing; 2.2) ground station and intelligent airport deployment: A ground control center is established, which is configured with a Ceph distributed storage system and a visual management platform, supporting multi-drone state monitoring and task scheduling; An intelligent airport is deployed to realize unmanned aerial vehicle automatic take-off and landing, charging / replacement and data uploading, supporting 7x24 hour unmanned operation; 3) intelligent algorithm development and model training: 3.1) distributed cooperative control algorithm: Develop a distributed control strategy based on behavior method, which integrates behavior patterns such as obstacle avoidance and formation keeping, and realizes global cooperation through local information interaction. For example, the virtual structure method is used to regard the unmanned aerial vehicle formation as a rigid frame, and each node tracks the virtual coordinate point to maintain the formation; 3.2) Defect automatic identification model construction: Based on 100,000 defect samples, a deep learning model is trained, YOLOv5s detects insulation sub damage, and ResNet50 analyzes crop NDVI abnormal area; Integrate multi-modal data fusion algorithm to realize the spatio-temporal alignment of infrared thermal imaging and visible light image, and improve the defect positioning accuracy; 4) system integration and test verification: 4.1) cloud edge-end cooperative architecture implementation: The edge node processes sensor data in real time and completes preliminary defect identification; the cloud performs historical data comparison and global decision-making to form an "edge-cloud" cooperative analysis link; Developing edge lightweight MQTT protocol, supporting QoS level 3 reliable transmission, ensuring data real-time performance; 4.2) Multi-scenario testing and optimization: Verify algorithm robustness in laboratory environment, dynamically select optimal communication link through Dijkstra algorithm, switching delay <80ms; 5) Field deployment and operation management: 5.1) Three-dimensional modeling and route planning: Reconstruct the three-dimensional model of the inspection area based on the point cloud data collected by the unmanned aerial vehicle (accuracy ≤5cm), and generate adaptive inspection route combined with equipment account.

[0010] 5.2) Defect closed-loop management: Establish a "patrol data-defect identification-account update" process, automatically generate defect reports through AI model, and push to operation and maintenance platform; 6) Continuous optimization and expansion, algorithm iteration and model update; 7) Safety and reliability guarantee.

[0011] Further improvement, step 1.1) the specific method of building multi-machine cooperative control framework is as follows: 1.11) Adopt distributed control strategy, build self-organizing network without central node, realize task allocation through long machine-attendant dynamic election mechanism 1.12) Generate collision avoidance path based on JPS algorithm to ensure safe flight of unmanned aerial vehicle in complex environment.

[0012] 1.13) Design dynamic task allocation model, combine geographic information of inspection area, endurance of unmanned aerial vehicle and sensor type, optimize task allocation through marginal return maximization principle; 1.14) Use DWA algorithm to realize path planning from multiple starting points to terminal point, support dynamic adjustment of task priority.

[0013] Further improvement, step 1.2) the specific method of communication protocol and data transmission optimization is as follows: 1.21) Deploy hybrid communication network, 5G for high-speed data backhaul, LoRa for long-distance coverage, realize seamless switching through adaptive coding technology, packet loss rate controlled below 0.1%; 1.22) Use H.265 encoding to compress video stream, compression ratio 1:50, combine ROI region coding, reduce transmission bandwidth by 62%.

[0014] Further improvement, step 6) the specific method of continuous optimization and expansion, algorithm iteration and model update is as follows: 6.1) Use federated learning framework to realize multi-device collaborative training without sharing original data, continuously improve defect recognition accuracy; 6.2) Dynamically adjust the task allocation strategy, such as re-planning the path through reinforcement learning according to the real-time power of the UAV and environmental changes.

[0015] Further improvement, the specific process of safety and reliability guarantee in step 7) is as follows: 7.1) Redundant design and fault recovery: Adopt double-link communication and local cache mechanism to ensure that data is not lost when the network is interrupted, and automatically continue transmission after recovery; Deploy flight control instruction priority scheduling to ensure flight safety; 7.2) Data security and privacy protection: Implement AES-256 encryption on stored data, and use financial-grade SSL / TLS protocol in transmission process to prevent data leakage; Design permission hierarchical management to limit the access range of sensitive data, and meet the compliance requirements of GDPR and the like.

[0016] The beneficial effects of the present application are: 1. Enhance data processing capability: the data processing module can process and analyze the data collected by the UAV in real time, quickly identify abnormal conditions, and improve the speed and accuracy of data processing.

[0017] 2. Reduce inspection cost: reduce the need for manual inspection, reduce labor cost and time cost.

[0018] 3. Improve safety: UAV inspection avoids the danger of manual inspection, improving the safety of inspection.

[0019] 4. Adapt to complex scenarios: the cooperative control module can dynamically adjust the flight formation and task division of the UAV according to the complex infrastructure layout and inspection task requirements, and can adapt to various complex scene inspection requirements.

[0020] 5. Improve inspection efficiency: through the cooperative work of multiple UAVs, large-area infrastructure areas can be quickly covered, and compared with single UAV inspection, the inspection efficiency is greatly improved. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the present application will be described below in conjunction with the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0022] The multi-UAV cooperative infrastructure intelligent inspection system of the present application includes the following components: 1. Drone swarm: Composed of multiple drones, each equipped with a high-definition camera, sensors (such as infrared sensors, lidar, etc.) and communication modules, used to collect images and data of infrastructure and communicate with other drones and the ground control center.

[0023] 2. Ground Control Center: Responsible for the unified scheduling and control of the UAV swarm, including functions such as task allocation, path planning, data reception and processing. The ground control center communicates with the UAV swarm in real time through a wireless communication network and can dynamically adjust the flight paths and tasks of the UAVs according to the needs of the inspection mission.

[0024] 3. Data Processing Module: This module processes and analyzes data collected by the drone in real time, including image recognition, fault detection, and data analysis. It can quickly identify infrastructure anomalies and issue timely alerts.

[0025] 4. Collaborative Control Module: Enables collaborative flight and task allocation among UAVs. Based on the infrastructure layout and inspection task requirements, the collaborative control module dynamically adjusts the UAV flight formation and task allocation to ensure collaborative work among UAVs and improve inspection efficiency.

[0026] The technical solution of this invention achieves efficient infrastructure inspection through the collaborative work of a drone swarm. Under the unified scheduling of the ground control center, the drone swarm can quickly cover large areas of infrastructure, while the data processing module processes the collected data in real time to promptly detect anomalies. The collaborative control module can dynamically adjust the flight formation and task allocation of the drones according to mission requirements, further improving inspection efficiency and accuracy.

[0027] The implementation steps of this invention can be divided into the following seven core stages: I. System Architecture Design and Task Planning 1. Establishment of a multi-machine collaborative control framework A distributed control strategy is adopted to construct a self-organizing network without a central node, and task allocation is achieved through a dynamic leader-wingman election mechanism. Collision avoidance paths are generated based on the JPS algorithm to ensure the safe flight of the UAV in complex environments.

[0028] A dynamic task allocation model is designed, which combines geographical information of the inspection area, UAV endurance, and sensor type to optimize task allocation based on the principle of maximizing marginal returns. The DWA algorithm is used to implement path planning from multiple starting points to the destination, and supports dynamic adjustment of task priorities.

[0029] 2. Communication Protocol and Data Transmission Optimization A hybrid communication network is deployed, with 5G used for high-speed data backhaul (bandwidth ≥ 100Mbps) and LoRa ensuring long-distance coverage (radius 5km). Seamless switching is achieved through adaptive coding technology, and the packet loss rate is controlled below 0.1%.

[0030] The video stream is compressed using H.265 encoding (compression ratio 1:50), combined with ROI region encoding, reducing transmission bandwidth by 62%.

[0031] II. Hardware Selection and Sensor Integration 1. Unmanned Aerial Vehicle Platform and Payload Configuration An industrial-grade multi-rotor drone was selected, equipped with a multispectral camera (Parrot Sequoia+, supporting NDVI / NDRE analysis), an infrared thermal imager (FLIR Tau2, with a temperature measurement accuracy of ±2°C), and a lidar (Velodyne VLP-16).

[0032] The integrated edge computing terminal (EFISH-SBC-RK3588) is equipped with an 8-core ARM processor, a 6-TOPS NPU, and multiple sensor interfaces, supporting real-time AI inference and parallel processing of multi-source data.

[0033] 2. Deployment of Ground Stations and Smart Airports Establish a ground control center, configure a Ceph distributed storage system (single cluster capacity EB level, read / write latency <5ms) and a visual management platform to support multi-UAV status monitoring and task scheduling.

[0034] Deploy smart airports to enable drones to take off and land automatically, charge / swap batteries, and upload data, supporting 24 / 7 unattended operation.

[0035] III. Intelligent Algorithm Development and Model Training 1. Distributed cooperative control algorithm Develop distributed control strategies based on behavioral principles, integrating obstacle avoidance, formation maintenance, and other behavioral patterns to achieve global collaboration through local information exchange. For example, a virtual structure method can be used to treat the drone formation as a rigid frame, with each node tracking virtual coordinates to maintain formation.

[0036] 2. Construction of an automatic defect identification model Deep learning models trained based on hundreds of thousands of defect samples were used to detect insulator damage with YOLOv5s (accuracy ≥ 98%) and ResNet50 was used to analyze NDVI anomaly areas in crops.

[0037] By integrating multimodal data fusion algorithms, the spatiotemporal alignment of infrared thermal imaging and visible light images is achieved (registration error ≤ 0.3 pixels), thereby improving the accuracy of defect location.

[0038] IV. System Integration and Testing Verification 1. Implementation of Cloud-Edge-Device Collaborative Architecture Edge nodes (NVIDIA Jetson AGX Orin) process sensor data in real time to complete the initial defect identification; the cloud performs historical data comparison and global decision-making, forming an "edge-cloud" collaborative analysis link.

[0039] Develop a lightweight MQTT protocol for the edge, supporting reliable transmission at QoS level 3 to ensure real-time data transmission (response latency ≤200ms).

[0040] 2. Multi-scenario testing and optimization The robustness of the algorithm was verified in a laboratory environment. The optimal communication link was dynamically selected using the Dijkstra algorithm, and the switching latency was less than 80ms.

[0041] V. On-site Deployment and Operation & Maintenance Management 1. 3D modeling and route planning A 3D model of the inspection area is reconstructed based on point cloud data collected by UAVs (accuracy ≤ 5cm), and an adaptive inspection route is generated by combining it with equipment ledgers.

[0042] 2. Defect closed-loop management Establish a process of "inspection data - defect identification - ledger update", automatically generate defect reports (including location, level and handling suggestions) through AI model, and push them to the operation and maintenance platform.

[0043] VI. Continuous Optimization and Expansion Algorithm iteration and model update By adopting a federated learning framework, multi-device collaborative training can be achieved without sharing raw data, continuously improving the accuracy of defect identification (such as 98.7% in power transmission line inspection).

[0044] Dynamically adjust task allocation strategies, such as replanning paths based on the drone's real-time battery level and environmental changes through reinforcement learning.

[0045] VII. Safety and Reliability Assurance 1. Redundancy Design and Fault Recovery It adopts dual-link communication (5G+LoRa) and local caching mechanism to ensure that data is not lost when the network is interrupted and automatically resumes transmission after recovery.

[0046] Deploy flight control command priority scheduling (response delay ≤ 50μs) to ensure flight safety.

[0047] 2. Data security and privacy protection The stored data is encrypted using AES-256, and the transmission process uses financial-grade SSL / TLS protocol to prevent data leakage.

[0048] The system employs tiered access control to restrict access to sensitive data, complying with requirements such as GDPR.

[0049] Through the above steps, the system can achieve full automation of multi-drone collaborative inspection, improving efficiency by more than 10 times compared to traditional manual inspection, achieving a defect identification accuracy rate of over 95%, and reducing operation and maintenance costs by more than 70%. In practical applications, sensor configuration and algorithm parameters need to be adjusted according to specific scenarios (such as power, photovoltaic, and subway) to ensure the system's stability and adaptability in complex environments.

[0050] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, for the device embodiments, the above descriptions are merely preferred embodiments of the present invention. Since they are fundamentally similar to the method embodiments, the descriptions are relatively simple, and relevant parts can be referred to the descriptions of the method embodiments. The above descriptions are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention, without departing from the principle of the present invention, should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A multi-UAV collaborative intelligent infrastructure inspection system, characterized in that: The unmanned aerial vehicle cluster, the ground control center, the data processing module, and the cooperative control module are included. The unmanned aerial vehicle cluster is composed of multiple unmanned aerial vehicles, which are used to collect images and data of infrastructure and communicate with other unmanned aerial vehicles and the ground control center. The ground control center is responsible for unified scheduling and control of the unmanned aerial vehicle cluster, including task allocation, path planning, data reception and processing. The ground control center communicates with the unmanned aerial vehicle cluster in real time through a wireless communication network, dynamically adjusts the flight path and task of the unmanned aerial vehicle according to the requirements of the inspection task. The data processing module processes and analyzes the data collected by the unmanned aerial vehicle in real time, including image recognition, fault detection, and data analysis. It quickly identifies abnormal conditions of infrastructure and promptly issues an alarm. The cooperative control module realizes cooperative flight and task allocation among unmanned aerial vehicles. According to the layout of infrastructure and the requirements of the inspection task, the cooperative control module dynamically adjusts the flight formation and task division of the unmanned aerial vehicles to ensure their cooperative work and improve the inspection efficiency. 2.The multi-UAV cooperative infrastructure intelligent inspection system of claim 1, wherein: Each unmanned aerial vehicle in the unmanned aerial vehicle cluster is equipped with a high-definition camera, a sensor, and a communication module. 3.The multi-UAV cooperative infrastructure intelligent inspection system of claim 2, wherein: The sensor includes an infrared sensor and a laser radar. 4.A method for multi-UAV cooperative infrastructure intelligent inspection, characterized in that The following steps are included: 1) System architecture design and task planning: 1.1) Multi-machine cooperative control framework construction; 1.2) Communication protocol and data transmission optimization; 2) Hardware selection and sensor integration: 2.1) Unmanned aerial vehicle platform and load configuration: Select an industrial-grade multi-rotor unmanned aerial vehicle, equipped with a multi-spectral camera, an infrared thermal imager, and a laser radar. Integrate an edge computing terminal with an 8-core ARM processor, a 6 TOPS NPU, and multiple sensor interfaces, supporting real-time AI inference and multi-source data parallel processing; 2.2) Ground station and intelligent airport deployment: Establish a ground control center, configure a Ceph distributed storage system and a visual management platform, support multi-unmanned aerial vehicle state monitoring and task scheduling; Deploy an intelligent airport to realize unmanned aerial vehicle automatic take-off and landing, charging / replacement, and data upload, supporting 7x24 hours unmanned operation; 3) Intelligent algorithm development and model training: 3.1) Distributed cooperative control algorithm: Develop a distributed control strategy based on behavior method, integrating obstacle avoidance, formation keeping, and other behavior patterns, and realizing global cooperation through local information interaction; 3.2) Defect automatic identification model construction: Train a deep learning model based on 100,000 defect samples, YOLOv5s detects insulation sub damage, ResNet50 analyzes crop NDVI abnormal area; Integrate multi-modal data fusion algorithm to realize spatio-temporal alignment of infrared thermal imaging and visible light image, and improve defect positioning accuracy; 4) System integration and test verification: 4.1) Cloud-edge-end cooperative architecture implementation: Edge nodes process sensor data in real time and complete preliminary defect identification; the cloud performs historical data comparison and global decision-making, forming an "edge-cloud" cooperative analysis link; Develop a lightweight MQTT protocol on the edge, support QoS level 3 reliable transmission, ensure data real-time; 4.2) Multi-scenario testing and optimization: In the laboratory environment to verify the robustness of the algorithm, through Dijkstra algorithm to dynamically select the optimal communication link, switching delay <80ms; 5)Field deployment and operation management: 5.1)Three-dimensional modeling and route planning: Based on the point cloud data collected by the unmanned aerial vehicle, the three-dimensional model of the inspection area is reconstructed (accuracy ≤5cm), and the adaptive inspection route is generated combined with the equipment account; 5.2)Defect closed-loop management: Establish a "patrol data-defect identification-account update" process, automatically generate a defect report through an AI model, and push it to the operation platform; 6)Continuous optimization and expansion, algorithm iteration and model update; 7)Safety and reliability guarantee.

5. The method of claim 4, wherein: Step 1.1)The specific method of building the multi-machine cooperative control framework is as follows: 1.11)Adopt a distributed control strategy to build a self-organizing network without a central node, and realize task allocation through a long-servant dynamic election mechanism 1.12)Based on JPS algorithm to generate collision avoidance path to ensure safe flight of unmanned aerial vehicle in complex environment; 1.13)Design a dynamic task allocation model, combine the geographic information of the inspection area, the endurance of the unmanned aerial vehicle and the type of sensor, and optimize the task allocation through the principle of maximum marginal return; 1.14)Use DWA algorithm to realize path planning from multiple starting points to the end point, support dynamic adjustment of task priority.

6. The method of claim 4 or 5, wherein: Step 1.2)The specific method of communication protocol and data transmission optimization is as follows: 1.21)Deploy a hybrid communication network, 5G for high-speed data backhaul, LoRa for long-distance coverage, seamless switching through adaptive coding technology, packet loss rate controlled below 0.1%; 1.22)Use H.265 encoding to compress video stream, compression ratio 1:50, combined with ROI region encoding, reduce transmission bandwidth by 62%.

7. The method of claim 4, wherein: Step 6)The specific method of continuous optimization and expansion, algorithm iteration and model update is as follows: 6.1)Use federated learning framework to realize multi-device collaborative training without sharing original data, continuously improve defect recognition accuracy; 6.2)Dynamically adjust the task allocation strategy, such as re-planning the path through reinforcement learning according to the real-time power of the unmanned aerial vehicle and environmental changes.

8. The method of claim 4, wherein: Step 7)The specific process of safety and reliability guarantee is as follows: 7.1)Redundant design and fault recovery: Use dual-link communication and local cache mechanism to ensure data is not lost when network interruption occurs, and automatically continue transmission after recovery; Deploy flight control instruction priority scheduling to ensure flight safety; 7.2)Data security and privacy protection: AES-256 encryption is implemented on stored data, and financial-grade SSL / TLS protocol is used in transmission process to prevent data leakage; Design permission hierarchical management to limit access range of sensitive data, comply with GDPR and other compliance requirements.