Water conservancy facility autonomous inspection system and method

Abstract

The invention provides a water conservancy facility autonomous inspection system and method, and relates to the technical field of water conservancy facility inspection, and the system comprises an unmanned aerial vehicle unit which comprises a plurality of unmanned aerial vehicles and an unmanned aerial vehicle airport, and the unmanned aerial vehicle airport is arranged in a regional reservoir group in a grid manner; a plurality of flight control platforms, wherein at least one flight control platform is arranged in each reservoir; the flight control platform constructs an exception recognition model, generates an inspection task and a corresponding priority after receiving the sensing data, and issues a scheduling instruction, and the exception recognition model recognizes an abnormal object according to the inspection data; the flight control platform is in communication connection with the unmanned aerial vehicle unit through the communication unit; and the flight control platform cooperatively schedules the unmanned aerial vehicle, and the unmanned aerial vehicle executes the inspection task according to the priority. Gridding deployment is carried out on the regional reservoir group, so that the unmanned aerial vehicle collects the overall data of the regional reservoir group, and the limitation of single reservoir inspection is broken through; the multiple flight control platforms cooperatively distribute the unmanned aerial vehicles to execute inspection tasks, and cooperative inspection of the regional reservoir group is realized.

Description

A self-inspection system and method for water conservancy facilities Technical Field

[0001] This invention relates to the field of water conservancy facility inspection technology, specifically to an autonomous inspection system and method for water conservancy facilities. Background Technology

[0002] Reservoirs, as water conservancy infrastructure, are crucial to people's livelihoods and social stability. Reservoir inspection is a core means of ensuring dam safety and understanding the reservoir area's condition. With the development of sensing technology, communication technology, and artificial intelligence, reservoir inspection technology has evolved from a purely manual mode to semi-automation, and is gradually moving towards intelligent and unmanned operation. However, current reservoir inspection technology focuses on improving the inspection capabilities of individual reservoirs, centering on data collection and processing for single reservoirs; it lacks collaborative inspection of regional reservoir groups, resulting in the inability to conduct task coordination, data fusion, and joint analysis of regional reservoir groups. Summary of the Invention

[0003] The purpose of this invention is to provide an autonomous inspection system and method for water conservancy facilities, and the technical problem to be solved is how to achieve collaborative inspection of a group of regional reservoirs.

[0004] This invention is achieved through the following technical solution:

[0005] The first aspect provides an autonomous inspection system for water conservancy facilities, including:

[0006] The unmanned aerial vehicle (UAV) unit includes several UAVs and UAV airports adapted to the UAVs. The UAV airports are deployed in a grid pattern in the regional reservoir group; the UAVs collect inspection data.

[0007] Several flight control platforms are deployed, with at least one flight control platform deployed in each reservoir of the aforementioned reservoir group. The aforementioned flight control platform calls an anomaly identification model. The flight control platform is used to receive the perception data of the corresponding reservoir, generate inspection tasks and the priority of the inspection tasks, and issue scheduling instructions. The aforementioned anomaly identification model identifies abnormal objects in the reservoir based on the inspection data.

[0008] The communication unit connects the flight control platform to the UAV unit. The communication unit transmits the inspection data collected by the UAV back to the flight control platform and sends the scheduling instructions and inspection tasks issued by the flight control platform to the UAV. The flight control platform coordinates the scheduling of the UAV, and the UAV performs inspection tasks according to priority.

[0009] The aforementioned drone airport is deployed in a grid pattern across a regional reservoir cluster. Adapted drones collect inspection data from the reservoirs and surrounding areas from an aerial perspective, enabling the drones to gather comprehensive data on the entire reservoir cluster. This overcomes the limitations of individual reservoir inspections and provides a foundation for collaborative inspections of the entire cluster, ensuring that each reservoir and the areas between them are inspected. Compared to traditional methods, this approach allows for more comprehensive data collection across the entire reservoir cluster. By coordinating the allocation of drones across multiple flight control platforms, inspection tasks for different drones can be planned according to the overall needs of the reservoir cluster, achieving coordinated task allocation, avoiding duplicate or missed inspections, and improving inspection efficiency. Each flight control platform, upon receiving inspection data from its corresponding reservoir, can identify anomalies in that individual reservoir. The flight control platform and the drone units communicate in real-time via communication units, allowing the flight control platform to promptly monitor the drone's operational status and collected inspection data, while the drones can also promptly receive and execute commands from the flight control platform, ensuring the smooth operation of the collaborative inspection process.

[0010] Furthermore, the aforementioned communication unit includes:

[0011] The protocol adaptation layer communicates with the flight control platform and the UAV unit. This protocol adaptation layer adopts the MAVLink 2.0 protocol. The protocol adaptation layer is used to transmit the inspection tasks and scheduling instructions issued by the flight control platform, as well as the status information of the UAV.

[0012] The video stream transmission layer communicates with the flight control platform and the UAV unit. This video stream transmission layer uses RTSP and HLS protocols and is used to transmit the video streams captured by the UAV.

[0013] The heterogeneous network access layer includes dual communication channels for 4G / 5G cellular networks and Mesh networks; the heterogeneous network access layer is used to allocate communication channels for different types of communication data according to preset QoS policies.

[0014] The aforementioned protocol adaptation layer, video stream transmission layer, and heterogeneous network access layer are all connected to the flight control platform.

[0015] Furthermore, the SRT protocol is used to transmit the aforementioned inspection data.

[0016] Furthermore, the aforementioned flight control platform communicates with at least one UAV through a protocol adaptation layer. This flight control platform is used to receive the UAV's status information and issue inspection tasks and scheduling instructions.

[0017] Furthermore, after issuing scheduling instructions and inspection tasks, the aforementioned flight control platform uses a communication unit to send the scheduling instructions and inspection tasks to the corresponding UAVs and UAV airports, coordinating multiple UAVs and UAV airports to perform inspection tasks.

[0018] Furthermore, the aforementioned drone is equipped with a positioning module, which enables real-time positioning of the drone.

[0019] The second aspect provides a method for autonomous inspection of water conservancy facilities, which employs any of the aforementioned inspection systems; the method includes the following steps:

[0020] Call the anomaly identification model on the flight control platform;

[0021] The flight control platform described above performs the following operations:

[0022] After receiving the reservoir's sensing data, an inspection route and corresponding priority are generated based on the aforementioned sensing data;

[0023] Obtain the current location of each drone, and determine multiple drones adjacent to the inspection route according to the priority of the above inspection route and the current location of the drone, so as to obtain the scheduled drones.

[0024] Based on the above-mentioned scheduling of drones, scheduling instructions are determined, and inspection tasks are generated based on the above-mentioned inspection routes.

[0025] The aforementioned flight control platform sends dispatch instructions and inspection tasks to the dispatch drone via the communication unit, and the dispatch drone collects inspection data along the inspection route.

[0026] The aforementioned dispatching drone transmits inspection data back to the flight control platform via the communication unit, and the aforementioned anomaly identification model identifies abnormal objects in the reservoir based on the inspection data.

[0027] Furthermore, the aforementioned anomaly detection model utilizes the YOLOv7 network architecture and a multispectral image fusion algorithm to identify floating objects on the water surface based on the YOLOv7 network architecture and to identify garbage in the reservoir based on the multispectral image fusion algorithm; the aforementioned abnormal objects include floating objects and garbage.

[0028] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0029] The aforementioned UAV airport is deployed in a grid pattern across a regional reservoir cluster. Adapted UAVs collect inspection data from the reservoirs and surrounding areas from an aerial perspective, enabling the UAVs to gather comprehensive data on the entire reservoir cluster. This overcomes the limitations of individual reservoir inspections and provides a foundation for collaborative inspections of the entire cluster, ensuring that each reservoir and the areas between them are inspected. Compared to traditional methods, this approach allows for more comprehensive data collection across the entire reservoir cluster. By coordinating the allocation of UAVs across multiple flight control platforms, inspection routes for different UAVs can be planned according to the overall needs of the reservoir cluster, achieving coordinated task allocation, avoiding duplicate or missed inspections, and improving inspection efficiency. Each flight control platform, upon receiving inspection data from its corresponding reservoir, can identify anomalies in that individual reservoir. The flight control platform and the UAV units communicate in real-time via communication units, allowing the flight control platform to promptly monitor the UAVs' operational status and collected inspection data, while the UAVs can also promptly receive and execute commands from the flight control platform, ensuring the smooth operation of the collaborative inspection process. Attached Figure Description

[0030] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings:

[0031] Figure 1 is a block diagram of the autonomous inspection system;

[0032] Figure 2 is a flowchart of the inspection method.

[0033] The attached diagram shows the markings and corresponding component names:

[0034] 1. Unmanned Aerial Vehicle (UAV) Unit; 2. Communication Unit; 3. Flight Control Platform. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of this invention are only for explaining this invention and are not intended to limit this invention.

[0036] First embodiment:

[0037] A self-inspection system for water conservancy facilities, comprising:

[0038] Unmanned Aerial Vehicle (UAV) Unit 1 includes several UAVs and UAV airports adapted to the UAVs. The UAV airports are deployed in a grid pattern in a regional reservoir group. Inspection data is collected by the UAVs.

[0039] Several flight control platforms 3 are provided, with at least one flight control platform 3 deployed in each reservoir of the aforementioned reservoir group. Each flight control platform 3 receives sensing data from the corresponding reservoir, generates inspection tasks and their priorities, and issues scheduling instructions. The flight control platform 3 invokes an anomaly identification model to identify abnormal objects within the reservoir based on the inspection data. This anomaly identification model may include at least a YOLOv7 network architecture and a multispectral image fusion algorithm. A floating object detection model is developed based on the YOLOv7 network architecture to identify floating objects on the water surface, adapting to different water scenarios. A multispectral image fusion algorithm (which may be a fusion of PCA and HSV) is used to identify garbage within the reservoir, improving the accuracy of garbage identification. The aforementioned abnormal objects may include both floating objects and garbage.

[0040] Communication unit 2, the aforementioned flight control platform 3 communicates with UAV unit 1 through communication unit 2; through the aforementioned communication unit 2, the inspection data collected by the UAV is transmitted back to the flight control platform 3, and the scheduling instructions and inspection tasks issued by the flight control platform 3 are sent to the UAV; through the aforementioned flight control platform 3, the UAV is coordinated and scheduled, and the UAV performs inspection tasks according to priority.

[0041] The aforementioned UAV airport is deployed in a grid pattern across a regional reservoir cluster. Adapted UAVs collect inspection data from the reservoirs and surrounding areas from an aerial perspective, enabling the UAVs to gather comprehensive data on the entire reservoir cluster. This overcomes the limitations of individual reservoir inspections and provides a foundation for collaborative inspections of the entire cluster, ensuring that each reservoir and the areas between them are inspected. Compared to traditional methods, this approach allows for more comprehensive data collection across the entire reservoir cluster. Multiple flight control platforms 3 collaboratively allocate UAVs, planning inspection routes for different UAVs based on the overall needs of the reservoir cluster. This collaborative task allocation avoids duplicate or missed inspections, improving inspection efficiency. Each flight control platform 3, upon receiving inspection data from its corresponding reservoir, can identify anomalies in that reservoir. Flight control platforms 3 and UAV units 1 communicate in real-time via communication units 2, allowing flight control platforms 3 to promptly monitor the UAVs' operational status and collected inspection data. Simultaneously, the UAVs can receive and execute commands from flight control platforms 3, ensuring the smooth operation of the collaborative inspection process.

[0042] In addition, the data processed by multiple flight control platforms can be integrated and analyzed at a higher level to determine the correlation and impact range of abnormal situations from a regional perspective, and realize data fusion and linkage analysis of regional reservoir groups.

[0043] This invention, in addition to drones, can also utilize unmanned surface vessels and other monitoring equipment to improve the integrated "sky-ground-hydraulic" monitoring and sensing network for reservoirs. It leverages IoT cards to achieve wide-area networking of drone equipment, ensuring real-time data transmission and remote command transmission. Existing technologies cannot dynamically adjust the inspection priorities and focus of multiple reservoirs based on macro-level information such as basin rainfall, upstream reservoir discharge, and regional geological disaster coordination. When this invention receives a forecast of heavy rainfall in the basin, it automatically generates a collaborative task sequence prioritizing inspections of key upstream reservoirs and high-risk slopes and banks, enhancing the monitoring and sensing capabilities for flood and drought disaster prevention, water resource management and allocation, and water conservancy project construction and operation management.

[0044] Second embodiment:

[0045] Based on the first embodiment, the communication unit 2 includes:

[0046] The protocol adaptation layer is connected to the flight control platform 3 and the UAV unit 1. The protocol adaptation layer adopts the MAVLink 2.0 protocol. The protocol adaptation layer is used to transmit the inspection tasks and scheduling instructions issued by the flight control platform 3, as well as the status information of the UAV.

[0047] The video stream transmission layer is communicatively connected to the flight control platform 3 and the UAV unit 1. This video stream transmission layer adopts the RTSP and HLS protocols and is used to transmit the video streams acquired by the UAV.

[0048] The heterogeneous network access layer includes dual communication channels for 4G / 5G cellular networks and Mesh networks; the heterogeneous network access layer is used to allocate communication channels for different types of communication data according to preset QoS policies; and the inspection data is transmitted using the SRT protocol.

[0049] The aforementioned protocol adaptation layer, video stream transmission layer, and heterogeneous network access layer are all connected to the flight control platform 3.

[0050] The aforementioned communication unit 2 adopts a multi-protocol converged communication architecture. Specifically, it uses the MAVLink 2.0 protocol as the core communication standard, is compatible with the protocol adaptation layer of mainstream flight control protocols such as DJMSDK / PSDK and ArduPilot, and integrates the RTSP / HLS protocol video streaming layer to achieve separate transmission of real-time image transmission and flight control. It also features heterogeneous network adaptive access, specifically by constructing a dual-channel redundant communication system of 4G / 5G private network and Mesh network, dynamically allocating scheduling commands and inspection data through QoS policies, and using the SRT protocol to ensure the reliability of data transmission in complex environments.

[0051] The aforementioned drone airport can utilize a carbon fiber composite material shell, integrating a temperature control system and an automatic charging module to ensure stable operation in complex environments. The airport deployment employs a grid-based strategy, covering the entire watershed of the regional reservoir group and supporting multi-drone collaborative operations. Furthermore, based on spatiotemporal network modeling, multi-objective optimization algorithms can be developed to achieve dynamic task allocation for drones. By monitoring drone status and environmental data in real time, inspection routes can be dynamically adjusted to ensure the efficiency of inspection task execution.

[0052] Third embodiment:

[0053] Based on the second embodiment, the flight control platform 3 communicates with at least one UAV through a protocol adaptation layer. The flight control platform 3 is used to receive the status information of the UAV and issue inspection tasks and scheduling instructions.

[0054] During the inspection process, if the status information transmitted back to the flight control platform 3 by a certain drone indicates that the drone is malfunctioning or has insufficient power, the flight control platform 3 will send a dispatch command to activate a drone that is not operating near the malfunctioning drone to replace the inspection work of the malfunctioning drone; the malfunctioning drone must avoid operating drones on its return trip to reduce interference with the operating drones.

[0055] In a specific embodiment, after the flight control platform issues scheduling instructions and inspection tasks, the communication unit 2 sends the scheduling instructions and inspection tasks to the corresponding UAVs and UAV airports, and coordinates multiple UAVs and UAV airports to perform inspection tasks.

[0056] When a drone malfunctions or runs out of power, it needs to be returned to the drone airport for repair or charging.

[0057] Fourth embodiment:

[0058] Based on any of the above embodiments, the above-mentioned drone is equipped with a positioning module, which is used to locate the drone in real time.

[0059] This positioning module supports automatic switching between GPS / GLONASS / BeiDou systems. Through a layered decoupling design, it can achieve seamless integration with UAV airports and data centers, while ensuring the safety and real-time performance of flight control, and meeting the needs of multi-aircraft collaborative operations in grid-based deployment scenarios.

[0060] Fifth embodiment:

[0061] A method for autonomous inspection of water conservancy facilities, wherein the method employs any of the aforementioned inspection systems; the method includes the following steps:

[0062] Call the anomaly identification model on flight control platform 3;

[0063] The flight control platform 3 described above shall perform the following operations:

[0064] After receiving the reservoir's sensing data, an inspection route and corresponding priority are generated based on the aforementioned sensing data;

[0065] Obtain the current location of each drone, and determine multiple drones adjacent to the inspection route according to the priority of the above inspection route and the current location of the drone, so as to obtain the scheduled drones.

[0066] Based on the above-mentioned scheduling of drones, scheduling instructions are determined, and inspection tasks are generated based on the above-mentioned inspection routes.

[0067] The aforementioned flight control platform 3 sends dispatch instructions and inspection tasks to the dispatch drone through the communication unit 2, and the dispatch drone collects inspection data along the inspection route.

[0068] The aforementioned dispatching drone transmits inspection data back to the flight control platform 3 via communication unit 2, and the aforementioned anomaly identification model identifies abnormal objects in the reservoir based on the inspection data.

[0069] When generating inspection routes, the priority of key inspection areas should be considered. Key inspection areas can be accident-prone areas, garbage and floating debris accumulation points, and accidents can include illegal sand mining, illegal fishing, swimming and fishing in the wild, changes in river channels, etc. Multiple drone airports can be deployed in key inspection areas, with multiple drones taking turns to operate at high intensity. When a drone's battery is low, it can be charged at the nearest drone airport. When determining the drones to be dispatched, a spatiotemporal constraint scheduling algorithm can be developed to take into account the drone's endurance, prepare for subsequent multi-drone collaboration, and avoid drones with short endurance indicating low battery halfway through the journey.

[0070] In a specific embodiment, the above-mentioned anomaly recognition model utilizes the YOLOv7 network architecture and multispectral image fusion algorithm to develop a floating object detection model based on the YOLOv7 network architecture, which identifies floating objects on the water surface and adapts to different water scenarios; it also uses a multispectral image fusion algorithm (which can be a fusion of PCA and HSV) to identify garbage in the reservoir, improving the accuracy of garbage identification; the above-mentioned abnormal objects include floating objects and garbage.

[0071] The autonomous inspection of this invention can realize the unified scheduling of multiple reservoirs. It can plan reasonable inspection tasks based on the sensing data, geographical location and key monitoring areas of each reservoir, and prioritize the inspection tasks. Combined with the UAV unit 1, it can complete the inspection tasks of multiple reservoirs.

[0072] In addition, drones can also be equipped with a self-testing function. This self-testing function module can be an existing module that detects whether the drone is malfunctioning. If a malfunction is found, a replacement drone that is not in operation can be found to take over the inspection task.

[0073] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A self-inspection system for water conservancy facilities, characterized in that, include: The unmanned aerial vehicle (UAV) unit (1) includes several UAVs and UAV airports adapted to the UAVs. The UAV airports are deployed in a grid pattern in a regional reservoir group. Inspection data is collected by the UAVs. Several flight control platforms (3), each reservoir in the regional reservoir group is equipped with at least one flight control platform (3); the flight control platform (3) calls the anomaly identification model, the flight control platform (3) is used to receive the perception data of the corresponding reservoir, generate the inspection task and the priority of the inspection task, and issue the scheduling instruction, the anomaly identification model identifies abnormal objects in the reservoir according to the inspection data; communication unit (2), the flight control platform (3) communicates with the UAV unit (1) through the communication unit (2); the inspection data collected by the UAV is transmitted back to the flight control platform (3) through the communication unit (2), and the scheduling instruction and inspection task issued by the flight control platform (3) are sent to the UAV; the UAV is coordinated and scheduled through the flight control platform (3), and the UAV executes the inspection task according to the priority.

2. The autonomous inspection system for water conservancy facilities according to claim 1, characterized in that, The communication unit (2) includes: a protocol adaptation layer, which is connected to the flight control platform (3) and the UAV unit (1) and adopts the MAVLink 2.0 protocol; the protocol adaptation layer is used to transmit the inspection tasks and scheduling instructions issued by the flight control platform (3) and the status information of the UAV; a video stream transmission layer, which is connected to the flight control platform (3) and the UAV unit (1) and adopts the RTSP and HLS protocols; the video stream transmission layer is used to transmit the video stream collected by the UAV; a heterogeneous network access layer, which includes dual communication channels of 4G / 5G cellular network and Mesh network; the heterogeneous network access layer is used to allocate communication channels for different types of communication data according to the preset QoS policy; the protocol adaptation layer, the video stream transmission layer and the heterogeneous network access layer are all connected to the flight control platform (3).

3. The autonomous inspection system for water conservancy facilities according to claim 2, characterized in that, The inspection data is transmitted using the SRT protocol.

4. The autonomous inspection system for water conservancy facilities according to claim 2, characterized in that, The flight control platform (3) communicates with at least one UAV through a protocol adaptation layer. The flight control platform (3) is used to receive the status information of the UAV and issue inspection tasks and scheduling instructions.

5. The autonomous inspection system for water conservancy facilities according to claim 1, characterized in that, After issuing scheduling instructions and inspection tasks, the flight control platform sends the scheduling instructions and inspection tasks to the corresponding UAVs and UAV airports through the communication unit (2), and coordinates multiple UAVs and UAV airports to perform inspection tasks.

6. The autonomous inspection system for water conservancy facilities according to claim 1, characterized in that, The drone is equipped with a positioning module, which enables real-time positioning of the drone.

7. A method for autonomous inspection of water conservancy facilities, characterized in that, The inspection method adopts the inspection system described in any one of claims 1 to 6; the inspection method includes the following steps: calling the anomaly identification model on the flight control platform (3); the flight control platform (3) performs the following operations: after receiving the sensing data of the reservoir, generating an inspection route and corresponding priority based on the sensing data; obtaining the current position of each UAV, and determining multiple UAVs adjacent to the inspection route according to the priority of the inspection route and the current position of the UAV, thereby obtaining the dispatched UAV; determining the dispatch instruction based on the dispatched UAV, and generating an inspection task based on the inspection route; the flight control platform (3) sends the dispatch instruction and inspection task to the dispatched UAV through the communication unit (2), and the dispatched UAV collects inspection data along the inspection route; the dispatched UAV transmits the inspection data back to the flight control platform (3) through the communication unit (2), and the anomaly identification model identifies abnormal objects in the reservoir according to the inspection data.

8. The method for autonomous inspection of water conservancy facilities according to claim 7, characterized in that, The anomaly detection model utilizes the YOLOv7 network architecture and a multispectral image fusion algorithm. It identifies floating objects on the water surface based on the YOLOv7 network architecture and identifies garbage in the reservoir based on the multispectral image fusion algorithm. The anomaly objects include floating objects and garbage.

Images