Unmanned aerial vehicle cooperative operation method for water conservancy emergency monitoring scene

Abstract

The invention discloses an unmanned aerial vehicle cooperative operation method suitable for a water conservancy emergency monitoring scene, and the method achieves the global optimization of an inspection route and the balanced distribution of tasks under a multi-constraint condition through the full-link technology fusion of equipment integration, intelligent planning, precise collection, intelligent processing, spatial analysis and GIS visualization. And rapid identification and accurate geographic positioning of specific targets such as trapped persons, water body region segmentation and water line extraction provide accurate data support for emergency disposal. Meanwhile, virtual-real fusion of an unmanned aerial vehicle video stream and a GIS scene is realized through an image and real geographic coordinate matching technology, multi-dimensional data such as a target position, a waterline change and alarm information are visually superposed and presented, and by matching with view interaction and historical playback functions, emergency commanders can grasp the scene situation in real time, the decision difficulty is greatly reduced, and the emergency commanders can conveniently carry out emergency commanders. And the scientificity and efficiency of rescue scheduling and dangerous case disposal are improved.

Description

Technical Field

[0001] This invention relates to the field of water conservancy emergency monitoring technology, and is particularly applicable to a method for collaborative operation of unmanned aerial vehicles (UAVs) in water conservancy emergency monitoring scenarios. Background Technology

[0002] Due to their mobility, flexibility, and rapid deployment, drones have become an indispensable technological tool in disaster emergency monitoring. However, current applications of drones in water conservancy emergency support are limited and often operate independently, lacking a multi-functional, integrated, and highly collaborative technology and platform system. This makes it difficult to meet the comprehensive needs for multi-dimensional and timely information under complex disaster situations. Summary of the Invention

[0003] The purpose of this invention is to provide a method for collaborative operation of unmanned aerial vehicles (UAVs) in water conservancy emergency monitoring scenarios, which solves the problem of difficulty in integrating the functions of multiple UAVs and achieving efficient collaborative work to obtain on-site information in water conservancy emergency monitoring scenarios.

[0004] To achieve the above objectives, the UAV collaborative operation method for water conservancy emergency monitoring scenarios described in this invention includes the following steps: S1. Construct multiple constraints including UAV performance constraints, inspection task constraints, inspection time constraints, inspection space constraints, and environmental conditions; include a multi-objective fitness function with objectives of minimizing total inspection time, maximizing inspection coverage integrity, minimizing total flight distance, and balancing task load; use a genetic algorithm to output the inspection route and inspection task type for each UAV. S2. Select the appropriate drone based on the type of inspection task and conduct flight stability tests; S3. Each drone flies along the inspection route of step S1, collecting video and image data in real time, and simultaneously recording GPS positioning data and flight attitude data. S4. Based on the UAV flight attitude data and camera parameters, the real-time acquired video and image data are preprocessed, mapped to real-world coordinates through a georeferenced transformation method, and overlaid with a GIS base map. An OpenGL rendering engine is then used to generate a monitoring screen that combines virtual and real elements. S5 identifies water areas in real-time acquired video and image data based on a deep learning semantic segmentation model, and obtains the water edge line through an edge detection algorithm. S6, targeting the waterline, employs a change detection algorithm to compare with historical waterline baseline data, calculates the displacement distance of the waterline and the area change of the water body, and completes water level detection and flood evolution analysis; S7 uses the YOLO v8 deep learning model to identify rescue targets in real-time video and image data; based on UAV flight attitude data and camera parameters, it determines the real-world coordinates of the rescue targets through georeferencing transformation and triggers multi-level alarms.

[0005] Furthermore, the performance constraints of the UAVs mentioned in step S1 include the maximum endurance time, maximum flight speed, payload limit, and sensor detection radius of a single UAV; the inspection task constraints include the set of mandatory inspection points, coverage integrity requirements, and target identification types; the inspection time constraints are based on setting a total inspection time limit according to the emergency event level, and converting the total inspection time limit into a maximum task time limit for a single UAV; the inspection space constraints are based on a GIS map and include the latitude and longitude range of the inspection area and the set of no-fly zones; the environmental conditions include wind speed and visibility.

[0006] Furthermore, the genetic algorithm introduces an adaptive crossover mutation mechanism, adopts an integer encoding scheme, and uses flight path time window allocation to ensure that the flight altitude difference of multiple UAVs in the same area is ≥50m and the horizontal distance is ≥100m, thereby avoiding mid-air collisions.

[0007] Furthermore, in step S4, the distortion of video and image data is corrected using the Zhang Zhengming calibration method; augmented reality technology is used to generate a monitoring screen that combines virtual and real elements through coordinate alignment and real-time rendering.

[0008] Furthermore, the S5 step specifically includes: S5.1, acquire real-time video and image data and corresponding flight attitude data of the UAV, and perform image distortion correction based on camera parameters; S5.2 performs adaptive histogram equalization on the corrected video and image data to filter out environmental noise; S5.3 uses a pre-trained U-Net++ model to output a binarized mask for the water region; S5.4; Calculate the Intersection over Union (IoU) of the mask, and filter out valid results with IoU ≥ 0.88; Perform morphological closure operation on the mask to fill small voids in the water area, remove isolated noise points at the edges, and segment the water area; S5.5 uses Canny edge detection, sets the Canny algorithm parameters, scans the gradient map to extract discrete water edge pixel sets, retains continuous edge segments with a length ≥ 50 pixels, and obtains the water edge. S5.6; The extracted waterline data is mapped to real-world coordinates using a georeferenced transformation method, overlaid with a GIS base map, and then an OpenGL rendering engine is used to generate a monitoring screen that combines virtual and real elements.

[0009] Furthermore, S7 includes the following steps: inputting real-time acquired video and image data into the optimized YOLO v8 deep learning model to identify water conservancy targets; outputting the bounding box coordinates, category labels, and confidence scores of the water conservancy targets; extracting the center pixel coordinates of the bounding box of the water conservancy targets, and calculating the real-world coordinates of the water conservancy targets through georeferencing transformation by combining UAV flight attitude data and camera parameters; displaying the target location and category information in the form of icons and labels in the GIS scene, and simultaneously triggering multi-level alarms.

[0010] Furthermore, the alarm includes alarm type, coordinates, and timestamp.

[0011] Furthermore, based on the real-world coordinates of the S7 step water conservancy target or the instructions of the emergency command personnel, determine whether to activate the megaphone / searchlight; remotely activate the megaphone through the drone flight control system to play preset rescue instructions or real-time voice commands; turn on the searchlight, adjust the illumination angle, and focus on the target area or rescue operation area; confirm the megaphone / lighting effect through the transmitted video stream, and adjust the equipment working status according to on-site feedback until the rescue mission is completed.

[0012] Furthermore, the integer encoding scheme integrates "inspection area division + route planning + task allocation" into a single encoding, achieving synchronous representation of routes and tasks.

[0013] Furthermore, the multi-objective fitness function employs a weighted summation of multiple objectives.

[0014] The advantage of this invention lies in the integration of the entire technology chain, including "equipment integration, intelligent planning, precise data collection, intelligent processing, spatial analysis, and GIS visualization," which achieves the following technical effects: (1) High-efficiency collaborative monitoring: This invention is based on multi-UAV task planning with improved genetic algorithm, which breaks through the limitations of single UAV operation. Under multiple constraints, it realizes global optimization of inspection route and balanced task allocation. Combined with dual-mode transmission to ensure real-time data transmission, it greatly improves the coverage and timeliness of emergency monitoring.

[0015] (2) Precise and intelligent perception: This invention combines deep learning models with photogrammetry technology to achieve rapid identification and precise geographic positioning of specific targets such as trapped personnel. At the same time, it completes water area segmentation and water boundary line extraction, and combines change detection algorithms to quantify dynamic changes such as flood spread, providing precise data support for emergency response.

[0016] (3) Intuitive decision support: This invention achieves virtual-real fusion of UAV video stream and GIS scene through image and real geographic coordinate matching technology, and intuitively overlays and presents multi-dimensional data such as target location, waterline changes, and alarm information. Combined with view interaction and historical playback functions, it enables emergency command personnel to grasp the on-site situation in real time, greatly reduces the difficulty of decision-making, and improves the scientificity and efficiency of rescue dispatch and emergency response. Attached Figure Description

[0017] Figure 1 This is a flowchart of the UAV collaborative operation method for water conservancy emergency monitoring scenarios according to the present invention. Detailed Implementation

[0018] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0019] like Figure 1 As shown, the UAV collaborative operation method for water conservancy emergency monitoring scenarios according to the present invention includes the following steps: Step S1 obtains the inspection routes and inspection task types for each drone, mainly including the following steps: S1.1, construct multiple constraints including UAV performance constraints, inspection task constraints, inspection time constraints, inspection space constraints, and environmental conditions.

[0020] Drone performance constraints include the maximum endurance of a single drone. Maximum flight speed Maximum payload Sensor detection radius ; The inspection task constraints include the set of mandatory inspection points P={p1,p2,...,pn} and coverage integrity requirements. Target recognition type .

[0021] The inspection time constraint is the total inspection time limit set based on the emergency event level. The total inspection time limit is converted into a maximum task time limit for a single drone. That is, the maximum task time limit for the i-th drone. , where k is the load distribution coefficient among multiple drones.

[0022] The inspection space constraints are based on a GIS map and include the latitude and longitude range of the inspection area. The set of no-fly zones, F = {f1, f2, ..., fm}, is represented by polygon coordinates.

[0023] The environmental conditions include wind speed. ,visibility Used to correct flight speed .

[0024] S1.2 is a multi-objective fitness function that includes objectives F1 (minimizing total inspection time), F2 (maximizing inspection coverage integrity), F3 (minimizing total flight distance), and F4 (balancing task load).

[0025] The multi-objective fitness function integrates multiple objectives through weighted summation to comprehensively optimize the core objective: F(Chrom)=α⋅F1+β⋅F2+γ⋅F3+δ⋅F4 F1, F2, F3, and F4 are the target terms, all of which are normalized to the interval [0, 1] to avoid the impact of differences in dimensions on the optimization effect; α, β, γ, and δ are weighting coefficients that satisfy α + β + γ + δ = 1. The weighting coefficients are adjusted according to the emergency scenario. For example, in flood rescue scenarios, timeliness is prioritized, and α is set to 0.4 to 0.5; in river inspection scenarios, coverage integrity is prioritized, and β is set to 0.4 to 0.5.

[0026] in: F1: Minimize total inspection time: For the flight distance of drone i, Its correction speed, Let k be the target detection time for grid k.

[0027] F2: Objective to maximize coverage integrity: The number of mandatory inspection points to be covered. This represents the total number of locations that must be checked.

[0028] F3: Minimize total flight distance: F4: Task load balancing objective: The standard deviation of the mission time for each drone is given. This represents the average task time.

[0029] S1.3 employs a genetic algorithm to output the inspection routes and task types for each UAV. The genetic algorithm incorporates an adaptive crossover and mutation mechanism, uses an integer encoding scheme, and allocates routes via time windows to ensure that the altitude difference between multiple UAVs in the same area is ≥50m and the horizontal distance is ≥100m, thus avoiding mid-air collisions.

[0030] This invention develops an improved genetic algorithm based on Python combined with NumPy and SciPy frameworks. It uses integer encoding to synchronously encode the inspection area grid cells, inspection sequence, and UAV number, constructing a multi-objective fitness function with the goals of minimizing total inspection time, maximizing coverage integrity, minimizing flight distance, and balancing task load. An adaptive crossover and mutation mechanism is introduced, and the synchronous optimization of flight routes and tasks is achieved through 100-200 initial populations and 50-100 iterations. Simultaneously, a multi-UAV flight route time window allocation mechanism is embedded to ensure safe flight spacing, and a dynamic replanning module is designed to handle equipment failures or sudden environmental changes. Ultimately, it achieves rapid and reasonable allocation of inspection routes and tasks for each UAV under multiple constraints, achieving efficient collaborative operation. The specific implementation steps are as follows: 1. An integer coding scheme is adopted to integrate the "inspection area division + route planning + task allocation" into a single code: The inspection area is divided into M×N grid units of 100m×100m, each numbered 1,2,...,M×N; the population structure is Chrom=[g1,g2,...,gK,u1,u2,...,uK], where gk is the number of the k-th inspection grid unit. The number of the drone responsible for this unit (e.g.) (∈{1,2,...,U}, where U is the total number of UAVs), to achieve synchronous representation of routes and tasks.

[0031] 2. Initial population generation Randomly generated A population of 100-200 groups must be generated while satisfying the following conditions: each grid cell is assigned at least one drone; the total flight distance of a single drone within its corresponding grid cell must be [missing information]. The chromosome has no no-fly zone grid cell numbering to ensure the feasibility of the initial solution.

[0032] 3. Input quantization constraint parameters and initialize 100-200 groups of individuals. Use the roulette wheel selection method. The probability of an individual being selected is: The top 20% of individuals in terms of fitness are retained and directly enter the next generation, improving convergence efficiency.

[0033] After 50 to 100 iterations, the system outputs the inspection waypoint sequence, responsible area, and task type for each drone. A time window allocation mechanism ensures that the drone's flight altitude difference is ≥50m and horizontal distance is ≥100m, generating a conflict-free collaborative operation plan and simultaneously outputting emergency backup routes.

[0034] S2. Based on the inspection task type, select a suitable drone and conduct flight stability tests. This involves selecting compatible equipment such as high-definition cameras, thermal infrared sensors, loudspeakers, and searchlights, ensuring the total weight of the carried equipment does not exceed the drone's payload. Conduct collaborative tests on multiple drones and their carried equipment to verify synchronized operation and ensure there are no functional conflicts or signal interference.

[0035] S3. Import the waypoint sequence of the inspection route from step S1 into the UAV flight control system, and perform route pre-calibration using GPS / BeiDou dual-mode positioning to ensure waypoint deviation ≤2m. Each UAV flies according to the inspection route of step S1, collecting 4K video streams and image data in real time, and simultaneously recording GPS positioning data, including latitude and longitude, elevation, and flight attitude data, including roll angle, pitch angle, and yaw angle. All data is appended with millisecond-level timestamps.

[0036] S4. Based on the UAV flight attitude (POS) data and camera parameters (focal length, pixel size, principal point coordinates), the camera distortion is corrected using the Zhang Zhengming calibration method to ensure data accuracy.

[0037] After preprocessing the real-time acquired video and image data, based on the principles of photogrammetry, the preprocessed video and image data are mapped to real-world coordinates using a georeferenced transformation method, following the steps of "pixel → image physical coordinates → camera coordinate system → geodetic coordinate system (WGS84)". The video and image data mapped to real-world coordinates are then overlaid with a high-precision GIS base map, and an OpenGL rendering engine is used to achieve coordinate alignment, generating a monitoring screen that combines virtual and real elements.

[0038] S5 uses a deep learning semantic segmentation model to identify water areas in real-time video and image data and obtains the water edge line through an edge detection algorithm.

[0039] S5.1 acquires real-time video and image data (including high-definition visible light and thermal infrared images) and corresponding flight attitude data (including GPS positioning: latitude B, longitude L, altitude H; attitude data: roll angle ϕ, pitch angle ω, yaw angle κ) from the UAV. Millisecond-level timestamp matching ensures a one-to-one correspondence between each image frame and real-time flight parameters, avoiding coordinate transformation misalignment. Based on camera calibration results, the system utilizes positive image radial and tangential distortion of camera parameters to eliminate the influence of lens optical errors on pixel coordinates.

[0040] S5.2 performs adaptive histogram equalization on the corrected video and image data and uses Gaussian filtering to remove environmental noise, ensuring clear water area features.

[0041] S5.3 uses a pre-trained U-Net++ model, inputting the pre-processed image into the model and outputting a binarized mask of the water area.

[0042] S5.4 Calculate the Intersection over Union (IoU) of the mask and filter out valid results with IoU ≥ 0.88; perform morphological closure operations on the mask (e.g., 3×3 expansion kernel → 3×3 erosion kernel) to fill small voids in the water area, remove isolated noise points at the edges, and segment the water area.

[0043] S5.5 uses Canny edge detection. The Canny algorithm parameters are set, the gradient map is scanned to extract discrete water edge pixel sets, and continuous edge segments with a length of ≥50 pixels are retained to obtain the water edge.

[0044] S5.6 maps the extracted waterline data to real-world coordinates using a georeference transformation method, converting pixel coordinates into geodetic coordinates as geographic elements, and overlays them with the GIS base map. Using an OpenGL rendering engine, display styles are set and overlaid onto the GIS scene to generate a monitoring screen that combines virtual and real elements.

[0045] S6 employs a change detection algorithm for the waterline, compares it with historical waterline baseline data, calculates the displacement distance of the waterline and the area change of the water body, and completes water level detection and flood evolution analysis.

[0046] S7 uses the YOLO v8 deep learning model to identify rescue targets in real-time video and image data; based on UAV flight attitude data and camera parameters, it determines the real-world coordinates of the rescue targets through georeferencing transformation and triggers multi-level alarms.

[0047] Real-time video and image data are input into an optimized YOLO v8 deep learning model to identify water conservancy targets, such as trapped people and houses. The model outputs the bounding box coordinates, category labels, and confidence scores for these targets. The model extracts the center pixel coordinates of the bounding boxes and, combined with UAV flight attitude data and camera parameters, calculates the real-world coordinates of the targets through georeferencing transformation. The target location and category information are displayed in a GIS scene using icons and labels, simultaneously triggering multi-level alarms. Alarm content includes alarm type, coordinates, and timestamp.

[0048] Finally, based on the real-world coordinates of the S7 step water conservancy target or the instructions of the emergency command personnel, determine whether to activate the megaphone / searchlight; remotely activate the megaphone through the drone flight control system to play preset rescue instructions or real-time voice commands; turn on the searchlight, adjust the illumination angle, and focus on the target area or rescue operation area; confirm the megaphone / lighting effect through the transmitted video stream, and adjust the equipment working status according to on-site feedback until the rescue mission is completed.

Claims

1. A method for collaborative operation of unmanned aerial vehicles (UAVs) in water conservancy emergency monitoring scenarios, characterized in that, Includes the following steps: S1. Construct multiple constraints including UAV performance constraints, inspection task constraints, inspection time constraints, inspection space constraints, and environmental conditions; include a multi-objective fitness function with objectives of minimizing total inspection time, maximizing inspection coverage integrity, minimizing total flight distance, and balancing task load; use a genetic algorithm to output the inspection route and inspection task type for each UAV. S2. Select the appropriate drone based on the type of inspection task and conduct flight stability tests; S3. Each drone flies along the inspection route of step S1, collecting video and image data in real time, and simultaneously recording GPS positioning data and flight attitude data. S4. Based on the UAV flight attitude data and camera parameters, the real-time acquired video and image data are preprocessed, mapped to real-world coordinates through a georeferenced transformation method, and overlaid with a GIS base map. An OpenGL rendering engine is then used to generate a monitoring screen that combines virtual and real elements. S5 identifies water areas in real-time acquired video and image data based on a deep learning semantic segmentation model, and obtains the water edge line through an edge detection algorithm. S6, targeting the waterline, employs a change detection algorithm to compare with historical waterline baseline data, calculates the displacement distance of the waterline and the area change of the water body, and completes water level detection and flood evolution analysis; S7 uses the YOLO v8 deep learning model to identify rescue targets in real-time video and image data; based on UAV flight attitude data and camera parameters, it determines the real-world coordinates of the rescue targets through georeferencing transformation and triggers multi-level alarms.

2. The UAV collaborative operation method for water conservancy emergency monitoring scenarios according to claim 1, characterized in that: The performance constraints of the UAVs mentioned in step S1 include the maximum endurance, maximum flight speed, payload limit, and sensor detection radius of a single UAV; the inspection task constraints include the set of mandatory inspection points, coverage integrity requirements, and target identification types; the inspection time constraints are based on setting a total inspection time limit according to the emergency event level, and converting the total inspection time limit into a maximum task time limit for a single UAV; the inspection space constraints are based on a GIS map and include the latitude and longitude range of the inspection area and the set of no-fly zones; the environmental conditions include wind speed and visibility.

3. The UAV collaborative operation method for water conservancy emergency monitoring scenarios according to claim 1, characterized in that: The genetic algorithm introduces an adaptive crossover mutation mechanism, adopts an integer encoding scheme, and uses flight path time window allocation to ensure that the flight altitude difference of multiple UAVs in the same area is ≥50m and the horizontal distance is ≥100m, thereby avoiding mid-air collisions.

4. The UAV collaborative operation method for water conservancy emergency monitoring scenarios according to claim 1, characterized in that: In step S4, the distortion of video and image data is corrected using the Zhang Zhengming calibration method; augmented reality technology is used to generate a monitoring screen that combines virtual and real elements through coordinate alignment and real-time rendering.

5. The method for collaborative operation of unmanned aerial vehicles (UAVs) in water conservancy emergency monitoring scenarios according to claim 1, characterized in that, Step S5 specifically includes: S5.1, acquire real-time video and image data and corresponding flight attitude data of the UAV, and perform image distortion correction based on camera parameters; S5.2 performs adaptive histogram equalization on the corrected video and image data to filter out environmental noise; S5.3 uses a pre-trained U-Net++ model to output a binarized mask for the water region; S5.4; Calculate the Intersection over Union (IoU) of the mask, and filter out valid results with IoU ≥ 0.88; Perform morphological closure operation on the mask to fill small voids in the water area, remove isolated noise points at the edges, and segment the water area; S5.5 uses Canny edge detection, sets the Canny algorithm parameters, scans the gradient map to extract discrete water edge pixel sets, retains continuous edge segments with a length ≥ 50 pixels, and obtains the water edge. S5.6; The extracted waterline data is mapped to real-world coordinates using a georeferenced transformation method, overlaid with a GIS base map, and then an OpenGL rendering engine is used to generate a monitoring screen that combines virtual and real elements.

6. The method for collaborative operation of unmanned aerial vehicles (UAVs) in water conservancy emergency monitoring scenarios according to claim 1, characterized in that, S7 includes the following steps: inputting real-time acquired video and image data into the optimized YOLO v8 deep learning model to identify water conservancy targets; outputting the bounding box coordinates, category labels, and confidence scores of the water conservancy targets; extracting the center pixel coordinates of the bounding box of the water conservancy targets, and calculating the real-world coordinates of the water conservancy targets through georeferencing transformation by combining UAV flight attitude data and camera parameters; displaying the target location and category information in the GIS scene in the form of icons and labels, and simultaneously triggering multi-level alarms.

7. The UAV collaborative operation method for water conservancy emergency monitoring scenarios according to claim 1, characterized in that: The alarm includes alarm type, coordinates, and timestamp.

8. The UAV collaborative operation method for water conservancy emergency monitoring scenarios according to claim 1, characterized in that: Based on the real-world coordinates of the S7 step water conservancy target or instructions from emergency command personnel, determine whether to activate the megaphone / searchlight; remotely activate the megaphone via the drone flight control system to play preset rescue instructions or real-time voice commands; turn on the searchlight, adjust the illumination angle, and focus on the target area or rescue operation area; confirm the megaphone / lighting effect through the transmitted video stream, and adjust the equipment's working status according to on-site feedback until the rescue mission is completed.

9. The UAV collaborative operation method for water conservancy emergency monitoring scenarios according to claim 3, characterized in that: The integer encoding scheme integrates "inspection area division + route planning + task allocation" into a single encoding, enabling synchronous representation of routes and tasks.

10. The method for collaborative operation of unmanned aerial vehicles (UAVs) in water conservancy emergency monitoring scenarios according to claim 1, characterized in that: The multi-objective fitness function employs a weighted summation of multiple objectives.

Images