Unmanned aerial vehicle intelligent river patrol method and system based on multi-source data

Abstract

The invention relates to an unmanned aerial vehicle intelligent river patrol method and system based on multi-source data, and the method comprises the following steps: S1, obtaining the geographic range GIS data of a river patrol region, and generating a patrol task plan in combination with historical events and latest external early warning data; s2, based on the inspection task plan, the unmanned aerial vehicle takes off according to the plan, collects high-definition videos and multispectral data in real time, receives sensing data, obtains an original multi-source data packet marked by time and space, and uploads the original multi-source data packet to a cloud platform; s3, the cloud platform preprocesses the original multi-source data packet, automatically corrects and fuses the multi-source data to obtain fused standardized multi-source data; s4, detecting an abnormal target according to the fused standardized multi-source data, intelligently judging an abnormal event in combination with historical hydrological environment data, and generating a space-time sequence abnormal event to-be-selected set; and S5, according to the space-time sequence abnormal event to-be-selected set, reinforcement learning is introduced to adjust the unmanned aerial vehicle cruise task. The river inspection efficiency and reliability are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent inspection, and in particular to an intelligent river patrol method and system based on multi-source data using unmanned aerial vehicles (UAVs). Background Technology

[0002] In the vast, dynamic, and complex management of water environments, traditional methods of manual patrols and fixed-point monitoring, relying on manual and fixed-site information collection, often fail to promptly detect anomalies such as sudden pollution, illegal sand mining, and floating debris accumulation that are highly mobile and dispersed. Furthermore, intensive personnel patrols are costly, inefficient, and easily affected by weather and terrain factors.

[0003] Existing river patrol and environmental monitoring data are mostly single-type data, such as video inspections or single water quality indicators, lacking real-time integration of multi-source heterogeneous data (images, environmental sensors, external early warnings, etc.), making it difficult to achieve comprehensive intelligent analysis and trend prediction. Furthermore, traditional patrol routes are usually statically planned, making it difficult to quickly optimize and adjust patrol tasks and routes in response to sudden high-risk events. This results in limited patrol resources failing to prioritize coverage of key spatiotemporal areas most in need of intervention. Summary of the Invention

[0004] To address the aforementioned problems, the present invention aims to provide an intelligent unmanned aerial vehicle (UAV) river patrol method and system based on multi-source data, which effectively improves the efficiency and reliability of river patrol.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A method for intelligent river patrol using unmanned aerial vehicles (UAVs) based on multi-source data includes the following steps:

[0007] S1: Obtain GIS data of the geographical scope of the river patrol area, and generate a patrol task plan by combining historical events and the latest external early warning data;

[0008] S2: Based on the inspection mission plan, the UAV took off as planned, collected high-definition video and multispectral data in real time, received sensing data, obtained spatiotemporally labeled raw multi-source data packets and uploaded them to the cloud platform.

[0009] S3: The cloud platform preprocesses the original multi-source data packets, automatically corrects and merges the multi-source data, and obtains the merged standardized multi-source data.

[0010] S4: Based on the fused standardized multi-source data, detect abnormal targets, combine historical hydrological and environmental data, intelligently identify abnormal events, and generate a spatial-temporal anomaly event candidate set;

[0011] S5: Based on the spatial-temporal anomaly event candidate set, reinforcement learning is introduced to adjust the UAV cruise mission.

[0012] Furthermore, GIS data of the geographical area of ​​the river patrol zone is obtained, and combined with historical events and the latest external early warning data, a patrol task plan is generated, as follows:

[0013] Import the vector range data of the target river from the river management platform or GIS system, and combine it with digital elevation model and infrastructure spatial data to form the basic base map for river spatial analysis;

[0014] It integrates a historical database of abnormal events from previous drone inspections, ground patrols, and IoT monitoring, including event types, frequency of occurrence, impact levels, and location information. It also summarizes the changes in indicators of historical hydrological and water quality monitoring points, and combines the basic base map of river spatial analysis to assign risk scores to each section of the river, classify the risk scores, and delineate priority monitoring and routine monitoring areas for the current cycle.

[0015] Based on the regional classification results of each river section, and combined with the available number of UAV platforms, base stations, and permitted take-off and landing conditions, an inspection task plan is generated.

[0016] Furthermore, the vector extent data of the target river is imported from the river management platform or GIS system, and combined with the digital elevation model and infrastructure spatial data to form the basic base map for river spatial analysis, as follows:

[0017] Import vector spatial data of the target river section through the API interface, including the main river, tributary polygon / line segment vectors, riverbank shapefile data, and preset node coordinate sets;

[0018] It accesses high-resolution digital elevation model data, supports raster format, and uses block clipping and coordinate projection transformation to ensure alignment with river vector data;

[0019] Collect vector data on infrastructure directly related to river management, including bridges, sluice gates, inlets, sewage outlets, and monitoring fences;

[0020] Spatial reference unification is performed on all vector and raster data to achieve spatial consistency;

[0021] Overlay river vectors, DEM rasters, and infrastructure locations onto the same GIS base map;

[0022] By using spatial connectivity technology, the type and attribute information of infrastructure are bound to the spatial location of the infrastructure to generate a queryable attribute-based river GeoDatabase;

[0023] Based on DEM data, an elevation profile and slope distribution along the main river line are generated, and elevation constraints are provided for subsequent route planning / risk analysis.

[0024] Using a GIS platform, various types of data are rendered hierarchically into a river spatial analysis base map.

[0025] Furthermore, each section of the river is assigned a risk score, and these risk scores are categorized to determine priority monitoring and routine monitoring zones for the current period, as detailed below:

[0026] Based on the river spatial analysis base map, the river is divided into several segments S. i It integrates historical anomaly events from drone inspections, ground patrols, and the Internet of Things, including: event type, frequency of occurrence, impact level, and location information;

[0027] Each segment S i Historical indicators of monitoring points within the spatial range are linked, and the frequency of exceeding standards and the weight of anomalies are statistically analyzed and normalized; short-term external risk warning data and sensitive area information are imported and spatially mapped to segments.

[0028] A linear weighted fusion model is adopted, with each segment S i The overall risk score is :

[0029] ;

[0030] ;

[0031] ;

[0032] ;

[0033] ;

[0034] Where α, β, γ, and δ are the weights of each factor; F event (S i ), F wq (S i ), F env (S i ) and F warn (S i These are historical anomaly scores, water quality anomaly scores, environmental sensitivity scores, and short-term risk scores, respectively. The weight of the k-th event type; The impact level of the k-th type of event; and N j,k For segment S i and segment S j The number of occurrences of the k-th type of event in history; j is used to index all divided river segments; w p The weight of indicator p; n i,p,over and n j,p,over For segment Si and segment S j The number of times the corresponding monitoring point index p exceeded the standard; d(S) i C) represents the segment S i Distance to the nearest sensitive target set C; D max Q represents the maximum distance across the entire region. q Location of external early warning source; w q,warn Weights for different types of early warnings; d(S) i Q q ) is a segment S i Distance to the location of the external early warning source;

[0035] For all R(S) i Sort the data and use the natural breakpoint method to prioritize monitoring areas S. H and regular monitoring area S N Output a list of priority and regular monitoring areas.

[0036] Furthermore, based on the regional classification results for each section of the river, and combined with the available number of UAV platforms, base stations, and permitted take-off and landing point conditions, an inspection task plan is generated, as follows:

[0037] Let D be the total number of available drones, and for each drone U d The flight time is T d The range is R d Maximum range is L d Let B = {B1, B2, ..., Bm} be the available base stations, and Bm be the m-th available base station.

[0038] UAV d Assigning segments S d , so that:

[0039] ;

[0040] in, This represents the union of the sets of inspection paths of all drones;

[0041] Each U d Select base station B that is closest to the allocated segment. j As a take-off and landing point :

[0042] ;

[0043] in, Indicates base station B j Inspection path S i The distance between the center points; center(S) i ) is a segment S i The center point;

[0044] Each U drone d ,from Depart, inspect all S d Segmented, ultimately returning to :

[0045] ;

[0046] Constraint: Total distance ≤ L d , ;

[0047] For point Time Spatial distance; For the nth visited inspection path in the inspection sequence, π d (n) represents the permutation π d The index of the nth task; This represents the total number of paths (task units) that the d-th drone needs to inspect;

[0048] Generate inspection task plans, including the take-off and landing base stations for each drone and the coverage segments of the river section. d The inspection sequence for each route is: starting point - segment - destination.

[0049] Furthermore, based on the inspection mission plan, the UAV took off as scheduled, collecting high-definition video and multispectral data in real time, while simultaneously receiving sensing data, acquiring spatiotemporally labeled raw multi-source data packets, and uploading them to the cloud platform, as detailed below:

[0050] According to the inspection mission plan, the drone will collect high-definition video of the river channel and shoreline area at a fixed frame rate in the preset route segments. The original video stream will have GPS synchronized position and angle data to achieve spatiotemporal annotation of each frame of video.

[0051] The multispectral camera is activated simultaneously according to the inspection task requirements, and multi-band images are captured at regular intervals and external parameters are recorded. The video data is managed with unified timestamps.

[0052] During the inspection process, the drone terminal continuously receives IoT data from shore-based ground sensing devices and IoT sensor nodes, which are then integrated in real time according to standard message protocols.

[0053] The drone collects all the raw data from this inspection, generates a multi-source raw data package with unified spatiotemporal annotations, and uploads it to the cloud platform.

[0054] Furthermore, the cloud platform preprocesses the original multi-source data packets, automatically corrects and fuses the multi-source data, and obtains fused standardized multi-source data, as follows:

[0055] The original multi-source data packets are automatically unpacked in the cloud according to task batches, distinguishing data of different time periods and types, and the data is automatically archived according to file type, collection time, and spatial tag.

[0056] Geographic back projection is performed on each sampling point using UAV GPS data and ground base station spatial coordinates; image frames that are blurry, have abnormal exposure, or have abnormal GPS signals are detected and removed.

[0057] According to spatial blocks and sampling time, all types of data are aligned within a unified grid or segment range to form fused standardized multi-source data.

[0058] Furthermore, based on the fused standardized multi-source data, abnormal targets are detected, and combined with historical hydrological and environmental data, abnormal events are intelligently identified, generating a spatial-temporal anomaly event candidate set, as follows:

[0059] Based on the fused standardized multi-source data, and using the pre-trained deep learning detection model YOLOv8, abnormal targets are detected. For each detected abnormal target k, the spatial location (xi, xi, xi) of the abnormal target k in the i′th detection is obtained. i′,k ,y i′,k ), detection time t i′ Category tag C i′,k Detection confidence level P i′,k All detected targets with a confidence level reaching or exceeding the set threshold will be displayed as (x i′,k ,y i′,k ,t i′ C i′,k ,P i′,k The format of the five-tuple is collected to form a preliminary list of abnormal targets;

[0060] Combine IoT sensor data with external hydrological parameter data and compare them with statistical thresholds set based on historical analysis or industry standards:

[0061] When the observed value exceeds the threshold, it is determined that there is a water quality anomaly at the current spatiotemporal point, and the location of the anomaly (x) is recorded. j′ ,y j′ The data points identified as abnormal are also summarized, including time (t), anomaly type, and measured value, to form an environmental monitoring anomaly list.

[0062] Align the preliminary list of abnormal targets and the list of environmental monitoring anomalies within the same time window and the same area to form a spatial-temporal anomaly event candidate set.

[0063] Furthermore, based on the candidate set of spatial-temporal anomalous events, reinforcement learning is introduced to adjust the UAV patrol mission, as follows:

[0064] The state space S is the state s at each moment. t It is characterized by the following elements: the current location of the UAV, the remaining battery power, the current flight path node, the real-time distribution of the spatial-temporal anomaly event candidate set, the historical mission completion status of each river segment or region, and the remaining time until the end of the mission window;

[0065] Action space A, including operable actions a related to the cruise strategy. t This includes: specifying the next inspection target point for the drone, dynamically modifying the path order, and deciding whether to return to base or temporarily stop;

[0066] The reward function is :

[0067] ;

[0068] ;

[0069] ;

[0070] ;

[0071] ;

[0072] ;

[0073] Among them, w c w r w e w d w b These are the weighting coefficients for each part of the reward; R coverage For abnormal coverage rewards, R response In response to timeliness rewards, R energy For energy efficiency rewards, R distance Rewards for path optimization, R balance Balanced rewards for inspections; E covered This is the set of newly covered abnormal events in this action; P e S is the confidence level of the abnormal event e; e It is the weight of the abnormal event e; E responded It is the set of abnormal events in this action response; Δt e λ is the time delay from the detection of the abnormal event e to the drone's response; λ is the time decay coefficient, which controls the degree of penalty for the delayed response. E represents the reward decay due to time delay; remain E is the remaining energy of the drone after performing the action. totalThe drone is fully charged; ΔE is the energy consumed in this action; ΔE opt It represents the energy consumed under the theoretically optimal path; β′ and γ′ are the balance parameters; d actual It is the actual length of the selected path; d optimal It is the shortest path length from the current location to the target point; α′ is the path deviation penalty coefficient; t current It is the current time; t last_visit,i It is segmented s i The last time it was inspected; μ is the equalization coverage excitation coefficient; Regions are all river sections that need to be inspected;

[0074] The inspection and scheduling engine can perceive the current status of the drone fleet in real time and obtain the latest distribution data of the spatial-temporal anomaly event candidate set.

[0075] And based on the current state s t The RL agent uses Q-learning to calculate and select the next optimal action a. t ;

[0076] The platform automatically generates and distributes the adjusted inspection task plan, and the drone receives it and flies along the new path.

[0077] A drone-based intelligent river patrol system based on multi-source data includes a processor, a memory, and a computer program stored in the memory. When the processor executes the computer program, it specifically performs the steps in the drone-based intelligent river patrol method described above.

[0078] The present invention has the following beneficial effects:

[0079] 1. This invention fully integrates GIS geographic information, historical events and the latest external early warning information, combined with high-definition video, multispectral images and various sensor data collected by UAVs, and automatically corrects and integrates them through a cloud platform to form standardized multi-source data, which greatly improves the comprehensiveness and intelligent collaboration level of the system monitoring.

[0080] 2. This invention can not only automatically detect water pollution and abnormal targets based on multimodal AI algorithms, but also make intelligent judgments by combining historical hydrological environment, so as to effectively identify abnormal events in multiple dimensions and multiple times and spaces. By generating a set of spatial-temporal abnormal events, it can push high-risk points and key events in real time, so that the inspection work can quickly focus on key issues and greatly improve the timeliness of abnormal detection and emergency response capabilities of river and lake inspections.

[0081] 3. This invention introduces a reinforcement learning scheduling mechanism, which dynamically adjusts the inspection path and operation tasks of UAVs based on the distribution and attributes of abnormal events, thereby achieving continuous self-optimization of task allocation and route planning. This not only improves the coverage of high-risk areas, but also rationally allocates limited UAVs and energy resources, thereby improving work efficiency and realizing the intelligentization of UAV river patrol and maximizing resource utilization. Attached Figure Description

[0082] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0083] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0084] refer to Figure 1 In this embodiment, a method for intelligent river patrol using unmanned aerial vehicles (UAVs) based on multi-source data is provided, including the following steps:

[0085] S1: Obtain GIS data of the geographical scope of the river patrol area, and combine it with historical events and the latest external early warning data (such as flood season, high pollution incidence) to generate a patrol task plan;

[0086] S2: Based on the inspection mission plan, the UAV took off as planned, collected high-definition video and multispectral data in real time, received sensing data, obtained spatiotemporally labeled raw multi-source data packets and uploaded them to the cloud platform.

[0087] S3: The cloud platform preprocesses the original multi-source data packets, automatically corrects and merges the multi-source data, and obtains the merged standardized multi-source data.

[0088] S4: Based on the fused standardized multi-source data, detect abnormal targets, combine historical hydrological and environmental data, intelligently identify abnormal events, and generate a spatial-temporal anomaly event candidate set;

[0089] S5: Based on the spatial-temporal anomaly event candidate set, reinforcement learning is introduced to adjust the UAV cruise mission.

[0090] In this embodiment, GIS data of the geographical area of ​​the river patrol region is acquired, and combined with historical events and the latest external early warning data (such as flood season and high pollution incidence) to generate the patrol task plan, as follows:

[0091] Import vector range data (such as main stream and tributaries, shoreline, node coordinates, etc.) of the target river from the river management platform or GIS system, and combine it with the spatial data of digital elevation model (DEM) and infrastructure (such as bridges, sluice gates, water inlets, sewage outlets) to form the basic base map for river spatial analysis;

[0092] It integrates a historical database of abnormal events from previous drone inspections, ground patrols, and IoT monitoring, including event types (such as water pollution, sewage discharge, and illegal sand mining), frequency of occurrence, impact level, and location information. It also summarizes changes in indicators from historical hydrological and water quality monitoring points (such as COD, ammonia nitrogen, and algal bloom records). Combined with a basic base map of river spatial analysis, it assigns a risk score to each river section. The calculation method typically includes weighting based on historical event frequency, environmental sensitivity assessment, and the fusion of factors such as short-term meteorological and external sudden risks. The risk scores are then classified to delineate priority monitoring and routine monitoring areas for the current period.

[0093] Based on the regional classification results of each river section, and combined with the available number of UAV platforms, base stations, and permitted take-off and landing conditions, an inspection task plan is generated.

[0094] In this embodiment, the vector range data of the target river is imported from the river management platform or GIS system, and combined with the digital elevation model and infrastructure spatial data to form a basic base map for river spatial analysis, as follows:

[0095] Import vector spatial data of the target river segment through API interfaces (such as WFS, WMS, RESTful API), including the main river, tributary polygon / line segment vectors, riverbank shapefile data, and preset node (turning point, confluence point, branching point) coordinate sets;

[0096] It can access high-resolution digital elevation model data (such as 30m SRTM, 10m domestic DEM, etc.), supports raster formats (GeoTIFF, IMG), and uses block clipping and coordinate projection transformation (using CGCS2000 coordinate system) to ensure alignment with river vector data;

[0097] Collect vector (point / line / area) data of infrastructure directly related to river management, including bridges, sluice gates, water inlets, sewage outlets, and monitoring fences;

[0098] Spatial reference unification (using CGCS2000 projection) is performed on all vector and raster data to achieve spatial consistency;

[0099] Overlay river vectors, DEM rasters, and infrastructure locations onto the same GIS base map;

[0100] By using spatial join technology, the type and attribute information of infrastructure (such as number, type, and maintenance person in charge) are bound to its spatial location to generate a queryable attribute-based river GeoDatabase;

[0101] Based on DEM data, an elevation profile and slope distribution along the main river line are generated, providing elevation constraints for subsequent route planning and risk analysis.

[0102] Using GIS platforms (such as ArcGIS / QGIS / MapGIS), various types of data are rendered hierarchically into a river spatial analysis base map.

[0103] In this embodiment, each river section is assigned a risk score, the risk scores are graded, and priority monitoring and routine monitoring areas are delineated for the current period, as detailed below:

[0104] Based on the river spatial analysis base map, the river is divided into several segments S. i (Such as automatic or equal-length segmentation of the main line) Integrates historical abnormal events from drone inspections, ground patrols and the Internet of Things, including: event type, (e.g., water pollution, sewage discharge, illegal sand mining) frequency of occurrence, impact level (which can be quantified as a score) and its location information (coordinates and river segment affiliation).

[0105] Each segment S i Historical indicators (such as COD, ammonia nitrogen, and algae) from monitoring points within the spatial range are linked, and the frequency of exceedances and anomaly weights are statistically analyzed and normalized. Short-term external risk warning data such as meteorological and environmental protection data, as well as information on sensitive areas (drinking water sources, protection zones, etc.), are imported and spatially mapped to segments.

[0106] A linear weighted fusion model is adopted, with each segment S i The overall risk score is :

[0107] ;

[0108] ;

[0109] ;

[0110] ;

[0111] ;

[0112] Where α, β, γ, and δ are the weights of each factor (e.g., α=0.4, β=0.3, γ=0.2, δ=0.1, which can be adjusted according to actual conditions); F event (S i ), F wq (S i ), F env (S i ) and F warn (S i These are historical anomaly scores, water quality anomaly scores, environmental sensitivity scores, and short-term risk scores, respectively. The weights for the k-th event type are: high: water pollution; low: sand mining. The impact level of the k-th type of event is (1-5); and N j,k For segment S i and segment S j The number of occurrences of the k-th type of event in history; j is used to index all divided river segments; w p The indicators are weighted as p (COD, ammonia nitrogen, etc. are weighted according to hazard); n i,p,over and n j,p,over For segment S i and segment S j The number of times the corresponding monitoring point index p exceeded the standard; d(S) i C) represents the segment S i Distance to the nearest sensitive target set C; D max Q represents the maximum distance across the entire region. q Location of external early warning source; w q,warn Weights for different types of early warnings; d(S) i Q q ) is a segment S i Distance to the location of the external early warning source; center(S) i ) is a segment S i The center point;

[0113] For all R(S) i Sort the data and use the natural breakpoint method to prioritize monitoring areas S. H and regular monitoring area S N Output a list of priority and regular monitoring areas. For example, the top 20% are classified as priority monitoring areas, and the rest as regular monitoring areas.

[0114] In this embodiment, based on the regional classification results of each river section, and combined with the available number of UAV platforms, base stations, and permitted take-off and landing point conditions, an inspection task plan is generated, as follows:

[0115] Let D be the total number of available drones, and for each drone U d The flight time is T d The range is R d Maximum range is L d Let B = {B1, B2, ..., Bm} be the available base stations, and Bm be the m-th available base station.

[0116] UAV d Assigning segments S d , so that:

[0117] ;

[0118] in, This represents the union of the sets of inspection paths of all drones;

[0119] Each U d Select base station B that is closest to the allocated segment. j As a take-off and landing point :

[0120] ;

[0121] in, Indicates base station B j Inspection path S i The distance between the center points; center(S) i ) is a segment S i The center point;

[0122] Each U drone d ,from Depart, inspect all S d Segmented, ultimately returning to :

[0123] ;

[0124] Constraint: Total distance ≤ L d , ;

[0125] in, For point Time Spatial distance; For the nth visited inspection path in the inspection sequence, π d (n) represents the permutation π d The index of the nth task; This represents the total number of paths (task units) that the d-th drone needs to inspect;

[0126] Generate inspection task plans, including the take-off and landing base stations for each drone and the coverage segments of the river section. d The inspection sequence for each route is: starting point - segment - destination.

[0127] In this embodiment, based on the inspection mission plan, the UAV takes off as planned, collects high-definition video and multispectral data in real time, receives sensing data, obtains spatiotemporally labeled raw multi-source data packets, and uploads them to the cloud platform, as detailed below:

[0128] According to the inspection mission plan, the drone will collect high-definition video of the river channel and shoreline area at a fixed frame rate in the preset route segments. The original video stream will have GPS synchronized position and angle data to achieve spatiotemporal annotation of each frame of video.

[0129] Multispectral cameras (such as red light, near infrared, and thermal infrared) are activated simultaneously according to the inspection task requirements, and multi-band images are captured at fixed intervals and external parameters (latitude, longitude, elevation, and attitude) are recorded. The timestamps are managed in a unified manner with the video data.

[0130] During the inspection process, the drone terminal continuously receives IoT data (such as water quality sensor flow, pollution alarm data, and real-time weather station reports) from shore-based ground sensing devices and IoT sensor nodes, and integrates them in real time according to standard message protocols (including spatiotemporal tags).

[0131] The drone collects all raw data from this inspection (video stream, multispectral, IoT data, auxiliary images / audio), generates a multi-source raw data package with unified spatiotemporal annotation (such as metadata JSON + video / image / sensor binary files), and uploads it to the cloud platform.

[0132] In this embodiment, the cloud platform preprocesses the original multi-source data packets, automatically corrects and fuses the multi-source data, and obtains fused standardized multi-source data, as detailed below:

[0133] The raw multi-source data packets are automatically unpacked in the cloud according to task batches, and the data of different time periods and types (such as multi-channel high-definition video streams, multispectral photos, and time series of various sensors) are distinguished. The data is automatically archived according to file type, acquisition time, and spatial label.

[0134] Using UAV GPS data and ground base station spatial coordinates, geo-projection is performed on each sampling point (video frame, image, sensor packet); blurry, abnormally exposed, and GPS signal abnormal image frames are detected and removed.

[0135] According to spatial blocks and sampling time, all types of data are aligned within a unified grid or segment range to form fused standardized multi-source data. For each river segment and corresponding time window, all video frames, corresponding multi-band images, and real-time IoT monitoring indicators in that area are aggregated to form spatial data elements.

[0136] In this embodiment, based on the fused standardized multi-source data, abnormal targets are detected, and combined with historical hydrological and environmental data, abnormal events are intelligently identified to generate a spatial-temporal anomaly event candidate set, as detailed below:

[0137] Based on the fused standardized multi-source data, and using the pre-trained deep learning detection model YOLOv8, the system detects anomalous targets in typical water areas, including pollution, floating debris, sand mining, illegal construction, and algal blooms. For each detected anomalous target k, the spatial location (xi) of the anomalous target k in the i′th detection is obtained. i′,k ,y i′,k ), detection time t i′ Category tag Ci′,k Detection confidence level P i′,k All detected targets with a confidence level reaching or exceeding the set threshold will be displayed as (x i′,k ,y i′,k ,t i′ C i′,k ,P i′,k The format of the five-tuple is collected to form a preliminary list of abnormal targets;

[0138] By combining IoT sensor data (such as water quality buoys and shore-based water stations) with external hydrological parameter data (such as rainfall and flow monitoring), for key water quality indicators such as COD and NH3-N, the system acquires all sensor data at each time point and compares it with statistical thresholds set based on historical analysis or industry standards.

[0139] When the observed value exceeds the threshold, it is determined that there is a water quality anomaly at that spatiotemporal point, and the location of the anomaly (x) is recorded. j′ ,y j′ The data points identified as abnormal are also summarized, including time (t), anomaly type (e.g., high COD), and measured value, to form an environmental monitoring anomaly list.

[0140] Align the preliminary list of abnormal targets and the list of environmental monitoring anomalies within the same time window and the same area to form a spatial-temporal anomaly event candidate set.

[0141] In this embodiment, reinforcement learning is introduced to adjust the UAV cruise mission based on the candidate set of spatial-temporal anomalous events, as follows:

[0142] The state space S is the state s at each moment. t It is characterized by the following elements: the current location of the UAV, the remaining battery power, the current flight path node, the real-time distribution of the spatial-temporal anomaly event candidate set (including location, risk level and confidence level), the historical mission completion status of each river segment or area, and the remaining time until the end of the mission window.

[0143] Action space A, including operable actions a related to the cruise strategy. t This includes: designating the next inspection target point for the drone (prioritizing or skipping high-risk points), dynamically modifying the path order (such as jumping / skipping to a new anomaly point), and deciding whether to return to base or temporarily stay;

[0144] The reward function is :

[0145] ;

[0146] ;

[0147] ;

[0148] ;

[0149] ;

[0150] ;

[0151] Among them, w c w r w e w d w b These are the weighting coefficients for each part of the reward; R coverage For abnormal coverage rewards, R response In response to timeliness rewards, R energy For energy efficiency rewards, R distance Rewards for path optimization, R balance Balanced rewards for inspections; E covered This is the set of newly covered abnormal events in this action; P e It is the confidence level of the abnormal event e (obtained from anomaly detection); S e It is the weight of the abnormal event e; E responded It is the set of abnormal events in this action response; Δt e λ is the time delay from the detection of the abnormal event e to the drone's response; λ is the time decay coefficient, which controls the degree of penalty for the delayed response. E represents the reward decay due to time delay; remain E is the remaining energy of the drone after performing the action. total The drone is fully charged; ΔE is the energy consumed in this action; ΔE opt It represents the energy consumed under the theoretically optimal path; β′ and γ′ are the balance parameters; d actual It is the actual length of the selected path; d optimal It is the shortest path length from the current location to the target point; α′ is the path deviation penalty coefficient; t current It is the current time; t last_visit,i It is segmented s i The last time it was inspected; μ is the equalization coverage excitation coefficient; Regions are all river sections that need to be inspected;

[0152] The inspection and scheduling engine can perceive the current status of the drone fleet in real time and obtain the latest distribution data of the spatial-temporal anomaly event candidate set.

[0153] And based on the current state s t The RL agent uses Q-learning to calculate and select the next optimal action a. tFor example, if a high-confidence outlier X suddenly appears, RL decides to directly insert or replace the nearest UAV's current target with point X, and dynamically plans the shortest or optimal path. If multiple outliers occur simultaneously, RL can optimize target allocation and order based on parameters such as event weights, spatial adjacency, and remaining UAV mileage.

[0154] The platform automatically generates and distributes the adjusted inspection task plan, and the drone receives it and flies along the new path.

[0155] A drone-based intelligent river patrol system based on multi-source data includes a processor, a memory, and a computer program stored in the memory. When the processor executes the computer program, it specifically performs the steps in the drone-based intelligent river patrol method described above.

[0156] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0157] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0158] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0159] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0160] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A UAV-based intelligent river patrol method based on multi-source data, characterized in that, Includes the following steps: S1: Obtain GIS data of the geographical scope of the river patrol area, and generate a patrol task plan by combining historical events and the latest external early warning data; S2: Based on the inspection mission plan, the UAV took off as planned, collected high-definition video and multispectral data in real time, received sensing data, obtained spatiotemporally labeled raw multi-source data packets and uploaded them to the cloud platform. S3: The cloud platform preprocesses the original multi-source data packets, automatically corrects and merges the multi-source data, and obtains the merged standardized multi-source data. S4: Based on the fused standardized multi-source data, detect abnormal targets, combine historical hydrological and environmental data, intelligently identify abnormal events, and generate a spatial-temporal anomaly event candidate set; S5: Based on the spatial-temporal anomaly event candidate set, reinforcement learning is introduced to adjust the UAV cruise mission.

2. The UAV intelligent river patrol method based on multi-source data according to claim 1, characterized in that, The process involves acquiring GIS data of the river patrol area's geographical scope and combining it with historical events and the latest external early warning data to generate a patrol task plan, as detailed below: Import the vector range data of the target river from the river management platform or GIS system, and combine it with digital elevation model and infrastructure spatial data to form the basic base map for river spatial analysis; It integrates a historical database of abnormal events from previous drone inspections, ground patrols, and IoT monitoring, including event types, frequency of occurrence, impact levels, and location information. It also summarizes the changes in indicators of historical hydrological and water quality monitoring points, and combines the basic base map of river spatial analysis to assign risk scores to each section of the river, classify the risk scores, and delineate priority monitoring and routine monitoring areas for the current cycle. Based on the regional classification results of each river section, and combined with the available number of UAV platforms, base stations, and permitted take-off and landing conditions, an inspection task plan is generated.

3. The UAV intelligent river patrol method based on multi-source data according to claim 2, characterized in that, The vector range data of the target river is imported from the river management platform or GIS system, and combined with the digital elevation model and infrastructure spatial data to form the basic base map for river spatial analysis, as detailed below: Import vector spatial data of the target river section through the API interface, including the main river, tributary polygon / line segment vectors, riverbank shapefile data, and preset node coordinate sets; It accesses high-resolution digital elevation model data, supports raster format, and uses block clipping and coordinate projection transformation to ensure alignment with river vector data; Collect vector data on infrastructure directly related to river management, including bridges, sluice gates, inlets, sewage outlets, and monitoring fences; Spatial reference unification is performed on all vector and raster data to achieve spatial consistency; Overlay river vectors, DEM rasters, and infrastructure locations onto the same GIS base map; By using spatial connectivity technology, the type and attribute information of infrastructure are bound to the spatial location of the infrastructure to generate a queryable attribute-based river GeoDatabase; Based on DEM data, an elevation profile and slope distribution along the main river line are generated, and elevation constraints are provided for subsequent route planning / risk analysis. Using a GIS platform, various types of data are rendered hierarchically into a river spatial analysis base map.

4. The UAV intelligent river patrol method based on multi-source data according to claim 3, characterized in that, Each section of the river is assigned a risk score, and these risk scores are then categorized to determine priority monitoring and routine monitoring zones for the current monitoring cycle, as detailed below: Based on the river spatial analysis base map, the river is divided into several segments S. i It integrates historical anomaly events from drone inspections, ground patrols, and the Internet of Things, including: event type, frequency of occurrence, impact level, and location information; Each segment S i Historical indicators of monitoring points within the spatial range are linked, and the frequency of exceeding standards and the weight of anomalies are statistically analyzed and normalized; short-term external risk warning data and sensitive area information are imported and spatially mapped to segments. A linear weighted fusion model is adopted, with each segment S i The overall risk score is : ； ； ； ； ； Where α, β, γ, and δ are the weights of each factor; F event (S i ), F wq (S i ), F env (S i ) and F warn (S i These are historical anomaly scores, water quality anomaly scores, environmental sensitivity scores, and short-term risk scores, respectively. The weight of the k-th event type; The impact level of the k-th type of event; and N j,k For segment S i and segment S j The number of occurrences of the k-th type of event in history; j is used to index all divided river segments; w p The weight of indicator p; n i,p,over and n j,p,over For segment S i and segment S j The number of times the corresponding monitoring point index p exceeded the standard; d(S) i C) represents the segment S i Distance to the nearest sensitive target set C; D max Q represents the maximum distance across the entire region. q Location of external early warning source; w q,warn Weights for different types of early warnings; d(S) i Q q ) is a segment S i Distance to the location of the external early warning source; For all R(S) i Sort the data and use the natural breakpoint method to prioritize monitoring areas S. H and regular monitoring area S N Output a list of priority and regular monitoring areas.

5. The UAV intelligent river patrol method based on multi-source data according to claim 4, characterized in that, Based on the regional classification results for each river section, and combined with the available number of UAV platforms, base stations, and permitted take-off and landing point conditions, an inspection task plan is generated, as follows: Let D be the total number of available drones, and for each drone U d The flight time is T d The range is R d Maximum range is L d Let B = {B1, B2, ..., Bm} be the available base stations, and Bm be the m-th available base station. UAV d Assigning segments S d , so that: ； in, This represents the union of the sets of inspection paths of all drones; Each U d Select base station B that is closest to the allocated segment. j As a take-off and landing point : ； in, Indicates base station B j Inspection path S i The distance between the center points; center(S) i ) is a segment S i The center point; Each U drone d ,from Depart, inspect all S d Segmented, ultimately returning to : ； Constraint: Total distance ≤ L d , ; For point Time Spatial distance; For the nth visited inspection path in the inspection sequence, π d (n) represents the permutation π d The index of the nth task; This represents the total number of paths that the d-th drone needs to inspect; Generate inspection task plans, including the take-off and landing base stations for each drone and the coverage segments of the river section. d The inspection sequence for each route is: starting point - segment - destination.

6. The UAV intelligent river patrol method based on multi-source data according to claim 1, characterized in that, Based on the inspection mission plan, the UAV took off as scheduled, collected high-definition video and multispectral data in real time, received sensing data, obtained spatiotemporally labeled raw multi-source data packets, and uploaded them to the cloud platform, as detailed below: According to the inspection mission plan, the drone will collect high-definition video of the river channel and shoreline area at a fixed frame rate in the preset route segments. The original video stream will have GPS synchronized position and angle data to achieve spatiotemporal annotation of each frame of video. The multispectral camera is activated simultaneously according to the inspection task requirements, and multi-band images are captured at regular intervals and external parameters are recorded. The video data is managed with unified timestamps. During the inspection process, the drone terminal continuously receives IoT data from shore-based ground sensing devices and IoT sensor nodes, which are then integrated in real time according to standard message protocols. The drone collects all the raw data from this inspection, generates a multi-source raw data package with unified spatiotemporal annotations, and uploads it to the cloud platform.

7. The UAV intelligent river patrol method based on multi-source data according to claim 6, characterized in that, The cloud platform preprocesses the original multi-source data packets, automatically corrects and fuses the multi-source data, and obtains fused standardized multi-source data, as detailed below: The original multi-source data packets are automatically unpacked in the cloud according to task batches, distinguishing data of different time periods and types, and the data is automatically archived according to file type, collection time, and spatial tag. Geographic back projection is performed on each sampling point using UAV GPS data and ground base station spatial coordinates; Detect and remove image frames that are blurry, have abnormal exposure, or have abnormal GPS signals; According to spatial blocks and sampling time, all types of data are aligned within a unified grid or segment range to form fused standardized multi-source data.

8. The UAV intelligent river patrol method based on multi-source data according to claim 1, characterized in that, The process involves detecting abnormal targets based on the fused standardized multi-source data, intelligently identifying abnormal events by combining historical hydrological and environmental data, and generating a spatial-temporal anomaly event candidate set, as detailed below: Based on the fused standardized multi-source data, and using the pre-trained deep learning detection model YOLOv8, abnormal targets are detected. For each detected abnormal target k, the spatial location (xi, xi, xi) of the abnormal target k in the i′th detection is obtained. i′,k ,y i′,k ), detection time t i′ Category tag C i′,k Detection confidence level P i′,k All detected targets with a confidence level reaching or exceeding the set threshold will be displayed as (x i′,k ,y i′,k ,t i′ C i′,k ,P i′,k The format of the five-tuple is collected to form a preliminary list of abnormal targets; Combine IoT sensor data with external hydrological parameter data and compare them with statistical thresholds set based on historical analysis or industry standards: When the observed value exceeds the threshold, it is determined that there is a water quality anomaly at the current spatiotemporal point, and the location of the anomaly (x) is recorded. j′ ,y j′ The data points identified as abnormal are also summarized, including time (t), anomaly type, and measured value, to form an environmental monitoring anomaly list. Align the preliminary list of abnormal targets and the list of environmental monitoring anomalies within the same time window and the same area to form a spatial-temporal anomaly event candidate set.

9. The intelligent river patrol method using unmanned aerial vehicles (UAVs) based on multi-source data according to claim 8, characterized in that, The process of adjusting the UAV cruise mission based on a candidate set of spatial-temporal anomalous events using reinforcement learning is as follows: The state space S is the state s at each moment. t It is characterized by the following elements: the current location of the UAV, the remaining battery power, the current flight path node, the real-time distribution of the spatial-temporal anomaly event candidate set, the historical mission completion status of each river segment or region, and the remaining time until the end of the mission window; Action space A, including operable actions a related to the cruise strategy. t This includes: specifying the next inspection target point for the drone, dynamically modifying the path order, and deciding whether to return to base or temporarily stop; The reward function is : ； ； ； ； ； ； Among them, w c w r w e w d w b These are the weighting coefficients for each part of the reward; R coverage For abnormal coverage rewards, R response In response to timeliness rewards, R energy For energy efficiency rewards, R distance Rewards for path optimization, R balance Balanced rewards for inspections; E covered This is the set of newly covered abnormal events in this action; P e S is the confidence level of the abnormal event e; e It is the weight of the abnormal event e; E responded It is the set of abnormal events in this action response; Δt e λ is the time delay from the detection of the abnormal event e to the drone's response; λ is the time decay coefficient, which controls the degree of penalty for the delayed response. E represents the reward decay due to time delay; remain E is the remaining energy of the drone after performing the action. total The drone is fully charged; ΔE is the energy consumed in this action; ΔE opt It represents the energy consumed under the theoretically optimal path; β′ and γ′ are the balance parameters; d actual It is the actual length of the selected path; d optimal α is the shortest path length from the current location to the target point; α′ is the path deviation penalty coefficient; t current It is the current time; t last_visit,i It is segmented s i The last time it was inspected; μ is the equalization coverage excitation coefficient; Regions are all river sections that need to be inspected; The inspection and scheduling engine can perceive the current status of the drone fleet in real time and obtain the latest distribution data of the spatial-temporal anomaly event candidate set. And based on the current state s t The RL agent uses Q-learning to calculate and select the next optimal action a. t ; The platform automatically generates and distributes the adjusted inspection task plan, and the drone receives it and flies along the new path.

10. A drone-based intelligent river patrol system based on multi-source data, characterized in that, It includes a processor, a memory, and a computer program stored in the memory. When the processor executes the computer program, it specifically performs the steps in the UAV intelligent river patrol method based on multi-source data as described in any one of claims 1-9.

Images