Unmanned aerial vehicle intelligent river patrol method and system based on multi-source data

Abstract

The present application relates to a multi-source data-based unmanned aerial vehicle intelligent river patrol method and system, comprising the following steps: S1: acquiring the geographic range GIS data of the river patrol area, and combining historical events and the latest external warning data to generate a patrol task plan; S2: based on the patrol task plan, the unmanned aerial vehicle takes off according to the plan, collects high-definition video and multispectral data in real time, receives sensing data at the same time, obtains original multi-source data packets with space-time labels, and uploads them to a cloud platform; S3: the cloud platform pre-processes the original multi-source data packets, automatically corrects and fuses the multi-source data, and obtains fused standardized multi-source data; S4: according to the fused standardized multi-source data, an abnormal target is detected, an abnormal event is intelligently discriminated in combination with historical hydrological environmental data, and a space-time abnormal event candidate set is generated; S5: according to the space-time abnormal event candidate set, reinforcement learning is introduced to adjust the unmanned aerial vehicle cruising task. The present application effectively improves the efficiency and reliability of river patrol.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent inspection, and in particular to a method and system for intelligent river inspection by unmanned aerial vehicle based on multi-source data. BACKGROUND

[0002] In the management of broad, dynamic and complex water environments, traditional manual inspection and fixed-point monitoring methods rely on manual and fixed-site information collection methods, which often fail to timely detect sudden pollution, illegal sand mining, floating object accumulation and other abnormal events with strong mobility and scattered distribution. At the same time, personnel-intensive inspection is costly and inefficient, and is easily affected by weather and terrain.

[0003] Existing river inspection and environmental monitoring data are mostly of a single type, such as video inspection or a single water quality indicator, and lack real-time integration of multi-source heterogeneous data (images, environmental sensors, external early warning, etc.), making it difficult to achieve comprehensive intelligent analysis and trend prediction. Traditional inspection routes are usually statically planned, making it difficult to quickly optimize and adjust the inspection tasks and routes in response to sudden high-risk events, resulting in limited inspection resources failing to prioritize the coverage of key spatiotemporal regions that require intervention. SUMMARY

[0004] To solve the above problems, the purpose of the present application is to provide a method and system for intelligent river inspection by unmanned aerial vehicle based on multi-source data, which effectively improves the efficiency and reliability of river inspection.

[0005] To achieve the above purpose, the present application adopts the following technical solutions:

[0006] A method for intelligent river inspection by unmanned aerial vehicle based on multi-source data, comprising the following steps:

[0007] S1: Obtain the geographic range GIS data of the river inspection area, and generate an inspection task plan in combination with historical events and the latest external early warning data;

[0008] S2: Based on the inspection task plan, the unmanned aerial vehicle takes off according to the plan, collects real-time high-definition video and multi-spectral data, receives sensing data, obtains spatiotemporal-labeled raw multi-source data packets, and uploads them to a cloud platform;

[0009] S3: The cloud platform pre-processes the raw multi-source data packets, automatically corrects and fuses the multi-source data, and obtains fused standardized multi-source data;

[0010] S4: Based on the fused standardized multi-source data, detect abnormal targets, intelligently identify abnormal events in combination with historical hydrological environmental data, and generate a spatial-temporal abnormal event candidate set;

[0011] S5: Based on the spatial-temporal abnormal event candidate set, introduce reinforcement learning to adjust the unmanned aerial vehicle cruise task.

[0012] Further, obtain the geographic range GIS data of the river inspection area, and generate the inspection task plan in combination with historical events and the latest external warning data, as follows:

[0013] Import the vector range data of the target river from the river management platform or GIS system, and combine the digital elevation model and infrastructure spatial data to form the basic base map for river spatial analysis;

[0014] Integrate the historical abnormal event library from past unmanned aerial vehicle inspections, ground patrols, and Internet of Things monitoring, including event type, occurrence frequency, impact level, and positioning information, and summarize the changes in historical hydrological and water quality monitoring point indicators, combine the basic base map for river spatial analysis, and assign a risk score to each river area. The risk score is classified, and the priority monitoring and regular monitoring areas in this period are determined;

[0015] Based on the classification results of each river area, combine the available number of unmanned aerial vehicle platforms, base stations, and allowed takeoff and landing points to generate an inspection task plan.

[0016] Further, import the vector range data of the target river from the river management platform or GIS system, and combine the digital elevation model and infrastructure spatial data to form the basic base map for river spatial analysis, as follows:

[0017] Import the vector spatial data of the target river section through the API interface, including the river trunk, tributary polygon / line segment vector, river bank line Shapefile data, and preset node coordinate set;

[0018] Access high-resolution digital elevation model data in raster format, and use block cropping and coordinate projection conversion to ensure alignment with river vector data;

[0019] Collect infrastructure vector data related to river management, including bridges, sluices, water intakes, sewage outlets, and monitoring fences;

[0020] Unify the spatial reference of all vector and raster data to achieve spatial consistency;

[0021] Overlay the river vector, DEM raster, and infrastructure point on the same GIS base map;

[0022] Bind the type and attribute information of the infrastructure to the spatial location of the infrastructure through spatial connection technology to generate an attribute-based river GeoDatabase that can be queried;

[0023] Based on the DEM data, generate the elevation profile and slope distribution along the main stream line, and provide elevation constraints for subsequent flight path planning and risk analysis;

[0024] Using GIS platform, all kinds of data are rendered into river spatial analysis base map according to levels.

[0025] Further, risk score is given to each river section, and the risk score is classified, and the priority monitoring and routine monitoring area in this period is delimited, as follows:

[0026] According to the river spatial analysis base map, the river is divided into several sections S i Integrate historical abnormal events from unmanned aerial vehicle inspection, ground patrol and Internet of Things, including: event type, frequency, impact level and positioning information;

[0027] Each section S i Link the historical indicators of monitoring points in the space range, and normalize the over-standard frequency and abnormal weight; and import short-term external risk warning data and sensitive area information, and map to the section;

[0028] Using linear weighted fusion model, each section S i The comprehensive risk score is :

[0029] ;

[0030] ;

[0031] ;

[0032] ;

[0033] ;

[0034] Wherein, α, β, γ, δ are the weights of each factor; F event (S i ), F wq (S i ), F env (S i ) and F warn (S i ) are the historical abnormal score, water quality abnormal score, environmental sensitivity score and short-term risk score respectively; is the weight of the kth event type; is the impact level of the kth event; and N j,k are the historical occurrence times of the kth event of section S i and section S j ; j is used to index all the divided river sections; w p is the weight of index p; n i,p,over and n j,p,over are the over-standard frequency and abnormal weight of index p of section Si and segment S j The number of times the corresponding monitoring point index p exceeds the standard; d(S i , C) is the distance from the segment S i to the nearest sensitive target set C; D max is the maximum distance in the entire area; Q q is the position of the external early warning source; w q,warn is the weight of different types of early warning; d(S i , Q q ) is the distance from the segment S i to the external early warning source position;

[0035] Sort all R(S i ), and divide the priority monitoring area S H and the regular monitoring area S N using the natural breakpoint method, and output the priority and regular monitoring area list.

[0036] Further, based on the classification results of each river channel segment, combined with the number of available unmanned aerial vehicle platforms, base stations and allowed take-off and landing points, a patrol task plan is generated, as follows:

[0037] Let D be the total number of available unmanned aerial vehicles, for each unmanned aerial vehicle U d , the flight time is T d , the endurance radius is R d , and the maximum range is L d ; Let B = {B1, B2,..., Bm} be the available base stations, and Bm be the mth available base station;

[0038] For the unmanned aerial vehicle U d , assign the segment S d , so that:

[0039] ;

[0040] Where, represents the union set of all unmanned aerial vehicle patrol paths;

[0041] For each U d , select the nearest base station B j to the assigned segment as the take-off and landing point :

[0042] ;

[0043] Where, represents the distance between the base station B j and the center point of the patrol path S i ; center(S i ) is the center point of the segment S i ;

[0044] Then each UAV U d , from , inspects all S d segments, and finally returns to :

[0045] ;

[0046] Constraint: total distance ≤ L d , ;

[0047] is the spatial distance from point to point ; is the n-th visited inspection path in the permutation of inspection sequence, π d (n) is the index of the n-th task in the permutation π d ; denotes the total number of paths (task units) that the d-th UAV needs to inspect;

[0048] The inspection task plan is generated, including the corresponding take-off and landing base station of each UAV, the covered river segment S d , and the inspection order of the starting point-sequential segments-end point of each route.

[0049] Further, based on the current inspection task plan, the UAV takes off according to the plan, collects high-definition video and multi-spectral data in real time, receives perception data at the same time, obtains original multi-source data packets with spatio-temporal annotation, and uploads them to the cloud platform, as follows:

[0050] According to the inspection task plan, the UAV collects high-definition video of the river and the shoreline area at a fixed frame rate in the preset route segment, and the original video stream has GPS synchronous position and angle data, realizing the spatio-temporal annotation of each frame of video.

[0051] According to the inspection task requirements, the multi-spectral camera is turned on synchronously, multi-band images are taken at regular intervals, and the external parameters are recorded, and the video data is timestamped and managed uniformly.

[0052] During the inspection synchronization process, the UAV terminal continuously receives IOT data from the ground perception equipment on the shore and the Internet of Things sensing nodes, which are integrated in real time according to the standard message protocol.

[0053] The UAV terminal summarizes all the original data of this inspection, generates multi-source original data packets with unified spatio-temporal annotation, and uploads them to the cloud platform.

[0054] Further, the cloud platform pre-processes the original multi-source data packets, automatically corrects and fuses the multi-source data, and obtains fused standardized multi-source data, as follows:

[0055] The original multi-source data packet is automatically unpacked in the cloud by task batch, and each period and each type of data is distinguished. The data is automatically archived according to file type, collection time, and spatial label.

[0056] Each sampling point is geographically back-projected through the UAV GPS data and the ground station spatial coordinates. Fuzzy, abnormal exposure, and GPS signal abnormal image frames are detected and removed.

[0057] According to the spatial block and sampling time, all types of data are aligned within a unified grid or segmented range to form the fused standardized multi-source data.

[0058] Further, according to the fused standardized multi-source data, abnormal targets are detected, and historical hydrological environmental data are combined to intelligently identify abnormal events, generating a spatial-temporal abnormal event candidate set, as follows:

[0059] According to the fused standardized multi-source data, based on the pre-trained deep learning detection model YOLOv8, abnormal targets are detected. For each detected abnormal target k, the spatial positioning (x i′,k ,y i′,k ), detection time t i′ , classification label C i′,k , and detection confidence P i′,k of the abnormal target k in the i'th detection are obtained. All detection targets with confidence reaching or exceeding the set threshold are collected in the format of five-tuple (x i′,k ,y i′,k ,t i′ ,C i′,k ,P i′,k ) to form a preliminary abnormal target list.

[0060] IoT sensor data and external hydrological parameter data are combined and compared with statistical thresholds set based on historical analysis or industry standards:

[0061] When the observed value exceeds the threshold, it is determined that there is a water quality anomaly at the current spatiotemporal point. The location (x j′ ,y j′ ), time t, anomaly type, and measurement value of the anomaly occurrence are recorded. All data points identified as abnormal are also summarized to form an environmental monitoring anomaly point list.

[0062] The preliminary abnormal target list and the environmental monitoring anomaly point list within the same time window and in the same region are aligned to form a spatial-temporal abnormal event candidate set.

[0063] Further, according to the spatial-temporal abnormal event candidate set, reinforcement learning is introduced to adjust the UAV cruise task, as follows:

[0064] State space S, state s at each moment t , characterized by: the current position of the UAV, the remaining power, the current route node, the real-time distribution of the space-time anomaly event candidate set, the historical task completion of each river section or region, and the remaining time to the end of the task window;

[0065] Action space A, including the operable action a to the cruise strategy t , including: specifying the next inspection target point of the UAV, dynamically modifying the path order, deciding to return or temporarily stay;

[0066] The reward function is :

[0067] ;

[0068] ;

[0069] ;

[0070] ;

[0071] ;

[0072] ;

[0073] where w c , w r , w e , w d , w b are the weight coefficients of each part of the reward; R coverage is the abnormal coverage reward, R response is the response timeliness reward, R energy is the energy efficiency reward, R distance is the path optimization reward, R balance is the inspection balance reward; E covered is the set of abnormal events newly covered by this action; P e is the confidence of the abnormal event e; S e is the weight of the abnormal event e; E responded is the set of abnormal events responded by this action; Δt e is the time delay from the detection of the abnormal event e to the response of the UAV; λ is the time decay coefficient, controlling the degree of punishment for delayed response; represents the reward decay caused by time delay; E remain is the remaining energy of the UAV after executing the action; E totalis the full power of the UAV; ΔE is the energy consumed by this action; ΔE opt is the energy consumed by the theoretical optimal path; β', γ' are balance parameters; d actual is the length of the actual selected path; d optimal is the length of the shortest path from the current position to the target point; α' is the path deviation penalty coefficient; t current is the current time; t last_visit,i is the last time when the section s i was inspected; μ is the balanced coverage incentive coefficient; Regions is all the river sections to be inspected;

[0074] The inspection scheduling engine perceives the current UAV team state in real time and obtains the latest distribution data of the space-time abnormal event candidate set;

[0075] and based on the current state s t , the RL agent calculates and selects the next optimal action a t using Q-learning;

[0076] The platform automatically generates and issues the adjusted inspection task plan, and the UAV receives and flies according to the new path.

[0077] A UAV intelligent river inspection system based on multi-source data, comprising a processor, a memory and a computer program stored in the memory, wherein when the processor executes the computer program, the steps of the UAV intelligent river inspection method based on multi-source data are executed.

[0078] The present application has the following beneficial effects:

[0079] 1. The present application fully integrates GIS geographic information, historical events and the latest external warning information, combines high-definition video, multispectral images and various sensing data collected by the UAV, and automatically corrects and fuses through the cloud platform to form standardized multi-source data, greatly improving the comprehensiveness and intelligent collaboration level of system monitoring;

[0080] 2. The present application not only can automatically detect water pollution and abnormal target problems based on multi-modal AI algorithms, but also can intelligently distinguish in combination with historical hydrological environment, realize effective identification of multi-dimensional and multi-time-space abnormal events, and through generation of a space-time abnormal event set, real-time push high-risk points and key events, so that the inspection work can quickly focus on key issues, greatly improving the timeliness of abnormal detection and emergency response capability of river and lake inspection;

[0081] 3、The application introduces a reinforcement learning scheduling mechanism, dynamically adjusts the inspection path and task of the unmanned aerial vehicle according to the distribution and attributes of abnormal events, realizes continuous self-optimization of task allocation and route planning, not only improves the coverage rate of high-risk areas, but also reasonably allocates limited unmanned aerial vehicles and energy resources, improves work efficiency, and realizes intelligent unmanned aerial vehicle river patrol and maximization of resource utilization. BRIEF DESCRIPTION OF DRAWINGS

[0082] Figure 1 The method flowchart of the application. DETAILED DESCRIPTION

[0083] The application will be further described in detail below in combination with the drawings and specific embodiments:

[0084] Reference Figure 1 In the embodiment, a multi-source data-based unmanned aerial vehicle intelligent river patrol method is provided, including the following steps:

[0085] S1: Obtain the geographic range GIS data of the river patrol area, and combine historical events and the latest external warning (such as flood season, high pollution) data to generate an inspection task plan;

[0086] S2: Based on the inspection task plan, the unmanned aerial vehicle takes off according to the plan, collects high-definition video and multispectral data in real time, receives sensing data at the same time, obtains original multi-source data packets with space-time labels, and uploads them to a cloud platform;

[0087] S3: The cloud platform pre-processes the original multi-source data packets, automatically corrects and fuses the multi-source data, and obtains fused standardized multi-source data;

[0088] S4: According to the fused standardized multi-source data, detect abnormal targets, intelligently identify abnormal events in combination with historical hydrological environment data, and generate a space-time abnormal event candidate set;

[0089] S5: According to the space-time abnormal event candidate set, introduce reinforcement learning to adjust the unmanned aerial vehicle cruise task.

[0090] In the embodiment, the geographic range GIS data of the river patrol area is obtained, and the historical events and the latest external warning (such as flood season, high pollution) data are combined to generate the inspection task plan this time, which is as follows:

[0091] Import the vector range data (such as river trunk and tributaries, shorelines, node coordinates, etc.) of the target river from the river management platform or GIS system, and combine digital elevation model (DEM) and infrastructure (such as bridges, sluices, water intakes, and sewage outlets) spatial data to form the basic base map for river spatial analysis;

[0092] Integrate the historical abnormal event library from previous UAV inspection, ground patrol and Internet of Things monitoring, including event type (such as water pollution, sewage, illegal sand mining), frequency, impact level and positioning information, and summarize the changes of historical hydrological and water quality monitoring points (such as COD, ammonia nitrogen, algae outbreak records), combined with the basic map of river spatial analysis, give a risk score to each river area, the calculation method usually includes weighting the historical event frequency, weighting the environmental sensitivity evaluation, and fusing factors such as short-term weather and external sudden risks; classify the risk score and divide the priority monitoring and regular monitoring areas in this period;

[0093] Based on the classification results of each river area, combined with the number of available UAV platforms, base stations and allowed take-off and landing points, generate an inspection task plan.

[0094] In this embodiment, the vector range data of the target river is imported from the river management platform or GIS system, combined with the digital elevation model and infrastructure spatial data to form the basic map of river spatial analysis, as follows:

[0095] Import the vector spatial data of the target river section through API interface (such as WFS, WMS, RESTful API), including river main stem, branch polygon / line segment vector, river bank line Shapefile data and preset node (turning point, inflow point, diversion point) coordinate set;

[0096] Access high-resolution digital elevation model data (such as 30m SRTM, 10m domestic DEM, etc.), support raster format (GeoTIFF, IMG), use block cropping and coordinate projection conversion (use CGCS2000 coordinate system) to ensure alignment with river vector data;

[0097] Collect infrastructure vector (point / line / surface) data related to river management, including bridges, sluices, water intakes, sewage outlets and monitoring fences;

[0098] Unify the spatial reference of all vector and raster data (use CGCS2000 projection) to achieve spatial consistency;

[0099] Overlay river vector, DEM raster and infrastructure point on the same GIS base map;

[0100] Bind the type and attribute information (such as number, type, operation and maintenance person) of the infrastructure with its spatial location through spatial connection (Spatial Join) technology to generate a queryable attribute type river GeoDatabase;

[0101] Based on DEM data, generate elevation profile along the main stream line and slope distribution, and provide elevation constraints for subsequent flight path planning / risk analysis.

[0102] Using GIS platform (such as ArcGIS / QGIS / MapGIS), render various data into river spatial analysis base map according to hierarchy.

[0103] In this embodiment, risk score is given to each river section, and the risk score is graded to determine the priority monitoring and routine monitoring area in this period, as follows:

[0104] According to the river spatial analysis base map, the river is divided into several sections S i Integrate historical abnormal events from unmanned aerial vehicle inspection, ground patrol and Internet of Things, including: event type, (such as: water pollution, sewage, illegal sand mining) occurrence frequency, impact level (quantifiable as a score) and positioning information (coordinates and river section attribution);

[0105] Each section S i Link the historical indicators of monitoring points in the space range (such as COD, ammonia nitrogen, algae), and normalize the over-standard frequency and abnormal weight; and import meteorological, environmental and other short-term external risk warning data and sensitive area (drinking water source, protection zone, etc.) information, and map to the section;

[0106] Using linear weighted fusion model, the comprehensive risk score of each section S i is :

[0107] ;

[0108] ;

[0109] ;

[0110] ;

[0111] ;

[0112] Wherein, α, β, γ, δ are the weights of each factor (such as α = 0.4, β = 0.3, γ = 0.2, δ = 0.1, which can be adjusted according to actual situation); F event (S i ), F wq (S i ), F env (S i ) and F warn (S i ) are the historical abnormal score, water quality abnormal score, environmental sensitivity score and short-term risk score respectively; is the weight of the kth event type (high: water pollution; low: sand mining). Impact level (1-5) of the kth event; and N j,k Segment S i and segment S j The number of kth event history occurrences of segment S p The weight of index p (COD, ammonia nitrogen, etc. are weighted according to the harm); n i,p,over and n j,p,over Segment S i and segment S j The number of times the index p of the corresponding monitoring point exceeds the standard; d(S i ,C) is the distance from segment S i to the nearest sensitive target set C; D max The maximum distance in the whole area; Q q The location of the external early warning source; w q,warn The weight of different types of early warning; d(S i ,Q q ) is the distance from segment S i to the external early warning source location; center(S i ) is the center point of segment S i ;

[0113] Sort all R(S i ), and divide the priority monitoring area S H and the regular monitoring area S N using the natural breakpoint method, and output the priority and regular monitoring area list. For example, the top 20% is classified as a priority monitoring area, and the rest is regular.

[0114] In this embodiment, based on the classification results of each river channel area, combined with the number of available unmanned aerial vehicle platforms, base stations and allowed take-off and landing points, a patrol task plan is generated, as follows:

[0115] Let D be the total number of available unmanned aerial vehicles, for each unmanned aerial vehicle U d , the flight time is T d , the endurance radius is R d , and the maximum range is L d ; let B = {B1, B2,..., Bm} be the available base stations, and Bm be the mth available base station;

[0116] For unmanned aerial vehicle U d , assign segment S d , so that:

[0117] ;

[0118] where, represents the union set of the patrol path set of all unmanned aerial vehicles;

[0119] Each U d , select the nearest base station B j as the take-off and landing point :

[0120] ;

[0121] wherein, denotes the distance between base station B j and the center point of the inspection path S i ; center(S i ) is the center point of the section S i ;

[0122] Then each unmanned aerial vehicle U d , starting from , inspects all S d sections and finally returns to :

[0123] ;

[0124] Constraint: total distance ≤ L d , ;

[0125] wherein, is the spatial distance from point to point ; is the nth inspected path in the inspection sequence arrangement, π d (n) is the index of the nth task in the arrangement π d ; denotes the total number of paths (task units) that need to be inspected by the dth unmanned aerial vehicle;

[0126] Generate an inspection task plan, including the corresponding take-off and landing base station of each unmanned aerial vehicle, the covered river section S d , and the inspection order of the starting point-sequential sections-end point of each route.

[0127] In this embodiment, based on the inspection task plan, the unmanned aerial vehicle takes off according to the plan, collects high-definition video and multispectral data in real time, receives perception data at the same time, obtains original multi-source data packets with spatio-temporal annotation and uploads them to the cloud platform, as follows:

[0128] According to the inspection task plan, the unmanned aerial vehicle collects high-definition video of the river channel and the shoreline area at a fixed frame rate in the preset route section, and the original video stream has GPS synchronous position and angle data, realizing the spatio-temporal annotation of each frame of video;

[0129] Synchronously open multi-spectral cameras (such as red light, near-infrared, thermal infrared) according to the requirements of the inspection task, take multi-band images at regular intervals and record the external parameters (latitude, longitude, elevation, attitude), and manage the video data with a unified timestamp;

[0130] During the inspection synchronization process, the UAV terminal continuously receives IOT data (such as water quality sensor flow, pollution alarm data, and weather station instant messages) from the shore ground perception equipment and Internet of Things sensor nodes, which are all integrated in real time according to the standard message protocol (including space-time tags);

[0131] The UAV terminal aggregates all the original data (video stream, multi-spectral, IoT data, auxiliary pictures / voice) of this inspection, generates a multi-source original data package with unified space-time annotation (such as metadata JSON + video / image / sensor binary file), and uploads it to the cloud platform.

[0132] In this embodiment, the cloud platform preprocesses the original multi-source data package, automatically corrects and fuses the multi-source data, and obtains the fused standardized multi-source data, as follows:

[0133] The original multi-source data package is automatically unpacked in the cloud according to the task batch, and the data of each period and each type (such as multi-channel high-definition video stream, multi-spectral photo, and various sensor time series) is automatically archived according to file type, collection time, and spatial tag;

[0134] Through the UAV GPS data and ground station spatial coordinates, the geographic inverse projection is performed on each sampling point (video frame, image, and sensor package); and the image frames with blur, abnormal exposure, and abnormal GPS signal are detected and removed;

[0135] According to the space block and sampling time, all types of data are aligned within the unified grid or segmented range to form the fused standardized multi-source data, and for each river segment and corresponding time window, all video frames, corresponding multi-band images, and real-time IoT monitoring indicators in the region are aggregated to form a spatial data element.

[0136] In this embodiment, according to the fused standardized multi-source data, the abnormal targets are detected, the abnormal events are intelligently identified in combination with historical hydrological environmental data, and a spatial-temporal abnormal event candidate set is generated, as follows:

[0137] According to the fused standardized multi-source data, based on the pre-trained deep learning detection model YOLOv8, typical water area abnormal targets such as pollution, floating objects, sand mining, illegal buildings, and algae blooms are detected, and for each detected abnormal target k, the spatial positioning (x i′,k ,y i′,k ) of the abnormal target k in the i'th detection, the detection time t i′ , and the classification label C are obtained.i′,k , detection confidence P i′,k , all detection targets whose confidence reaches or exceeds a set threshold are collected in the format of (x i′,k , y i′,k , t i′ , C i′,k , P i′,k ) five-tuple, forming a preliminary abnormal target list;

[0138] Combined with IoT sensor data (such as water quality buoys, shore-based water stations, etc.) and external hydrological parameter data (such as rainfall, flow monitoring), for key water quality indicators such as COD, NH33-N, the system obtains all sensor values at each time point and compares them with statistical thresholds set based on historical analysis or industry standards:

[0139] When the observed value exceeds the threshold, it is determined that there is a water quality anomaly at that space-time point, and the location (x j′ , y j′ ), time t, anomaly type (such as high COD), and measurement value of the anomaly occurrence are recorded. All data points that are determined to be abnormal are also summarized to form an environmental monitoring anomaly point list;

[0140] Align the preliminary abnormal target list and the environmental monitoring anomaly point list in the same time window and the same area to form a space-time anomaly event candidate set.

[0141] In this embodiment, according to the space-time anomaly event candidate set, reinforcement learning is introduced to adjust the unmanned aerial vehicle patrol task, specifically as follows:

[0142] State space S, the state s t at each moment is characterized by the following elements: the current position of the unmanned aerial vehicle, the remaining power, the current route node, the real-time distribution of the space-time anomaly event candidate set (including location, risk classification, and confidence), the historical task completion of each river section or area, and the remaining time to the end of the task window;

[0143] Action space A, including the operable actions a t for the patrol strategy, including: specifying the next inspection target point of the unmanned aerial vehicle (preferentially or queuing high-risk points), dynamically modifying the path order (such as jumping / inserting to new abnormal points), deciding to return or temporarily stay;

[0144] The reward function is :

[0145] ;

[0146] ;

[0147] ;

[0148] ;

[0149] ;

[0150] ;

[0151] where w c , w r , w e , w d , w b are the weight coefficients of each part of the reward; R coverage is the abnormal coverage reward, R response is the response timeliness reward, R energy is the energy efficiency reward, R distance is the path optimization reward, R balance is the inspection balance reward; E covered is the set of abnormal events newly covered by this action; P e is the confidence of the abnormal event e (obtained from abnormal detection); S e is the weight of the abnormal event e; E responded is the set of abnormal events responded by this action; Δt e is the time delay from the detection of the abnormal event e to the response of the UAV; λ is the time decay coefficient, which controls the degree of punishment for delayed response; represents the reward decay caused by time delay; E remain is the remaining energy of the UAV after the action is performed; E total is the full energy of the UAV; ΔE is the energy consumed by this action; ΔE opt is the energy that should be consumed under the theoretically optimal path; β', γ' are balance parameters; d actual is the length of the actual selected path; d optimal is the length of the shortest path from the current position to the target point; α' is the path deviation penalty coefficient; t current is the current time; t last_visit,i is the time when the last segment s i was inspected; μ is the balanced coverage incentive coefficient; Regions is all the river segments to be inspected;

[0152] The inspection scheduling engine perceives the current UAV team state in real time and obtains the latest distribution data of the spatial-temporal abnormal event candidate set;

[0153] Based on the current state s t , the RL agent uses Q-learning to calculate and select the next optimal action a tFor example, if a high-confidence abnormal point X is found, the RL decision directly inserts or replaces the current target of the UAV with point X and dynamically plans the shortest or optimal path. If multiple abnormal events occur simultaneously, the RL can optimize target allocation and order according to the weight of the event, spatial adjacency, remaining mileage of the UAV, and the like;

[0154] The platform automatically generates and issues an adjusted inspection task plan, and the UAV receives and flies according to the new path after receiving the plan.

[0155] A UAV intelligent river inspection system based on multi-source data comprises a processor, a memory, and a computer program stored in the memory. When the processor executes the computer program, the steps of the UAV intelligent river inspection method based on multi-source data are executed.

[0156] Those skilled in the art will understand that embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0157] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system), and computer program product according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in a flow or multiple flows and / or blocks Figure 1 The functions specified in a flow or multiple flows and / or blocks

[0158] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product comprising instruction devices that implement the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in a flow or multiple flows and / or blocks Figure 1 The functions specified in a flow or multiple flows and / or blocks

[0159] These computer program instructions can also be loaded into a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 one or more flowcharts and / or blocks

[0160] The above descriptions are only the preferred embodiments of the present application, not intended to limit the present application to other forms described. Any person skilled in the art may make changes or modifications to the above-described technical contents as equivalent embodiments without departing from the technical solutions of the present application. However, any simple modification, equivalent change and modification of the above embodiments according to the technical essence of the present application without departing from the technical solutions of the present application still belongs to the protection scope of the technical solutions of the present application.

Claims

1. A UAV-based intelligent river patrol method based on multi-source data, characterized in that, Includes the following steps: S1: Obtain GIS data of the geographical scope of the river patrol area, and generate a patrol task plan by combining historical events and the latest external early warning data; S2: Based on the inspection mission plan, the UAV took off as planned, collected high-definition video and multispectral data in real time, received sensing data, obtained spatiotemporally labeled raw multi-source data packets and uploaded them to the cloud platform. S3: The cloud platform preprocesses the original multi-source data packets, automatically corrects and merges the multi-source data, and obtains the merged standardized multi-source data. S4: Based on the fused standardized multi-source data, detect abnormal targets, combine historical hydrological and environmental data, intelligently identify abnormal events, and generate a spatial-temporal anomaly event candidate set; S5: Based on the spatial-temporal anomaly event candidate set, reinforcement learning is introduced to adjust the UAV patrol mission; The process involves acquiring GIS data of the river patrol area's geographical scope and combining it with historical events and the latest external early warning data to generate a patrol task plan, as detailed below: Import the vector range data of the target river from the river management platform or GIS system, and combine it with digital elevation model and infrastructure spatial data to form the basic base map for river spatial analysis; It integrates a historical database of abnormal events from previous drone inspections, ground patrols, and IoT monitoring, including event types, frequency of occurrence, impact levels, and location information. It also summarizes the changes in indicators of historical hydrological and water quality monitoring points, and combines the basic base map of river spatial analysis to assign risk scores to each section of the river, classify the risk scores, and delineate priority monitoring and routine monitoring areas for the current cycle. Based on the regional classification results of each river section, and combined with the available number of UAV platforms, base stations, and permitted take-off and landing conditions, an inspection task plan is generated.

2. The UAV intelligent river patrol method based on multi-source data according to claim 1, characterized in that, The vector range data of the target river is imported from the river management platform or GIS system, and combined with the digital elevation model and infrastructure spatial data to form the basic base map for river spatial analysis, as detailed below: Import vector spatial data of the target river section through the API interface, including the main river, tributary polygon / line segment vectors, riverbank shapefile data, and preset node coordinate sets; It accesses high-resolution digital elevation model data, supports raster format, and uses block clipping and coordinate projection transformation to ensure alignment with river vector data; Collect vector data on infrastructure directly related to river management, including bridges, sluice gates, inlets, sewage outlets, and monitoring fences; Spatial reference unification is performed on all vector and raster data to achieve spatial consistency; Overlay river vectors, DEM rasters, and infrastructure locations onto the same GIS base map; By using spatial connectivity technology, the type and attribute information of infrastructure are bound to the spatial location of the infrastructure to generate a queryable attribute-based river GeoDatabase; Based on DEM data, an elevation profile and slope distribution along the main river line are generated, and elevation constraints are provided for subsequent route planning / risk analysis. Using a GIS platform, various types of data are rendered hierarchically into a river spatial analysis base map.

3. The UAV intelligent river patrol method based on multi-source data according to claim 2, characterized in that, Each section of the river is assigned a risk score, and these risk scores are then categorized to determine priority monitoring and routine monitoring zones for the current monitoring cycle, as detailed below: Based on the river spatial analysis base map, the river is divided into several segments S. i It integrates historical anomaly events from drone inspections, ground patrols, and the Internet of Things, including: event type, frequency of occurrence, impact level, and location information; Each segment S i Historical indicators of monitoring points within the spatial range are linked, and the frequency of exceeding standards and the weight of anomalies are statistically analyzed and normalized; short-term external risk warning data and sensitive area information are imported and spatially mapped to segments. A linear weighted fusion model is adopted, with each segment S i The overall risk score is : ； ； ； ； ； Where α, β, γ, and δ are the weights of each factor; F event (S i ), F wq (S i ), F env (S i ) and F warn (S i These are historical anomaly scores, water quality anomaly scores, environmental sensitivity scores, and short-term risk scores, respectively. The weight of the k-th event type; The impact level of the k-th type of event; and N j,k For segment S i and segment S j The number of occurrences of the k-th type of event in history; j is used to index all divided river segments; w p The weight of indicator p; n i,p,over and n j,p,over For segment S i and segment S j The number of times the corresponding monitoring point index p exceeded the standard; d(S) i C) represents the segment S i Distance to the nearest sensitive target set C; D max Q represents the maximum distance across the entire region. q Location of external early warning source; w q,warn Weights for different types of early warnings; d(S) i Q q ) is a segment S i Distance to the location of the external early warning source; For all R(S) i Sort the data and use the natural breakpoint method to prioritize monitoring areas S. H and regular monitoring area S N Output a list of priority and regular monitoring areas.

4. The UAV intelligent river patrol method based on multi-source data according to claim 3, characterized in that, Based on the regional classification results for each river section, and combined with the available number of UAV platforms, base stations, and permitted take-off and landing point conditions, an inspection task plan is generated, as follows: Let D be the total number of available drones, and for each drone U d The flight time is T d The range is R d Maximum range is L d Let B = {B1, B2, ..., Bm} be the available base stations, and Bm be the m-th available base station. UAV d Assigning segments S d , so that: ； in, This represents the union of the sets of inspection paths of all drones; Each U d Select base station B that is closest to the allocated segment. j As a take-off and landing point : ； in, Indicates base station B j Inspection path S i The distance between the center points; center(S) i ) is a segment S i The center point; Then each U drone d ,from Depart, inspect all S d Segmented, ultimately returning to : ； Constraint: Total distance ≤ L d , ; For point Time Spatial distance; For the nth visited inspection path in the inspection sequence, π d (n) represents the permutation of π d The index of the nth task; This represents the total number of paths that the d-th drone needs to inspect; Generate inspection task plans, including the take-off and landing base stations for each drone and the coverage segments of the river section. d The inspection sequence for each route is: starting point - segment - destination.

5. The UAV intelligent river patrol method based on multi-source data according to claim 1, characterized in that, Based on the inspection mission plan, the UAV took off as scheduled, collected high-definition video and multispectral data in real time, received sensing data, obtained spatiotemporally labeled raw multi-source data packets, and uploaded them to the cloud platform, as detailed below: According to the inspection mission plan, the drone will collect high-definition video of the river channel and shoreline area at a fixed frame rate in the preset route segments. The original video stream will have GPS synchronized position and angle data to achieve spatiotemporal annotation of each frame of video. The multispectral camera is activated simultaneously according to the inspection task requirements, and multi-band images are captured at regular intervals and external parameters are recorded. The video data is managed with unified timestamps. During the inspection process, the drone terminal continuously receives IoT data from shore-based ground sensing devices and IoT sensor nodes, which are then integrated in real time according to standard message protocols. The drone collects all the raw data from this inspection, generates a multi-source raw data package with unified spatiotemporal annotations, and uploads it to the cloud platform.

6. The UAV intelligent river patrol method based on multi-source data according to claim 5, characterized in that, The cloud platform preprocesses the original multi-source data packets, automatically corrects and fuses the multi-source data, and obtains fused standardized multi-source data, as detailed below: The original multi-source data packets are automatically unpacked in the cloud according to task batches, distinguishing data of different time periods and types, and the data is automatically archived according to file type, collection time, and spatial tag. Geographic back projection is performed on each sampling point using UAV GPS data and ground base station spatial coordinates; Detect and remove image frames that are blurry, have abnormal exposure, or have abnormal GPS signals; According to spatial blocks and sampling time, all types of data are aligned within a unified grid or segment range to form fused standardized multi-source data.

7. The UAV intelligent river patrol method based on multi-source data according to claim 1, characterized in that, The process involves detecting abnormal targets based on the fused standardized multi-source data, intelligently identifying abnormal events by combining historical hydrological and environmental data, and generating a spatial-temporal anomaly event candidate set, as detailed below: Based on the fused standardized multi-source data, and using the pre-trained deep learning detection model YOLOv8, abnormal targets are detected. For each detected abnormal target k, the spatial location (xi, xi, xi) of the abnormal target k in the i′th detection is obtained. i′,k ,y i′,k ), detection time t i′ Category tag C i′,k Detection confidence level P i′,k All detected targets with a confidence level reaching or exceeding the set threshold will be displayed as (x i′,k ,y i′,k ,t i′ C i′,k ,P i′,k The format of the five-tuple is collected to form a preliminary list of abnormal targets; Combine IoT sensor data with external hydrological parameter data and compare them with statistical thresholds set based on historical analysis or industry standards: When the observed value exceeds the threshold, it is determined that there is a water quality anomaly at the current spatiotemporal point, and the location of the anomaly (x) is recorded. j′ ,y j′ The data points identified as abnormal are also summarized, including time (t), anomaly type, and measured value, to form an environmental monitoring anomaly list. Align the preliminary list of abnormal targets and the list of environmental monitoring anomalies within the same time window and the same area to form a spatial-temporal anomaly event candidate set.

8. The UAV intelligent river patrol method based on multi-source data according to claim 7, characterized in that, The process of adjusting the UAV cruise mission based on a candidate set of spatial-temporal anomalous events using reinforcement learning is as follows: The state space S is the state s at each moment. t It is characterized by the following elements: the current location of the UAV, the remaining battery power, the current flight path node, the real-time distribution of the spatial-temporal anomaly event candidate set, the historical mission completion status of each river segment or region, and the remaining time until the end of the mission window; Action space A, including operable actions a related to the cruise strategy. t This includes: specifying the next inspection target point for the drone, dynamically modifying the path order, and deciding whether to return to base or temporarily stop; The reward function is : ； ； ； ； ； ； Among them, w c w r w e w d w b These are the weighting coefficients for each part of the reward; R coverage For abnormal coverage rewards, R response In response to timeliness rewards, R energy For energy efficiency rewards, R distance Rewards for path optimization, R balance Balanced rewards for inspections; E covered This is the set of newly covered abnormal events in this action; P e S is the confidence level of the abnormal event e; e It is the weight of the abnormal event e; E responded It is the set of abnormal events in this action response; Δt e λ is the time delay from the detection of the abnormal event e to the drone's response; λ is the time decay coefficient, which controls the degree of penalty for the delayed response. E represents the reward decay caused by time delay; remain E is the remaining energy of the drone after performing the action. total The drone is fully charged; ΔE is the energy consumed in this action; ΔE opt It represents the energy consumed under the theoretically optimal path; β′ and γ′ are the balance parameters; d actual It is the actual length of the selected path; d optimal α is the shortest path length from the current location to the target point; α′ is the path deviation penalty coefficient; t current It is the current time; t last_visit,i It is segmented s i The last time it was inspected; μ is the equalization coverage excitation coefficient; Regions are all river sections that need to be inspected; The inspection and scheduling engine can perceive the current status of the drone fleet in real time and obtain the latest distribution data of the spatial-temporal anomaly event candidate set. And based on the current state s t The RL agent uses Q-learning to calculate and select the next optimal action a. t ; The platform automatically generates and distributes the adjusted inspection task plan, and the drone receives it and flies along the new path.

9. A drone-based intelligent river patrol system based on multi-source data, characterized in that, It includes a processor, a memory, and a computer program stored in the memory. When the processor executes the computer program, it specifically performs the steps in the UAV intelligent river patrol method based on multi-source data as described in any one of claims 1-8.

Images