A Ground-Sensing Event-Driven Automated Collaborative Verification Method and System for Highway Slopes Using Unmanned Aerial Vehicles

Abstract

This invention relates to the field of remote sensing technology and discloses an automated collaborative verification method and system for highway slopes using UAVs based on ground-sensor event-driven methods. The method includes: acquiring ground sensor monitoring data and basic geographic information data of the slope; triggering alarm events by threshold judgment of the ground sensor monitoring data; generating regions of interest for the slope based on the basic geographic information data of the slope and matching modeling parameters to obtain a terrain-simulating modeling flight path; calculating the effective geometric parameters and average normal vector of the slope surface based on the three-dimensional coordinates of the sensors that triggered the alarm, constructing a local coordinate system for the slope surface, and obtaining a gridded emergency inspection flight path; encapsulating the terrain-simulating modeling flight path and the gridded emergency inspection flight path, and distributing the flight path tasks to the UAV hangar through a cloud application programming interface; the UAV executing the flight path tasks and transmitting back inspection data, performing multi-dimensional analysis on the inspection data, and completing a ground-air linkage closed-loop decision-making process.

Description

Technical Field

[0001] This invention relates to the field of remote sensing technology, specifically to an automated collaborative verification method and system for highway slopes using unmanned aerial vehicles (UAVs) based on ground sensing events. Background Technology

[0002] Monitoring geological hazards on highway slopes is a crucial aspect of road maintenance and operation. Currently, both ground-based monitoring and aerial inspection are well-developed single-technical methods. Ground-based monitoring uses ground sensors to collect key parameters such as slope displacement in real time, providing numerical data support for slope hazard early warning. Aerial inspection relies on unmanned aerial vehicles (UAVs) to conduct on-site slope inspections, acquiring on-site image data for direct visual verification of slope defects. Both methods are widely used in highway slope hazard monitoring.

[0003] In existing technologies, ground-based sensor monitoring systems and UAV-based airborne patrol systems operate independently, lacking an automated collaborative verification mechanism driven by ground-sensor events. When ground sensor monitoring data reaches a threshold and triggers a slope alarm event, the system cannot automatically complete UAV flight path planning, patrol task distribution, and hazard assessment through multi-source ground-air data fusion based on this event. This makes the verification work after a slope disaster alarm highly dependent on manual intervention and operation, failing to achieve automated collaboration between ground-sensor monitoring and UAV patrols. Ultimately, this results in low verification response efficiency after a slope disaster alarm, making it difficult to quickly verify and determine slope hazards. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention proposes an automated collaborative verification method and system for highway slopes using unmanned aerial vehicles (UAVs) based on ground sensing events, in order to solve the aforementioned technical problems.

[0005] Firstly, a method for automated collaborative verification of highway slopes using unmanned aerial vehicles (UAVs) based on ground-sensor event-driven approaches is provided, including:

[0006] Acquire ground sensor monitoring data and basic geographic information data of slopes;

[0007] The ground sensor monitoring data is used to trigger an alarm event by threshold judgment. Based on the basic geographic information data of the slope, a region of interest for the slope is generated and the modeling parameters are matched to obtain the terrain modeling route.

[0008] Based on the sensor's three-dimensional coordinates that trigger the alarm, the effective geometric parameters and average normal vector of the slope are calculated, a local coordinate system of the slope is constructed, and a gridded emergency inspection route is obtained.

[0009] The terrain-simulating modeling route and the gridded emergency inspection route are encapsulated, and the route tasks are distributed to the UAV hangar through a cloud application programming interface.

[0010] The UAV executes the route mission and transmits inspection data back, performs multi-dimensional analysis on the inspection data, and completes ground-air linkage closed-loop decision-making.

[0011] Furthermore, threshold judgment is performed on the ground sensor monitoring data to trigger alarm events. Based on the slope's basic geographic information data, a region of interest for the slope is generated and matched with modeling parameters to obtain a terrain-simulating modeling flight path, including:

[0012] Threshold judgment is performed on the monitoring data from the ground sensors to trigger a slope alarm event;

[0013] Based on the sensor identifier, retrieve the sensor's three-dimensional coordinates and the geographic information parameters of the slope to which it belongs, and construct the region of interest for the slope;

[0014] The wide-area terrain model data is called and the local terrain model data of the slope is obtained by clipping the region of interest of the slope. The modeling parameters are matched according to the warning level of the slope alarm event to obtain the terrain modeling route.

[0015] Furthermore, based on the sensor identifier, the three-dimensional coordinates of the sensor and the geographic information parameters of the slope to which it belongs are retrieved to construct the region of interest for the slope, including:

[0016] Based on the sensor identifier, retrieve the sensor's three-dimensional coordinates and the slope length, aspect, and gradient information of the slope to which it belongs from the sensor equipment management database;

[0017] Calculate the projection point of the sensor's three-dimensional coordinates onto the centerline of the road network to obtain the roadbed tangent vector and azimuth at the projection point;

[0018] An initial regular region is constructed using the sensor's three-dimensional coordinates as the center and the slope length. This initial regular region is then rotated around the sensor's three-dimensional coordinates until it is parallel to the road direction, outputting the slope's region of interest with a spatial coordinate system.

[0019] Furthermore, by calling wide-area terrain model data and clipping it through the region of interest of the slope, local terrain model data of the slope is obtained. Modeling parameters are matched according to the warning level of the slope alarm event to obtain the terrain modeling flight path, including:

[0020] Retrieve wide-area digital surface model data, and use the vector file of the slope region of interest to spatially clip the wide-area digital surface model to obtain local slope terrain model data;

[0021] The local terrain model data of the slope is imported into the flight path planning system. The modeling parameters are configured according to the warning level of the slope alarm event to obtain a terrain-following modeling flight path that matches the slope terrain.

[0022] Furthermore, based on the three-dimensional coordinates of the sensors that triggered the alarm, the effective geometric parameters and average normal vector of the slope are calculated, a local coordinate system for the slope is constructed, and a gridded emergency inspection route is obtained, including:

[0023] Neighborhood sampling is performed with the sensor's three-dimensional coordinates as the center, and slope threshold filtering is applied to calculate the effective average normal vector and effective geometric parameters of the slope.

[0024] A local three-dimensional coordinate system for the slope is constructed based on the effective average normal vector of the slope.

[0025] A virtual target point matrix is ​​generated in the local three-dimensional coordinate system of the slope, and the gridded emergency inspection routes for detailed slope investigation and high-altitude orthophoto are calculated.

[0026] Furthermore, neighborhood sampling is performed centered on the sensor's three-dimensional coordinates, and slope threshold filtering is applied to calculate the effective average normal vector and effective geometric parameters of the slope, including:

[0027] A set of grid points within a set range is obtained with the sensor's three-dimensional coordinates as the center, and the local slope of each grid point is calculated by traversing the set of grid points.

[0028] The set of grid points is filtered and selected according to the slope threshold, and grid points that meet the slope requirements are retained and their unit normal vectors are recorded.

[0029] The unit normal vectors of the selected grid points are averaged and normalized to obtain the effective average normal vector and effective geometric parameters of the slope.

[0030] Furthermore, a virtual target point matrix is ​​generated in the local three-dimensional coordinate system of the slope, and the gridded emergency inspection routes for detailed slope survey and high-altitude orthophoto are calculated, including:

[0031] The size of the waypoint matrix is ​​determined based on the warning level of the slope alarm event, the grid spacing is set, and a virtual target point matrix of the slope is generated in the local three-dimensional coordinate system of the slope.

[0032] The virtual target point matrix is ​​translated along the effective average normal vector of the slope to obtain the waypoints of the detailed slope survey route, and the flight attitude and gimbal shooting parameters are configured for the waypoints.

[0033] By setting the altitude in the vertical direction of the virtual target point matrix to obtain the waypoints for the high-altitude orthophoto route, and configuring the flight attitude and gimbal shooting parameters for the waypoints, a gridded emergency inspection route with dual tasks is formed.

[0034] Furthermore, the terrain-simulating modeling routes and gridded emergency inspection routes are encapsulated, and route tasks are distributed to the UAV hangar via a cloud application programming interface, including:

[0035] Call the waypoint labeling language template, write the waypoint coordinates, flight attitude parameters, and gimbal shooting parameters of the terrain modeling route and the gridded emergency inspection route into the corresponding tags of the template, and encapsulate the action instructions at the waypoint under each waypoint tag to form route encapsulation data;

[0036] The route encapsulation data is converted into a standard waypoint markup language file, the file is uploaded to the object storage system, and the corresponding file access link is obtained;

[0037] The file access link is sent to the target UAV hangar via the cloud application programming interface, and priority queue management is performed on all route tasks based on the sent route task type.

[0038] The terrain-simulating modeling route and the gridded emergency inspection route are encapsulated, and the route tasks are distributed to the UAV hangar through the cloud application programming interface.

[0039] Furthermore, the UAV executes the aforementioned route mission and transmits inspection data back, performs multi-dimensional analysis on the inspection data, and completes ground-air coordinated closed-loop decision-making, including:

[0040] The drones perform slope detailed survey, high-altitude orthophoto, and terrain modeling flight path tasks in sequence according to task priority, and transmit the inspection image data back to the cloud analysis server.

[0041] Based on the inspection image data received by the cloud analysis server, a deep learning model is used to identify the disease in the inspection image data and extract the corresponding slope disease feature information.

[0042] By retrieving historical digital elevation model data of the slope and performing spatial registration and differential calculation with the current modeling data, the geometric deformation characteristics of the slope are extracted. Combined with the monitoring status of ground sensors, the consistency assessment results of multi-source data are obtained, and the ground-air linkage closed-loop decision-making is completed.

[0043] Secondly, an automated collaborative verification system for highway slopes using unmanned aerial vehicles (UAVs) based on ground-sensor event-driven methods is provided, including:

[0044] The system comprises a data infrastructure layer, a route planning layer, a task distribution layer, an execution layer, and a data analysis and decision-making layer; the data infrastructure layer is communicatively connected to the route planning layer, task distribution layer, execution layer, and data analysis and decision-making layer, respectively.

[0045] The data basic layer is used to store and provide ground sensor monitoring data, slope basic geographic information data, and road network vector data. The flight route planning layer is used to generate terrain-following modeling flight routes and grid-based emergency inspection flight routes. The task distribution layer is used for flight route encapsulation and drone task distribution. The execution layer includes drones and drone hangars, which are used to execute flight route inspection tasks and transmit inspection data back. The data analysis layer is used for multi-dimensional analysis of inspection data and ground-air linkage closed-loop decision-making.

[0046] The invention adopting the above technical solution has the following advantages:

[0047] The present invention can achieve second-level linkage of ground movement and air following. After the ground sensor triggers the threshold, without manual intervention, the system automatically calculates the flight route and schedules the drone to take off. This automated review mode of replacing humans with machines compresses the emergency response time from hours to minutes, improving the timeliness of disaster emergency response.

[0048] The present invention introduces a ground-air data consistency evaluation matrix, and uses the macroscopic visual features (cracks, landslides) and microscopic geometric deformations (DEM difference) obtained by the drone to verify the numerical mutations of the ground sensors. Through the point-surface mutual verification logic, the system can automatically eliminate false alarms caused by equipment failures or environmental interference, and only trigger high-level warnings when multi-source data is consistent, thus solving the industry pain points of high false alarm rates and many ineffective alarms in traditional monitoring systems.

[0049] The present invention proposes a ROI construction algorithm based on road following attitude and a slope local coordinate system construction method. This enables the drone to automatically generate inspection flight routes that maintain a constant relative flight altitude and vertical viewing angle according to the true occurrence state of the slope, avoiding image distortion caused by viewing angle deviation in traditional manual flights. This standardized data collection method provides a high-quality data basis for subsequent identification of micro cracks and earthwork measurement.

[0050] The present invention uses drones to replace humans to enter high-risk slope sites. Especially in extreme working conditions such as heavy rain and earthquakes, managers can remotely obtain high-definition disaster site images and quantitative analysis reports. This not only avoids maintenance personnel being directly exposed to the risks of falling rocks and landslides, but also can quickly assess the impact on road traffic, providing a scientific basis for road closure or traffic diversion decisions, and maximizing the safety of people and vehicles along the road. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the specific embodiments of the present invention, the drawings required for the specific embodiments will be briefly introduced below. In all the drawings, the components or parts do not necessarily draw according to the actual ratio.

[0052] Figure 1This is a schematic diagram of the adaptive slope ROI construction and terrain modeling flight path planning in the UAV-based automated collaborative verification method and system for highway slopes based on ground sensing events, as described in this invention.

[0053] Figure 2 This is a schematic diagram of the generation of gridded emergency inspection routes in the automated collaborative verification method and system for highway slopes based on ground sensing events of the present invention.

[0054] Figure 3 This is the overall logical architecture diagram of the automated collaborative verification method and system for highway slopes based on ground sensing events driven by unmanned aerial vehicles (UAVs) according to the present invention.

[0055] Figure 4 This is a flowchart of the automated collaborative verification method for highway slopes using UAVs based on ground sensing events, as described in this invention. Detailed Implementation

[0056] The embodiments of the technical solution of the present invention will now be described in detail with reference to the accompanying drawings. These embodiments are merely illustrative of the technical solution of the present invention and are therefore intended to limit the scope of protection of the present invention.

[0057] like Figures 1-4 As shown, the automated collaborative verification method for highway slopes using unmanned aerial vehicles (UAVs) based on ground sensing events, according to the present invention, includes:

[0058] Step S01: Acquire ground sensor monitoring data and slope foundation geographic information data;

[0059] Step S02: Detect alarm events by thresholding the ground sensor monitoring data; generate the slope region of interest based on the slope's basic geographic information data; match the modeling parameters; and obtain the terrain modeling flight path.

[0060] Step S03: Based on the three-dimensional coordinates of the sensor that triggered the alarm, calculate the effective geometric parameters and average normal vector of the slope, construct the local coordinate system of the slope, and obtain the gridded emergency inspection route.

[0061] Step S04: Encapsulate the terrain modeling flight path and the gridded emergency inspection flight path, and distribute the flight path tasks to the UAV hangar through the cloud application programming interface;

[0062] Step S05: The UAV executes the flight path mission and transmits the inspection data back. The inspection data is then analyzed in multiple dimensions, and a closed-loop decision-making process involving ground and air is completed.

[0063] This invention is constructed based on a sensor device management library, a slope basic geographic information database (including DSM data, slope length L, slope height H, average slope, slope aspect, etc.), and road network center vector data.

[0064] Triggering mechanism: Coefficients are received in real time from ground sensor S i The monitoring data, when V t Exceeding the preset threshold V t This triggers alarm event E and initiates the subsequent automated verification process.

[0065] In this embodiment, a threshold judgment is performed on the ground sensor monitoring data to trigger an alarm event. A region of interest for the slope is generated based on the slope's basic geographic information data, and modeling parameters are matched to obtain the terrain modeling flight path, including:

[0066] Threshold judgment is performed on ground sensor monitoring data to trigger slope alarm events;

[0067] Based on the sensor identifier, retrieve the sensor's three-dimensional coordinates and the geographic information parameters of the slope to which it belongs, and construct the region of interest for the slope;

[0068] The system calls up wide-area terrain model data and obtains local terrain model data of the slope by clipping the region of interest of the slope. The modeling parameters are matched according to the warning level of the slope alarm event to obtain the terrain modeling route.

[0069] In this embodiment, the region of interest for the slope is constructed by retrieving the sensor's three-dimensional coordinates and the geographic information parameters of the slope to which it belongs based on the sensor identifier, including:

[0070] Based on the sensor identifier, retrieve the sensor's three-dimensional coordinates and the slope length, aspect, and gradient information of the slope to which it belongs from the sensor equipment management database;

[0071] Calculate the projection point of the sensor's three-dimensional coordinates onto the centerline of the road network to obtain the roadbed tangent vector and azimuth at the projection point;

[0072] An initial regular region is constructed using the sensor's three-dimensional coordinates as the center and the slope length. This initial regular region is then rotated around the sensor's three-dimensional coordinates until it is parallel to the road direction, outputting the region of interest for the slope with a spatial coordinate system.

[0073] Specifically, the core is to construct a region of interest (ROI) that is parallel to the road direction and conforms to the slope terrain.

[0074] (1) Sensor spatial positioning and parameter extraction: The system automatically retrieves the three-dimensional coordinates P0(x0, y0, z0) and basic geographic information of the slope to which the sensor belongs, such as average slope, slope aspect, and slope height, from the sensor device management library in the system based on the sensor's unique ID.

[0075] (2) Calculate the projection and tangent vector: Calculate the perpendicular projection point P of sensor P0(x0, y0, z0) onto the centerline of the road network. road Calculate point P road The tangent vector v of the roadbed at the locationroad Obtain its azimuth angle θ.

[0076] (3) Determine the ROI scale: Read the slope length L of the slope where the sensor is located and take it as the side length of the ROI area to ensure that the subsequent flight path fully covers the slope.

[0077] (4) Construct a rotating vector surface: Construct an initial square with side length L centered on sensor P0(x0, y0, z0).

[0078]

[0079] Then, rotate the square around the center point P0 by an angle θ so that its edges are parallel to the road direction, and output a ROI Shapefile (.shp) vector mask file with a spatial coordinate system.

[0080] In this embodiment, wide-area terrain model data is called and local terrain model data of the slope is obtained by clipping the region of interest of the slope. The modeling parameters are matched according to the warning level of the slope alarm event to obtain the terrain modeling route, including:

[0081] Retrieve wide-area digital surface model data, and use the vector file of the region of interest of the slope to spatially clip the wide-area digital surface model to obtain local terrain model data of the slope;

[0082] Import the local terrain model data of the slope into the flight path planning system, configure the modeling parameters according to the warning level of the slope alarm event, and obtain the terrain-following modeling flight path that matches the slope terrain.

[0083] Specifically, the core is to generate a local digital surface model (DSM) based on vector surface files, dynamically match modeling parameters according to the warning level (blue / yellow / orange / red), and automatically generate a terrain-following modeling flight path that maintains a constant relative flight altitude with the target slope.

[0084] (1) Local terrain extraction: retrieve the regional DSM file in the system, use the generated ROI Shapefile as a mask to spatially crop the wide-area DSM in the system, and output a local GeoTIFF DSM file centered on the warning point, which contains planar coordinates and preserves the real elevation undulation information of the area.

[0085] (2) Dynamic parameter matching: The cropped local DSM is automatically imported into the flight path planning system (cloud-based private planning engine). The system reads the terrain undulation of the area and dynamically configures the modeling parameters according to the warning level shown in Table 1 to generate variable altitude flight paths.

[0086] Table 1 Mapping Table of Warning Levels and Modeling Parameters

[0087]

[0088] In this embodiment, based on the three-dimensional coordinates of the sensor that triggered the alarm, the effective geometric parameters and average normal vector of the slope are calculated, a local coordinate system of the slope is constructed, and a gridded emergency inspection route is obtained, including:

[0089] Neighborhood sampling is performed with the sensor's three-dimensional coordinates as the center, and slope threshold filtering is applied to calculate the effective average normal vector and effective geometric parameters of the slope.

[0090] Construct a local three-dimensional coordinate system for the slope based on the effective average normal vector of the slope.

[0091] A virtual target point matrix is ​​generated in the local three-dimensional coordinate system of the slope, and the gridded emergency inspection routes for detailed slope investigation and high-altitude orthophoto are calculated.

[0092] In this embodiment, neighborhood sampling is performed centered on the sensor's three-dimensional coordinates, and slope threshold filtering is applied to calculate the effective average normal vector and effective geometric parameters of the slope, including:

[0093] The set of grid points within a set range is obtained with the sensor's three-dimensional coordinates as the center, and the local slope of each grid point is calculated by traversing the set of grid points.

[0094] The grid point set is filtered and selected based on the slope threshold, and the grid points that meet the slope requirements are retained and their unit normal vectors are recorded.

[0095] The unit normal vectors of the selected grid points are averaged and normalized to obtain the effective average normal vector and effective geometric parameters of the slope.

[0096] Specifically, a terrain filtering algorithm is used to eliminate the interference of horse trails and platforms on the calculation of normal vectors.

[0097] ① Neighborhood sampling: Obtain the set S of all grid points within a radius R centered at P0(x0, y0, z0). all。

[0098] ② Slope threshold filtering: Traverse each grid i in the set and calculate its local slope β i Set the slope threshold β th .

[0099] If β i <β th If it is determined to be a platform or horse track, it will be removed.

[0100] If β i ≥β th , retain and in set S slope Record the unit normal vector n of this point. i .

[0101] ③ Vector average: for S slope The effective average normal vector N, representing the true slope orientation, is obtained by averaging all normal vectors and normalizing them. avg .

[0102]

[0103] Where, N sum Let M be the total normal vector, M be the number of valid sampling points, i be the i-th valid grid cell currently traversed, and n be the number of valid sampling points. i Let n be the unit normal vector of the i-th grid cell. xi n yi n zi These represent the components of the normal vector at that point on the X, Y, and Z axes, respectively.

[0104]

[0105] Where, N avg For the effective average normal vector, N x N y N z These are the components of the standard vector in the X, Y, and Z directions.

[0106] ④ Output macroscopic geometric parameters: effective aspect α, effective slope. .

[0107]

[0108] At the same time, in order to generate a mesh that fits the slope, it is necessary to establish an N-based system. avg The local three-dimensional coordinate system (u,v,w).

[0109] Z-axis reference: Vertical axis Z of the world coordinate system world = (0, 0, 1).

[0110] w-axis (normal axis): i.e., the effective normal vector, w=N avg .

[0111] The u-axis (horizontal axis / contour line direction): a horizontal vector parallel to the slope surface. It is obtained by the cross product of the normal vector and the world Z-axis.

[0112]

[0113] v-axis (longitudinal axis / direction of maximum slope): a vector moving upwards or downwards along the slope.

[0114]

[0115] In this embodiment, a virtual target point matrix is ​​generated in the local three-dimensional coordinate system of the slope, and the gridded emergency inspection routes for detailed slope survey and high-altitude orthophoto are calculated, including:

[0116] The size of the waypoint matrix is ​​determined based on the warning level of the slope alarm event, the grid spacing is set, and a virtual target point matrix of the slope is generated in the local three-dimensional coordinate system of the slope.

[0117] The virtual target point matrix is ​​translated along the effective average normal vector of the slope to obtain the waypoints of the detailed slope survey route, and the flight attitude and gimbal shooting parameters are configured for the waypoints.

[0118] By setting the altitude in the vertical direction of the virtual target point matrix, the waypoints of the high-altitude orthophoto route are obtained. The flight attitude and gimbal shooting parameters are configured for the waypoints to form a gridded emergency inspection route with dual tasks.

[0119] Specifically, a higher warning level means a potentially larger area of ​​damage, requiring an expanded investigation area to determine the disaster's boundaries. A linear mapping rule is established between the matrix dimension N and the warning level k.

[0120] (1) Determine the size of the waypoint matrix: According to Table 2, establish a linear mapping N=2k-1 between the early warning level coefficient k and the matrix dimension N.

[0121] Table 2 Mapping Table of Warning Levels and Waypoint Matrix Size

[0122]

[0123] (2) Set grid spacing: L step =λ·L

[0124] λ (sampling density coefficient): an empirical constant, usually taken as 0.05 ~ 0.1.

[0125] (3) Generate target feature mesh

[0126] Generate a virtual target point matrix T on the physical slope. ij。

[0127]

[0128]

[0129] Where u is the horizontal unit vector of the parallel slope, and v is the vertical unit vector of the parallel slope.

[0130] (4) Dual-task route calculation

[0131] Task A (Detailed Slope Survey Route):

[0132] Translate outward by a safe distance D along the normal vector safe。

[0133] Waypoint calculation

[0134] D safe The safe shooting distance for the drone from the slope (e.g., 10m).

[0135] For each waypoint W in the matrix i,j Define two specific action instructions.

[0136] Flight yaw angle (to ensure the aircraft is directly facing the slope for filming):

[0137] α is the effective slope aspect

[0138] Gimbal tilt angle (to ensure shooting on vertical slopes):

[0139] , For effective slope

[0140] Mission B (High-altitude orthogonal flight path):

[0141] The drone is positioned directly above the target point.

[0142] H rel (Relative flight altitude): It needs to have a similar ground resolution (GSD) to Mission A. To cover a larger environmental background, a slightly higher value should be used.

[0143] H rel = 1.2×D safe

[0144] Z world (World Vertical Axis): Unit vector (0, 0, 1).

[0145] Waypoint calculation

[0146] Flight yaw angle (to ensure the aircraft is directly facing the slope for filming):

[0147] α is the effective slope aspect

[0148] Gimbal tilt angle (to ensure vertical downward shooting):

[0149] Φ2=-90°

[0150] Tasks A and B are executed in a time-sharing manner. Task A is executed first, and after the drone returns to the airport and the inspection data is uploaded to the cloud, Task B is executed to ensure the timeliness of emergency tasks.

[0151] In this embodiment, the terrain-following modeling flight path and the grid-based emergency inspection flight path are encapsulated, and the flight path tasks are distributed to the UAV hangar through the cloud application programming interface, including:

[0152] Call the waypoint markup language template, write the waypoint coordinates, flight attitude parameters, and gimbal shooting parameters of the terrain-following modeling flight path and the grid-based emergency inspection flight path into the corresponding tags of the template, and encapsulate the action instructions at each waypoint under the waypoint tags to form flight path encapsulation data;

[0153] Convert the flight path encapsulation data into a standard waypoint markup language file, upload the file to the object storage system and obtain the corresponding file access link;

[0154] Send the file access link to the target UAV hangar through the cloud application programming interface, and perform priority queue management on all flight path tasks based on the types of the issued flight path tasks;

[0155] Complete the encapsulation of the terrain-following modeling flight path and the grid-based emergency inspection flight path, and distribute the flight path tasks to the UAV hangar through the cloud application programming interface.

[0156] Specifically, template call: The <wpml:missionConfig> defined in template.kml is called as "waypoint flight".

[0157] Parameter mapping: Write the waypoint coordinates W generated by the algorithm i,j into the <Point> tag.

[0158] Write the calculated flight yaw angle Ψ into <wpml:aircraftYaw> (lock the nose direction).

[0159] Write the calculated gimbal pitch angle Φ into <wpml:gimbalPitch>.

[0160] Action encapsulation: Insert <wpml:actionGroup> under each waypoint, including the instructions "hover for 1s" and "take a photo".

[0161] Flight path encapsulation: The system background integrates the DJI WPML (Waypoint Markup Language) standard library, and uses a Python script to serialize the calculated inspection waypoints W i,j , flight yaw angle Ψ, gimbal pitch angle Φ, and the "take a photo / hover" action sequence into a standard.kmz flight path file.

[0162] Use the MQTT protocol channel of the DJI Cloud API to achieve instant task issuance.

[0163] Resource uploading: The system uploads the three generated .kmz files (Emergency Inspection Task A, Emergency Inspection Task B, and Terrain Modeling Task) to object storage (OSS / MinIO) and obtains the download link URL.

[0164] Instructions issued:

[0165] Call the flight_task_resource / publish interface to send a task order to the target hangar.

[0166] Priority queue management: The system maintains a task queue in the background, setting task A to have the highest priority (immediately jump the queue), task B to have the second highest priority, and modeling the scheduling strategy of the lowest priority task (execute when idle).

[0167] In this embodiment, the UAV executes a flight path mission and transmits inspection data back, performs multi-dimensional analysis on the inspection data, and completes a ground-air coordinated closed-loop decision-making process, including:

[0168] The drones perform slope detailed survey, high-altitude orthophoto, and terrain modeling flight path tasks in sequence according to task priority, and transmit the inspection image data back to the cloud analysis server.

[0169] Based on the inspection image data received by the cloud analysis server, a deep learning model is used to identify the disease in the inspection image data and extract the corresponding slope disease feature information.

[0170] By retrieving historical digital elevation model data of the slope and performing spatial registration and differential calculation with the current modeling data, the geometric deformation characteristics of the slope are extracted. Combined with the monitoring status of ground sensors, the consistency assessment results of multi-source data are obtained, and the ground-air linkage closed-loop decision-making is completed.

[0171] Specifically, the system establishes an integrated workflow of sensing, transmission, and computing to ensure seamless data collection and analysis, and is divided into three execution phases:

[0172] (1) Execution Phase I: Emergency Response (Task A - Detailed Slope Survey)

[0173] Triggering mechanism: Ground sensor alarm → Algorithm generates flight path → Hangar receives instructions.

[0174] Operation execution: The drone takes off automatically and performs task A (close to the slope to capture high-definition details).

[0175] Private network fast transfer: After the task is completed and returned to the homing, the hangar uses the 5G private network to prioritize the extraction and uploading of the gridded photos of task A to the cloud analysis server.

[0176] (2) Execution Phase II: Environmental Verification (Mission B - High Altitude Orthophoto)

[0177] Triggering mechanism: When the system detects that the data upload for task A is complete and the battery level meets the threshold (e.g., >30%), it automatically issues the instruction for task B.

[0178] Operation Execution: The UAV performs high-altitude orthophoto flight to acquire orthophoto images of the environment near the sensors.

[0179] (3) Execution Phase III: Terrain Modeling

[0180] Triggering mechanism: If the preliminary analysis in stage I or II finds suspected deformation, the system will automatically add a modeling task; if the first two steps are determined to be obvious false alarms, this step will be canceled to save resources.

[0181] Operation execution: Perform terrain-following flight to collect full-coverage multi-angle images of the slope.

[0182] Cloud-based modeling: After the images are uploaded, the DJI Terra API or ContextCapture Center engine is automatically invoked to generate high-precision real-world 3D models and DEMs.

[0183] 2. Multidimensional data intelligent analysis and decision-making

[0184] The returned multi-source data is processed in parallel to extract disaster features from two dimensions: "visual features" and "geometric deformation".

[0185] (1) AI-based disease identification based on emergency inspection data

[0186] Algorithm engine: Deploy YOLOv8 or SegFormer deep learning models trained for slope scenarios.

[0187] Analysis objects: partial photos of the slope in Task A + orthophotos of Task B.

[0188] Targets to be identified: cracks (identification width > 2mm), falling rocks, landslides, retaining wall cracks, and blocked drainage ditches.

[0189] Output: Images showing the location of the damage, and calculation of indicators such as crack length and opening.

[0190] (2) Quantitative differential analysis of DEM based on modeling data

[0191] Benchmark matching: The system uses the "slope foundation information" of the sensor to automatically retrieve the historical benchmark DEM of the area.

[0192] Difference operations:

[0193] The DEM generated in the current period current With DEM base Perform spatial registration and interpolation calculation:

[0194]

[0195] Noise filtering: Vegetated areas are removed (using the Visible Light Vegetation Index (VDVI) mask), retaining only deformation data for bare land and structured areas.

[0196] Output: Settlement / uplift thermogram, calculate earthwork deformation.

[0197] (3) Consistency evaluation and decision-making

[0198] Construct a "multi-source consistency evaluation matrix" to comprehensively judge the authenticity of the danger.

[0199] ① Evaluation index system

[0200] S sensor (Ground sensor status): 1 (Alarm) / 0 (Normal)

[0201] S visual (AI visual recognition): 1 (cracks / slides detected) / 0 (no obvious anomalies)

[0202] S geo (DEM geometric deformation): 1 (displacement > threshold δ) / 0 (displacement < δ)

[0203] ②According to the ground-air linkage verification logic table shown in Table 3

[0204] Table 3 Ground-Air Joint Verification Logic Table

[0205]

[0206] ③Result feedback closed loop

[0207] False alarm cancellation: If the alarm is determined to be false, the system automatically sends a command to the IoT platform to reset the alarm status of the sensor and records a "false alarm event" for sensor health analysis.

[0208] Report Generation: Automatically generates the "Air-Ground Joint Verification Quick Report," which includes:

[0209] Alarm source information (ID, location, trigger value), drone inspection trajectory and field of view, core supporting materials (close-up of cracks, displacement heat map), and final conclusion (false alarm / real alarm).

[0210] Push notifications: Send notifications to the responsible maintenance personnel via web pop-ups, SMS, or App messages.

[0211] In other embodiments, a ground-sensor event-driven automated collaborative verification system for highway slopes using unmanned aerial vehicles (UAVs) is provided, based on any of the preceding ground-sensor event-driven automated collaborative verification methods for highway slopes using unmanned aerial vehicles (UAVs), including:

[0212] The system consists of a data infrastructure layer, a route planning layer, a task distribution layer, an execution layer, and a data analysis and decision-making layer. The data infrastructure layer communicates with the route planning layer, the task distribution layer, the execution layer, and the data analysis and decision-making layer, respectively.

[0213] The data foundation layer stores and provides ground sensor monitoring data, slope basic geographic information data, and road network vector data. The flight path planning layer generates terrain-simulating modeling flight paths and gridded emergency inspection flight paths. The task distribution layer encapsulates flight paths and issues UAV tasks. The execution layer includes UAVs and UAV hangars, which are used to execute flight path inspection tasks and transmit inspection data back. The data analysis layer is used for multi-dimensional analysis of inspection data and ground-air linkage closed-loop decision-making.

[0214] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.

Claims

1. A method for automated collaborative verification of highway slopes using unmanned aerial vehicles (UAVs) based on ground-sensor event-driven technology, characterized in that: include: Acquire ground sensor monitoring data and basic geographic information data of slopes; The ground sensor monitoring data is used to trigger an alarm event by threshold judgment. Based on the basic geographic information data of the slope, a region of interest for the slope is generated and the modeling parameters are matched to obtain the terrain modeling route. Based on the sensor's three-dimensional coordinates that trigger the alarm, the effective geometric parameters and average normal vector of the slope are calculated, a local coordinate system of the slope is constructed, and a gridded emergency inspection route is obtained. The terrain-simulating modeling route and the gridded emergency inspection route are encapsulated, and the route tasks are distributed to the UAV hangar through a cloud application programming interface. The UAV executes the route mission and transmits inspection data back, performs multi-dimensional analysis on the inspection data, and completes ground-air linkage closed-loop decision-making. Threshold judgment is performed on the monitoring data from the ground sensors to trigger a slope alarm event; Based on the sensor identifier, retrieve the sensor's three-dimensional coordinates and the geographic information parameters of the slope to which it belongs, and construct the region of interest for the slope; The wide-area terrain model data is called and the local terrain model data of the slope is obtained by clipping the region of interest of the slope. The modeling parameters are matched according to the warning level of the slope alarm event to obtain the terrain modeling route. Based on the sensor identifier, retrieve the sensor's three-dimensional coordinates and the slope length, aspect, and gradient information of the slope to which it belongs from the sensor equipment management database; Calculate the projection point of the sensor's three-dimensional coordinates onto the centerline of the road network to obtain the roadbed tangent vector and azimuth at the projection point; An initial regular region is constructed using the sensor's three-dimensional coordinates as the center and the slope length. The initial regular region is then rotated around the sensor's three-dimensional coordinates until it is parallel to the road direction, and the region of interest of the slope with a spatial coordinate system is output. Neighborhood sampling is performed with the sensor's three-dimensional coordinates as the center, and slope threshold filtering is applied to calculate the effective average normal vector and effective geometric parameters of the slope. A local three-dimensional coordinate system for the slope is constructed based on the effective average normal vector of the slope. A virtual target point matrix is ​​generated in the local three-dimensional coordinate system of the slope, and the gridded emergency inspection route for detailed slope investigation and high-altitude orthophoto is calculated. A set of grid points within a set range is obtained with the sensor's three-dimensional coordinates as the center, and the local slope of each grid point is calculated by traversing the set of grid points. The set of grid points is filtered and selected according to the slope threshold, and grid points that meet the slope requirements are retained and their unit normal vectors are recorded. The unit normal vectors of the filtered grid points are averaged and normalized to obtain the effective average normal vector and effective geometric parameters of the slope. The size of the waypoint matrix is ​​determined based on the warning level of the slope alarm event, the grid spacing is set, and a virtual target point matrix of the slope is generated in the local three-dimensional coordinate system of the slope. The virtual target point matrix is ​​translated along the effective average normal vector of the slope to obtain the waypoints of the detailed slope survey route, and the flight attitude and gimbal shooting parameters are configured for the waypoints. By setting the altitude in the vertical direction of the virtual target point matrix to obtain the waypoints for the high-altitude orthophoto route, and configuring the flight attitude and gimbal shooting parameters for the waypoints, a gridded emergency inspection route with dual tasks is formed.

2. The automated collaborative verification method for highway slopes using UAVs based on ground-sensor event-driven technology as described in claim 1, characterized in that, Wide-area terrain model data is retrieved and local terrain model data of the slope is obtained by clipping the region of interest of the slope. Modeling parameters are matched according to the warning level of the slope alarm event to obtain the terrain modeling flight path, including: Retrieve wide-area digital surface model data, and use the vector file of the slope region of interest to spatially clip the wide-area digital surface model to obtain local slope terrain model data; The local terrain model data of the slope is imported into the flight path planning system. The modeling parameters are configured according to the warning level of the slope alarm event to obtain a terrain-following modeling flight path that matches the slope terrain.

3. The automated collaborative verification method for highway slopes using unmanned aerial vehicles based on ground sensing event-driven methods according to claim 1, characterized in that, The terrain-simulating modeling flight path and the gridded emergency inspection flight path are encapsulated, and the flight path tasks are distributed to the UAV hangar through a cloud application programming interface, including: Call the waypoint labeling language template, write the waypoint coordinates, flight attitude parameters, and gimbal shooting parameters of the terrain modeling route and the gridded emergency inspection route into the corresponding tags of the template, and encapsulate the action instructions at the waypoint under each waypoint tag to form route encapsulation data; The route encapsulation data is converted into a standard waypoint markup language file, the file is uploaded to the object storage system, and the corresponding file access link is obtained; The file access link is sent to the target UAV hangar via the cloud application programming interface, and priority queue management is performed on all route tasks based on the sent route task type. The terrain-simulating modeling route and the gridded emergency inspection route are encapsulated, and the route tasks are distributed to the UAV hangar through the cloud application programming interface.

4. The automated collaborative verification method for highway slopes using unmanned aerial vehicles based on ground sensing event-driven methods according to claim 1, characterized in that, The UAV executes the aforementioned route mission and transmits inspection data back, performs multi-dimensional analysis on the inspection data, and completes ground-air coordinated closed-loop decision-making, including: The drones perform slope detailed survey, high-altitude orthophoto, and terrain modeling flight path tasks in sequence according to task priority, and transmit the inspection image data back to the cloud analysis server. Based on the inspection image data received by the cloud analysis server, a deep learning model is used to identify the disease in the inspection image data and extract the corresponding slope disease feature information. By retrieving historical digital elevation model data of the slope and performing spatial registration and differential calculation with the current modeling data, the geometric deformation characteristics of the slope are extracted. Combined with the monitoring status of ground sensors, the consistency assessment results of multi-source data are obtained, and the ground-air linkage closed-loop decision-making is completed.

5. A UAV-based automated collaborative verification system for highway slopes driven by ground sensing events, characterized in that: The automated collaborative verification method for highway slopes based on ground-sensor event-driven UAVs according to any one of claims 1 to 4 includes: The system comprises a data foundation layer, a flight path planning layer, a task distribution layer, an execution layer, and a data analysis and decision-making layer. The data foundation layer is communicatively connected to the flight path planning layer, the task distribution layer, the execution layer, and the data analysis and decision-making layer. The data foundation layer stores and provides ground sensor monitoring data, slope basic geographic information data, and road network vector data. The flight path planning layer generates terrain-simulating modeling flight paths and gridded emergency inspection flight paths. The task distribution layer encapsulates flight paths and issues UAV tasks. The execution layer includes UAVs and a UAV hangar for executing flight path inspection tasks and transmitting inspection data back. The data analysis layer performs multi-dimensional analysis of inspection data and makes closed-loop decisions based on ground-air coordination.

Images