Highway intelligent inspection and abnormal state processing method based on unmanned aerial vehicle

Abstract

The invention discloses a highway intelligent inspection and abnormal state processing method based on an unmanned aerial vehicle, and belongs to the field of image processing and computer vision, and the method comprises the steps: carrying out the semantic disassembly and knowledge graph association retrieval of natural language input, and obtaining a situational inspection instruction; basic geographic information and a real-time dynamic environment are combined, and a dynamic safety corridor is generated through three-dimensional potential field repulsive force calculation; a sensor configuration scheme and a flight mode are extracted according to the associated scene features and mapped to a waypoint, and an optimized flight inspection task is generated; performing anomaly screening and fine identification on the video stream to generate anomaly confirmation information; and updating the knowledge graph and the configuration optimization relation by using the abnormal information and the disposal feedback data. According to the method, a closed-loop control technology integrating the knowledge graph, dynamic path planning and real-time image analysis is adopted, autonomous planning, accurate execution and intelligent feedback learning of the inspection task can be realized, and the utilization efficiency of image data in the inspection process and the accuracy of anomaly recognition are improved.

Description

Intelligent Highway Inspection and Abnormal Status Handling Method Based on Unmanned Aerial Vehicles Technical Field

[0001] This invention relates to the fields of image processing and computer vision technology, and in particular to a method for intelligent highway inspection and abnormal status handling based on unmanned aerial vehicles (UAVs). Background Technology

[0002] As a vital national infrastructure, the safe and stable operation of highways is of paramount importance. Traditional inspection methods, primarily relying on manual patrols or vehicle inspections, suffer from low efficiency, high risk, and significant susceptibility to weather and traffic conditions. With the development of drone and computer vision technologies, using drones equipped with high-definition cameras for highway inspection has become an emerging approach. This technology collects aerial images or video data, which is then analyzed in the background to identify abnormal conditions in road surfaces, bridges, guardrails, and other facilities. This represents a specific application of image data processing technology in the field of infrastructure maintenance.

[0003] In related technologies, Chinese invention patent CN119414859A discloses an adaptive detection and inspection system for low-altitude flight inspection. First, it acquires multiple sets of UAV description information, UAV swarm description information, and flight inspection task description information. This information covers the performance parameters and task requirements of UAVs in different inspection scenarios. Then, this information is loaded into a pre-learned target adaptive inspection decision model. By analyzing this information, a target estimation confidence score is generated to assess the success probability of the UAV swarm performing a specific inspection task. When the target estimation confidence score meets the set requirements, the corresponding UAV swarm is intelligently scheduled to perform the inspection task, thereby ensuring the efficient completion of the task. Thus, through intelligent evaluation and dynamic scheduling, the adaptability and flexibility of UAV inspection operations are effectively improved.

[0004] However, in the above-mentioned schemes, the definition of inspection tasks mainly relies on highly formalized feature description vectors, making it difficult to directly parse contextualized instructions from unstructured business requirements, thus limiting the depth of semantic understanding of complex inspection scenarios. At the same time, this method mainly focuses on the pre-scheduling and capability matching of inspection resources, and lacks a planning mechanism to construct a dynamic constrained space to ensure flight path safety when facing complex environmental factors that change in real time during the inspection process. In addition, it often adopts a single acquisition logic at the task execution level, lacking refined control methods to adaptively adjust perception parameters and flight modes based on the attributes of specific objects of interest, and fails to establish a data feedback closed loop from anomaly discovery to the final handling result, making it difficult for the system to achieve continuous updating and iteration of the knowledge system based on actual business results. Summary of the Invention

[0005] To address the aforementioned issues, this invention provides a method for intelligent highway inspection and anomaly handling based on unmanned aerial vehicles (UAVs). It employs a closed-loop control technology that integrates knowledge graphs, dynamic path planning, and real-time image analysis, enabling autonomous planning, precise execution, and intelligent feedback learning of inspection tasks. This improves the efficiency of image data utilization and the accuracy of anomaly identification during the inspection process.

[0006] The above objectives can be achieved through the following scheme: a method for intelligent highway inspection and anomaly handling based on unmanned aerial vehicles (UAVs), comprising: semantic decomposition and knowledge graph association retrieval of acquired natural language input to obtain contextualized inspection instructions containing the geographical range of the target road segment, the object of interest, and the characteristics of associated scenarios; receiving the contextualized inspection instructions, acquiring basic geographic information and real-time dynamic environmental data of the geographical range of the target road segment, and using the contextualized inspection instructions to perform three-dimensional potential field repulsion calculation on the basic geographic information and real-time dynamic environmental data to generate a dynamic safety corridor defining the airspace where the UAV can fly; extracting sensor configuration schemes and flight modes from pre-stored configuration optimization relationships based on the associated scenario characteristics, and mapping them to waypoints in the dynamic safety corridor to generate optimized flight inspection tasks; performing wide-area anomaly screening and local fine-grained identification on the video stream collected by the UAV during the execution of the optimized flight inspection tasks to generate anomaly confirmation information containing anomaly type and location; responding to the anomaly confirmation information to acquire the handling feedback data generated during the handling process, and using the anomaly confirmation information and handling feedback data to perform data update operations on the knowledge graph and the configuration optimization relationships.

[0007] Optionally, obtaining the contextualized inspection instruction containing the geographical range of the target road segment, the object of interest, and the features of the associated scenarios includes: performing semantic parsing on the natural language input to extract the road segment description text and the demand description text; performing topological matching between the road segment description text and the road segment node data in the knowledge graph to determine the geographical range of the target road segment; performing feature association matching and risk probability calculation between the demand description text and the historical disease data, meteorological time series data, and facility archive data associated with the geographical range of the target road segment in the knowledge graph to generate the features of the object of interest and the associated scenarios; and encapsulating the geographical range of the target road segment, the object of interest, and the features of the associated scenarios to generate the contextualized inspection instruction.

[0008] Optionally, generating a dynamic safety corridor defining the airspace where the UAV can fly includes: acquiring real-time traffic flow density data and real-time micro-meteorological data of the target road segment from a traffic monitoring platform and a meteorological monitoring network, respectively, to generate real-time dynamic environmental data; acquiring basic geographic information from a map service that provides geographic information as static obstacle parameters, and converting the real-time dynamic environmental data into dynamic repulsive force field parameters; calculating the spatial virtual repulsive force intensity of the static obstacle parameters and the dynamic repulsive force field parameters based on contextualized inspection commands, defining the three-dimensional spatial boundary, and generating a dynamic safety corridor defining the airspace where the UAV can fly.

[0009] Optionally, the step of extracting sensor configuration schemes and flight modes from pre-stored configuration optimization relationships based on the associated scene features includes: extracting material features from the associated scene features, indexing sensor type combinations from the configuration optimization relationships; extracting environmental risk features from the associated scene features, indexing mission flight modes including switching logic between cruise scanning mode and fixed-point detailed investigation mode from the configuration optimization relationships; and combining and binding the sensor type combinations with the mission flight modes to generate sensor configuration schemes and flight modes.

[0010] Optionally, the generation of the optimized flight inspection task includes: extracting an optimized repulsive potential energy path within the three-dimensional boundary of the dynamic safety corridor, and spatially discretizing the path to generate a flight waypoint sequence containing temporal information; parsing the cruise speed parameters and attitude control parameters in the flight mode, assigning the cruise speed parameters and attitude control parameters to each waypoint in the flight waypoint sequence, and generating basic flight path data with kinematic constraints; calculating the field of view coverage of the sensor configuration scheme on the flight waypoint sequence based on the geographical location of the object of interest in the contextualized inspection command, and selecting key observation waypoints for data collection from the flight waypoint sequence; binding the acquisition trigger command and parameters in the sensor configuration scheme to the key observation waypoints, and temporally encapsulating the binding result with the basic flight path data to generate the optimized flight inspection task.

[0011] Optionally, generating anomaly confirmation information including anomaly type and location includes: performing background subtraction and texture analysis on the video stream to generate a first-level detection result including suspected anomaly regions and preliminary attribute descriptions; controlling the drone to adjust the optical zoom and shooting angle for each suspected anomaly region in the first-level detection result to acquire a fine image; and using dedicated recognition parameters matching the object of interest to extract features and classify the fine image based on the preliminary attribute description to generate anomaly confirmation information including anomaly type and location.

[0012] Optionally, the method further includes: obtaining historical traffic flow data within the time window of the anomaly occurrence based on the location data and time data in the anomaly confirmation information; performing spatiotemporal correlation analysis on the anomaly confirmation information and the historical traffic flow data to generate a traffic event correlation determination result; if the traffic event correlation determination result indicates a correlation, extracting the relevant traffic event records and appending them to the anomaly confirmation information.

[0013] Optionally, the response to the handling feedback data generated during the abnormality confirmation information acquisition and handling process includes: when the abnormality type is an immediate abnormality, searching for roadside fixed monitoring devices whose geographical distance meets preset proximity conditions based on the location in the abnormality confirmation information; sending control commands containing target orientation and zoom parameters to the roadside fixed monitoring devices to generate a monitoring perspective covering the abnormal location; constructing a synchronous transmission channel containing a video stream from the drone's perspective and a video stream from the monitoring perspective, and pushing the data from the synchronous transmission channel to the handling terminal to obtain handling feedback data.

[0014] Optionally, the data update operation of the knowledge graph and the configuration optimization relationship using anomaly confirmation information and handling feedback data includes: converting the anomaly confirmation information into historical disease data nodes and writing them into the knowledge graph; extracting post-repair status data from the handling feedback data and using the post-repair status data to correct the facility's health decay parameter; performing correlation statistics on the accumulated anomaly confirmation information and the handling feedback data, and using the statistical results to adjust the mapping weights of scene features and configuration schemes in the configuration optimization relationship.

[0015] Based on the same inventive concept, this invention also provides a UAV-based intelligent highway inspection and abnormal state handling system, comprising: an instruction parsing module, used to perform semantic decomposition and knowledge graph association retrieval on the acquired natural language input to obtain a contextualized inspection instruction containing the geographical range of the target road segment, the object of interest, and the characteristics of the associated scene; a dynamic planning module, used to receive the contextualized inspection instruction, acquire basic geographic information and real-time dynamic environmental data of the geographical range of the target road segment, and use the contextualized inspection instruction to perform three-dimensional potential field repulsion calculation on the basic geographic information and real-time dynamic environmental data to generate a dynamic safety corridor defining the airspace where the UAV can fly; and any The task optimization module is used to extract sensor configuration schemes and flight modes from pre-stored configuration optimization relationships based on the associated scene features, and map them to waypoints in the dynamic safety corridor to generate optimized flight inspection tasks; the collaborative detection module is used to perform wide-area anomaly screening and local fine-grained identification on the video stream collected by the UAV during the execution of the optimized flight inspection task, and generate anomaly confirmation information containing anomaly type and location; the closed-loop handling and learning module is used to obtain handling feedback data generated during the handling process in response to the anomaly confirmation information, and use the anomaly confirmation information and handling feedback data to perform data update operations on the knowledge graph and the configuration optimization relationships.

[0016] Compared with the prior art, the present invention has the following advantages: The present invention automatically transforms manual inspection requirements into contextualized machine instructions through natural language parsing and knowledge graph association, which improves the intelligence level of task deployment and the convenience of human-computer interaction, so that the generation of inspection tasks no longer depends on cumbersome manual parameter configuration, but can quickly respond to fuzzy and comprehensive inspection intentions.

[0017] This invention proposes a dynamic safety corridor generation method based on real-time environmental data and a two-stage collaborative detection strategy. By planning safe flight airspace online and combining wide-area screening with local fine-grained identification, it not only ensures the safe operation of UAVs in complex and ever-changing highway environments, but also improves the efficiency and accuracy of anomaly detection, achieving a seamless connection from large-scale rapid inspection to precise diagnosis of key points.

[0018] This invention constructs a complete closed-loop learning mechanism from anomaly confirmation to handling feedback and then to system knowledge update. It can digitize and knowledge-encode the experience of each inspection and handling, and use this data to optimize its internal knowledge graph and configuration optimization relationship, enabling the system to have the ability to self-evolve and continuously optimize, thus becoming more intelligent and efficient in continuous application.

[0019] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 is a flowchart illustrating the intelligent highway inspection and abnormal status handling method based on unmanned aerial vehicles according to an embodiment of the present invention.

[0022] Figure 2 is a risk probability composition analysis diagram of the object of concern in an embodiment of the present invention.

[0023] Figure 3 is a schematic diagram of the virtual repulsive field in the flight space of the UAV according to an embodiment of the present invention.

[0024] Figure 4 is a schematic diagram of the structure of the UAV-based intelligent highway inspection and abnormal status handling system according to an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] Referring to Figure 1, one embodiment of the present invention proposes a method for intelligent highway inspection and abnormal state handling based on unmanned aerial vehicles (UAVs). It adopts a closed-loop control technology that integrates knowledge graphs, dynamic path planning, and real-time image analysis, which enables autonomous planning, precise execution, and intelligent feedback learning of inspection tasks, thereby improving the utilization efficiency of image data and the accuracy of anomaly identification during the inspection process.

[0027] The method described in this embodiment specifically includes: S1, performing semantic decomposition and knowledge graph association retrieval on the acquired natural language input to obtain a contextualized inspection instruction containing the geographical range of the target road segment, the object of interest, and the features of the associated scenarios; Optionally, obtaining the contextualized inspection instruction containing the geographical range of the target road segment, the object of interest, and the features of the associated scenarios includes: performing semantic parsing on the natural language input to extract the road segment description text and the demand description text; performing topological matching between the road segment description text and the road segment node data in the knowledge graph to determine the geographical range of the target road segment; performing feature association matching and risk probability calculation between the demand description text and the historical disease data, meteorological time series data, and facility archive data associated with the geographical range of the target road segment in the knowledge graph to generate the features of the object of interest and the features of the associated scenarios; and encapsulating the geographical range of the target road segment, the object of interest, and the features of the associated scenarios to generate a contextualized inspection instruction.

[0028] Specifically, the system acquires natural language input, such as "Inspect the road surface and right-side guardrail of the Beijing-Hong Kong-Macau Expressway from K350 to K355. There has been strong wind recently, so please check for any damage." Then, it calls a deep learning-based semantic parsing model. This model uses pre-trained word vectors and an attention mechanism to segment the input text into two core parts: a road segment description text, which reads "Beijing-Hong Kong-Macau Expressway from K350 to K355," and a requirement description text, which reads "road surface, right-side guardrail, strong winds, damage."

[0029] The extracted road segment description text is used as query input and retrieved from an internally maintained highway knowledge graph. This knowledge graph is a professional domain knowledge network stored in the form of a graph database. Road segment nodes are uniquely identified by "highway number - starting chainage - ending chainage," and their corresponding geospatial coordinate sequences and adjacency relationships with other road segments and facilities are recorded. Using a graph matching algorithm, nodes corresponding to "Beijing-Hong Kong-Macau Expressway K350" and "Beijing-Hong Kong-Macau Expressway K355" are found in the knowledge graph. All road segment nodes and their geographic coordinates between these two points are extracted along the topological connections, thereby generating a linear geographic feature defining the inspection range, i.e., the geographic range of the target road segment.

[0030] Keywords such as "road surface, guardrail, and strong wind" in the requirement description text are correlated with the identified geographical area of ​​the target road segment in a knowledge graph. Historical damage data associated with this road segment is retrieved, such as road surface crack reports from the past 12 months; meteorological time-series data is retrieved, such as the maximum wind speed level in the past 48 hours; and facility archive data is consulted to obtain information such as the material and installation year of the guardrails for this road segment. Based on this multi-source heterogeneous data, a risk assessment model is used to calculate the risk probability of the target object. For example, the formula for calculating the risk value R for guardrail damage is: ;in, The comprehensive risk index represents a specific object of concern, such as the guardrail at K351, and is a normalized value between 0 and 1. The historical damage factor is calculated by statistically analyzing the frequency of damage to similar facilities on the same road section during the same historical period, and is also normalized. Representing environmental impact factors, these are determined based on the risk levels corresponding to real-time or recent meteorological data. This correspondence is pre-defined by the operation and maintenance experience database. This represents the asset lifecycle factor, calculated as the ratio of the facility's service life to its design life. The longer the service life, the closer this value is to 1. These are weighting coefficients, and their sum is 1. These weights are dynamically adjusted according to different disease types. For example, for the inspection of "loose billboards," the environmental impact factor... weight This value might be set to a relatively high value, such as 0.5. Through this calculation, a list of objects of concern, sorted by risk probability, is output, such as "guardrail at K352" and "asphalt pavement at K354," and associated scene features containing information such as {weather: strong wind, risk type: structural damage, material of concern: metal, asphalt} are generated. As shown in Figure 2, the figure visually illustrates how the comprehensive risk index of different objects of concern is accumulated from historical damage factors, environmental impact factors, and asset life cycle factors according to different weighting coefficients. By quantifying the contribution of each factor, dynamic ranking of the risks of facilities in different road sections is achieved.

[0031] The aforementioned data is then integrated and encapsulated. The geographic coordinate sequence of the target road segment, the list of objects of interest including risk ranking, and the structured related scenario features are all encapsulated into a JSON or XML data object. This final data object is the contextualized inspection instruction, which fully defines "where to inspect," "what to focus on," and "in what scenario," providing all the necessary contextual information for subsequent UAV flights and reconnaissance missions.

[0032] For example, taking the routine inspection scenario of the Beijing-Hong Kong-Macau Expressway section from K350 to K355 as an example, the system first receives and processes the natural language command input by the operator through the command parsing module: "Inspect the road surface and right-side guardrail of the Beijing-Hong Kong-Macau Expressway section from K350 to K355. There has been strong wind recently, so please pay attention to whether there is any damage." The system uses a deep learning semantic parsing model to segment the text into a road segment description and a requirement description containing "road surface, guardrail, strong wind, and damage." The system then performs topological matching in the expressway knowledge graph to retrieve the node sequence identified by "Beijing-Hong Kong-Macau Expressway-K350-K355" and its geographical coordinates, thereby generating the geographical range of the linear target road segment. Next, the system performs feature association matching and risk probability calculation. For the guardrail at K351, which is the object of interest, historical data is retrieved to obtain historical damage factors. The environmental impact factor was determined to be 0.4, based on the wind force level of gale-force winds (level 9) in real-time meteorological data. The asset lifecycle factor is calculated as 0.9, based on the ratio of the facility's service life to its design life. The value is 0.7, and it is based on a preset weighting coefficient. Using the formula Calculations were performed to obtain the comprehensive risk index. The value is relatively high, so the system lists the guardrail at K351 as a high-priority object of concern and encapsulates the aforementioned coordinates, object list, and related scene features such as "weather: strong wind, material: metal" into a contextualized inspection instruction in JSON format. This method realizes the transformation from fuzzy natural language to machine-executable instructions, and through quantitative evaluation of multi-source data, it intelligently focuses limited inspection resources on high-risk areas, improving the targeting of task planning.

[0033] S2. Receive the contextualized inspection command, acquire basic geographic information and real-time dynamic environmental data of the target road segment, and use the contextualized inspection command to perform three-dimensional potential field repulsion force calculation on the basic geographic information and real-time dynamic environmental data to generate a dynamic safety corridor defining the airspace where the UAV can fly; Optionally, generating the dynamic safety corridor defining the airspace where the UAV can fly includes: acquiring real-time traffic flow density data and real-time micro-meteorological data of the target road segment from the traffic monitoring platform and the meteorological monitoring network respectively, and generating real-time dynamic environmental data; acquiring basic geographic information as static obstacle parameters from the map service providing geographic information, and converting the real-time dynamic environmental data into dynamic repulsion force field parameters; calculating the spatial virtual repulsion force intensity of the static obstacle parameters and the dynamic repulsion force field parameters based on the contextualized inspection command, defining the three-dimensional spatial boundary, and generating a dynamic safety corridor defining the airspace where the UAV can fly.

[0034] Specifically, data streams for the inspected target road sections are acquired in real time from the highway traffic monitoring platform and the regional meteorological monitoring network through standard data interfaces. Real-time traffic flow density data is obtained in the form of vehicles per kilometer and is associated with average vehicle speed information; real-time micro-meteorological data includes parameters such as wind speed, wind direction, and rainfall intensity, with a data refresh frequency typically between 1 and 5 minutes. All of this data is uniformly labeled as real-time dynamic environmental data.

[0035] Basic geographic information of the target road segment is retrieved from a built-in or cloud-connected high-precision 3D map service. This information includes, but is not limited to, the coordinates of the road centerline, road surface elevation, and 3D geometric models of bridges, gantries, traffic signs, roadside guardrails, and high-voltage power line towers along the route. This information is used as static obstacle parameters. The geometric surface of each static obstacle and the dynamic hazard zone calculated from real-time dynamic environmental data are both considered as repulsive force sources. For example, real-time traffic flow density data is transformed into a dynamically extending 3D moving envelope along the road segment, covering approximately 10 meters above the road surface. Its boundaries expand or contract according to traffic flow density and speed, serving as dynamic repulsive force field parameters. Severe weather conditions such as strong winds and heavy rain increase the thickness of the safety buffer zone around all obstacles, also contributing to the dynamic repulsive force field parameters.

[0036] The artificial potential field method is used to calculate the value of any point in space. The repulsive force intensity is calculated using the following formula: in, Represents the coordinates of a point in three-dimensional space. The total virtual repulsive force intensity at the location is a dimensionless risk scalar. and Representing the first The first static obstacle and the first The basic rejection coefficient for each dynamic hazard area is adjusted based on contextualized inspection instructions. For example, when an instruction requires a close-up inspection of a bridge... At that time, the bridge's repulsion coefficient was temporarily adjusted. Lower the coefficient to a smaller value, such as 0.2, while keeping the coefficients for other non-focused targets at a higher level, such as 1.0. and Representing points respectively To static obstacles Surface and dynamic hazardous areas The shortest Euclidean distance to the boundary. and It is the radius of influence of each repulsive force source, such as the radius of influence of a high-voltage line. It could be set to 15 meters, but the radius of influence in areas with heavy traffic is limited. It can be dynamically set to 25 meters. It is a piecewise function, when Less than When, its value follows The decrease is followed by a sharp increase, for example... ,when Greater than or equal to At that time, its value is 0. Set the global repulsive force strength threshold. This threshold is pre-configured based on the drone's performance metrics and safety redundancy requirements. All requirements are met. Less than or equal to spatial point The collection of these elements together forms a continuous, collision-free tubular or layered region in three-dimensional space. This region serves as a dynamic safety corridor defining the airspace where drones can fly, and its boundaries dynamically contract, expand, or deform in response to real-time changes in traffic conditions and weather. As shown in Figure 3, this figure illustrates the potential field distribution in three-dimensional space, where the raised "peaks" represent repulsive force sources generated by static obstacles and dynamic traffic flow, with their height corresponding to the intensity of the repulsive force. The grid plane below represents the system-set safety threshold. The flat area below the threshold plane in the figure constitutes a dynamic safety corridor with no risk of collision for drones, vividly illustrating the mapping process from the physical environment to the digital risk field.

[0037] For example, the dynamic planning module obtains environmental data of the K350 section of the Beijing-Hong Kong-Macau Expressway in real time through an interface, and constructs a system including a roadside static gantry. and the dynamic traffic flow area above the road surface A three-dimensional spatial model for the points to be evaluated in space. Measure its distance from the gantry surface It is 4 meters away from the boundary of the traffic flow area. The system is set to have an influence radius of 12 meters. The dynamic impact radius of the traffic flow area after being affected by strong winds is 10 meters. The distance is 20 meters, and a static repulsion coefficient is set. The dynamic repulsion coefficient is 1.0. The value is 1.5, using the potential field formula. Calculate the virtual repulsive force intensity, where the potential field function is... Calculated as Then the static repulsion term is The dynamic repulsion term is Total virtual repulsive force intensity Because this value is less than the globally set safety threshold of 0.05, the system makes a judgment point. This area belongs to a dynamic safety corridor zone suitable for flight. This method achieves refined quantification of three-dimensional spatial risks by establishing a potential field model that integrates dynamic and static elements, ensuring that the flight space generated by the UAV can both avoid fixed obstacles and adapt to real-time changing traffic and weather environments.

[0038] S3. Based on the associated scene features, extract sensor configuration schemes and flight modes from the pre-stored configuration optimization relationship, and map them to waypoints in the dynamic safety corridor to generate an optimized flight inspection task; Optionally, extracting sensor configuration schemes and flight modes from the pre-stored configuration optimization relationship based on the associated scene features includes: extracting material features from the associated scene features, indexing sensor type combinations from the configuration optimization relationship; extracting environmental risk features from the associated scene features, indexing a task flight mode including switching logic between cruise scanning mode and fixed-point detailed inspection mode from the configuration optimization relationship; combining and binding the sensor type combinations with the task flight mode to generate sensor configuration schemes and flight modes.

[0039] Specifically, material features, such as "asphalt pavement," "concrete bridge pier," or "metal sign," are extracted from contextualized inspection instructions. These material features serve as index keys for searching within a pre-stored configuration optimization relationship. This configuration optimization relationship is a mapping database formed through long-term data accumulation and expert experience calibration, associating the optical or physical properties of different materials under different defects with the most suitable sensor type. For example, when the material feature is "asphalt pavement" and the defect of interest is "micro-cracks," the index retrieves a sensor type combination that includes a high-resolution visible light zoom camera and a LiDAR, with the former used for morphological recognition and the latter for 3D structural measurement.

[0040] Continuing to analyze the environmental risk features within the associated scene characteristics, the drone's flight strategy is dynamically adjusted. Environmental risk features may include descriptions such as "gusts exceeding level 7," "proximity to high-voltage power lines," and "complex structures under bridges." Using these features, the configuration optimization relationship is queried again, and the corresponding mission flight mode is indexed. The mission flight mode defines the macroscopic behavioral logic of the drone throughout the mission, with the core being the switching logic between cruise scanning mode and fixed-point detailed inspection mode. Cruise scanning mode prioritizes efficiency in a wide-area detection state, where the drone flies along a predetermined route at a relatively constant speed, such as 8 to 12 meters per second, using a wide-angle lens for extensive coverage photography. Fixed-point detailed inspection mode prioritizes accuracy in a localized, precise inspection state. This mode is triggered when the drone approaches a high-risk target or initially detects a suspected anomaly during cruise scanning. The drone reduces its speed to hover or slowly circle around the target, activating a zoom lens or switching to a specific sensor to collect detailed data from multiple angles. The configuration optimization relationship will provide specific switching thresholds and execution parameters based on environmental risk characteristics. For example, in high wind speed environments, the hovering time of the fixed-point detailed investigation mode will be limited to within 30 seconds to ensure the stability of the machine.

[0041] The selected sensor type combination is logically bound to the mission flight mode containing the switching logic. This means that it clarifies which sensors and parameters should be activated in cruise scan mode, and how the sensor's operating status should be adjusted after switching to fixed-point detailed investigation mode, such as adjusting the camera's optical zoom from 1x to 20x and triggering the LiDAR to scan.

[0042] For example, the task optimization module deeply analyzes the contextualized inspection instructions, extracts the material characteristics of "asphalt pavement" and the environmental risk characteristics of "level 7 gusts" from the associated scene features, and performs index matching in the pre-stored configuration optimization relation database. For the detection requirement of micro-cracks in asphalt pavement, the system matches a sensor combination of a "high-resolution visible light zoom camera" for texture recognition and a "LiDAR" for smoothness measurement. For gusty environments, the system locks the "high-stability wind-resistant flight mode," which forcibly limits the drone's cruising speed to 5 meters per second and sets the maximum hovering time during fixed-point detailed inspections to 15 seconds to prevent attitude instability. The system then logically binds the selected sensor combination with the flight mode parameters, generating a final execution plan that includes "activating LiDAR scanning during cruising and switching to 30x zoom for photography after anomalies are detected." This method can dynamically adjust hardware configuration and operational logic according to specific material characteristics and environmental risks, effectively resolving the contradiction between data acquisition quality and flight safety in complex environments.

[0043] Optionally, the generation of the optimized flight inspection task includes: extracting an optimized repulsive potential energy path within the three-dimensional boundary of the dynamic safety corridor, and spatially discretizing the path to generate a flight waypoint sequence containing temporal information; parsing the cruise speed parameters and attitude control parameters in the flight mode, assigning the cruise speed parameters and attitude control parameters to each waypoint in the flight waypoint sequence, and generating basic flight path data with kinematic constraints; calculating the field of view coverage of the sensor configuration scheme on the flight waypoint sequence based on the geographical location of the object of interest in the contextualized inspection command, and selecting key observation waypoints for data collection from the flight waypoint sequence; binding the acquisition trigger command and parameters in the sensor configuration scheme to the key observation waypoints, and temporally encapsulating the binding result with the basic flight path data to generate the optimized flight inspection task.

[0044] Specifically, the continuous space within the dynamic safety corridor is meshed. Using the calculated virtual repulsive force intensity as the path cost function, a path search algorithm such as A* or RRT* is employed to search for a path in three-dimensional space that minimizes the sum of the repulsive potential energy at all points along the path—the optimized repulsive potential energy path. This path naturally remains in the central region of the dynamic safety corridor, maximizing its distance from all static and dynamic obstacles. Subsequently, this continuous path is spatially discretized. Based on a set waypoint spacing, such as 5 to 10 meters, it is transformed into an ordered set containing three-dimensional coordinates, sequential numbers, and estimated arrival times—a flight waypoint sequence with temporal information.

[0045] The system analyzes the determined mission flight mode, extracting cruise speed parameters (e.g., 8 meters per second) and attitude control parameters (e.g., maximum roll angle of 20 degrees and maximum angular velocity of 15 degrees per second). Then, it iterates through the flight waypoint sequence, calculating and assigning target velocity vectors and attitude quaternions to each waypoint. This calculation process ensures that the UAV's acceleration, deceleration, and turning maneuvers between adjacent waypoints are smooth and within dynamically permissible limits. The output of this process is a four-dimensional spatiotemporal trajectory containing complete kinematic constraints, i.e., the basic flight path data.

[0046] Based on the 3D geographic location model of the object of interest in the contextualized inspection command, and combined with the optical parameters in the determined sensor configuration scheme, such as focal length and sensor size, the field of view coverage is calculated for each waypoint on the basic flight path data. For any waypoint in the sequence, based on the UAV's position, attitude, and camera parameters at that point, the coverage polygon formed by the projection of its sensor field of view onto the ground or target structure is calculated. Geometric intersection detection is performed to determine whether this coverage polygon overlaps with the spatial bounding box of any object of interest. Once an overlap is detected, the waypoint is filtered out and marked as a key observation waypoint.

[0047] For each key observation waypoint, specific acquisition trigger commands and parameters are bound according to the sensor configuration scheme. For example, at a key observation waypoint for observing cracks in a bridge pier, the bound commands might include "adjust the gimbal pitch angle to -45 degrees," "set the camera optical zoom to 20x," and "trigger a high-definition photo." These "event" waypoints with commands are integrated and encapsulated with the complete basic flight path data in chronological order. This ultimately generates a structured, optimized flight inspection mission file, which defines in detail where the UAV should be, in what attitude it should be flying, and when and where it should perform what data acquisition operations from takeoff to landing every second.

[0048] For example, within the generated dynamic safety corridor, the system uses the A* algorithm to search for an optimized repulsive potential energy path with the minimum accumulated repulsive potential energy, and discretizes it into an ordered sequence of waypoints spaced 8 meters apart. Then, it traverses this sequence, assigning a target velocity vector and attitude quaternion to each waypoint according to the kinematic constraints in the flight mode, forming basic flight path data. Subsequently, for high-risk guardrail objects of interest in the instructions, the system calculates the field-of-view coverage polygon of each waypoint on the basic flight path by combining its three-dimensional geographical location and the focal length parameters of the airborne camera. Through geometric intersection detection, it is found that the field-of-view polygon of waypoint 24 completely covers the guardrail deformation risk point. Therefore, waypoint 24 is marked as a key observation waypoint and bound with specific acquisition instructions such as "adjust gimbal pitch angle to -45 degrees", "trigger continuous shooting", and "record laser point cloud". Finally, the complete data containing time-series trajectory and event actions is encapsulated into an optimized flight inspection task. This method ensures effective observation of key targets by using refined waypoint planning based on field-of-view coverage analysis, avoids the generation of invalid data, and improves the automation level and execution efficiency of inspection operations.

[0049] S4. Perform wide-area anomaly screening and local fine-grained identification on the video stream collected by the UAV during the optimized flight inspection task, and generate anomaly confirmation information containing anomaly type and location; Optionally, generating anomaly confirmation information containing anomaly type and location includes: performing background subtraction and texture analysis on the video stream to generate a first-level detection result containing suspected anomaly areas and preliminary attribute descriptions; controlling the UAV to adjust the optical zoom and shooting angle for each suspected anomaly area in the first-level detection result to collect fine images; and calling dedicated recognition parameters matching the object of interest according to the preliminary attribute description to extract features and classify the fine images to generate anomaly confirmation information containing anomaly type and location.

[0050] Specifically, a background subtraction algorithm combining Gaussian mixture models is applied to consecutive video frames. By comparing the current frame with a dynamically updated background model at the pixel level, it identifies newly appearing static objects or significant structural changes in the scene, such as road potholes or debris. Simultaneously, to detect surface defects that are difficult to detect through background modeling, such as road cracks or guardrail corrosion, a texture analysis algorithm based on the gray-level co-occurrence matrix is ​​run in parallel. This algorithm calculates features such as texture consistency and entropy in local image regions, and any regions deviating from the normal texture statistical model are marked. The outputs of these two algorithms are fused to generate a first-level detection result containing the two-dimensional coordinates and confidence scores of multiple suspected anomalous regions. This result also includes preliminary attribute descriptions such as "moving targets" and "texture mutations."

[0051] For each suspected anomaly area with a confidence level higher than a preset threshold, such as 0.75, a temporary mission command is sent to the UAV's flight control system. This command causes the UAV to switch from cruise scanning mode to fixed-point detailed inspection mode. While maintaining a safe distance, the UAV uses its high-precision positioning module to hover or slowly approach above or to the side of the suspected anomaly area. Simultaneously, the command controls the onboard gimbal and camera's optical zoom lens, adjusting the zoom magnification from the wide-angle 1x to 3x to a higher magnification, typically between 15x and 30x, and possibly from multiple different shooting angles, such as vertical overhead shots and 45-degree angled shots, to acquire a set of detailed images.

[0052] Based on the preliminary attribute descriptions in the first-level detection results, such as "texture mutations," and combined with the corresponding object of interest for the area, such as "road surface," a deep learning model specifically designed for identifying road surface defects is intelligently invoked from the model library. This model could be a convolutional neural network trained on a large number of road surface crack images. The acquired detailed images are then input into this dedicated model for feature extraction and classification. The model outputs the specific anomaly type, such as "transverse crack" or "network crack," and provides a confidence score for the classification. Once the confidence score exceeds the final confirmation threshold, such as 0.95, the result, along with the high-precision 3D geographic coordinates calculated using UAV RTK positioning data and image geometric relationships, is packaged into an anomaly confirmation message. This message is a standardized data record containing a unique anomaly ID, the exact anomaly type, the precise geographic location, a severity assessment, and associated detailed image evidence, thus completing the entire detection process.

[0053] For example, the collaborative detection module performs real-time analysis on the video stream transmitted back by the UAV in cruise mode. It uses a Gaussian mixture model for background subtraction and combines this with gray-level co-occurrence matrix analysis to identify texture features. At kilometer marker K351+300, a region with significantly decreased texture consistency is identified as a suspected anomaly with a confidence level of 0.78. The system generates a first-level detection result containing the attribute "texture mutation." In response to this result, the UAV immediately switches to a fixed-point detailed inspection mode, hovers, and zooms the camera to 20x, acquiring detailed images from both vertical and 45-degree oblique angles. The system then calls a dedicated road surface defect convolutional neural network model to process these images, identifying the specific defect type as "cracking" with a classification confidence level of 0.96. The system combines RTK positioning data to generate anomaly confirmation information including "Type: Cracking," "Coordinates: E113.12, N23.56," and "Severity: Medium." This method employs a collaborative detection strategy combining coarse screening and fine inspection, ensuring both the speed of wide-area inspection and the accuracy and completeness of anomaly identification evidence.

[0054] Optionally, the method further includes: obtaining historical traffic flow data within the time window of the anomaly occurrence based on the location data and time data in the anomaly confirmation information; performing spatiotemporal correlation analysis on the anomaly confirmation information and the historical traffic flow data to generate a traffic event correlation determination result; if the traffic event correlation determination result indicates a correlation, extracting the relevant traffic event records and appending them to the anomaly confirmation information.

[0055] Specifically, once a new anomaly confirmation message is generated, key location data, such as the station number K123+450 meters, and precise time data, such as the event timestamp, are immediately extracted from it. Based on this timestamp, an anomaly occurrence time window is defined. This time window is typically set to a period of time before the event and a brief extension after the event, such as from T minus 5 minutes to T plus 1 minute, to capture the traffic flow evolution before the event and the immediate impact after the event.

[0056] The system maps anomaly location data to specific traffic flow detector sections and requests historical traffic flow data for that section and its upstream 1-2 km range within a defined anomaly occurrence time window. The returned data is high-frequency time series data, with sampling intervals typically between 20 seconds and 1 minute, and includes key indicators such as traffic volume, average speed, and lane occupancy.

[0057] Time-series anomaly detection algorithms, such as the 3-sigma criterion based on a sliding window or exponentially weighted moving average, are used to identify abrupt changes in traffic flow parameters. A typical correlation determination rule is that if, within the time window of the anomaly, the average speed at the target section decreases by more than a preset threshold (e.g., 40 km / h) within a short period (e.g., two consecutive sampling periods), and lane occupancy increases sharply by more than 25%, a traffic event is determined. The result of this analysis is a Boolean-type traffic event correlation determination.

[0058] If the determination result indicates a correlation, data fusion and information enhancement operations will be performed. Based on the time and location of the traffic incident, matching traffic incident records automatically generated by other systems, such as video surveillance or loop detectors, will be retrieved from the traffic incident log database. These records may contain tags such as "congestion," "accident," or "emergency stop." These extracted traffic incident records will be appended to the original anomaly confirmation information to form a comprehensive incident file that includes physical damage details and related traffic dynamics.

[0059] For example, after generating a "crack" anomaly confirmation message with a timestamp of 14:00:00, the system automatically defines the anomaly occurrence time window from 13:55:00 to 14:05:00 and initiates a query to the traffic big data platform to obtain micro-traffic flow data of the K351 section within this time period. The system performs spatiotemporal correlation analysis on the returned data and detects that around 14:00:30, the average vehicle speed at this section drops sharply from 100 km / h to 55 km / h, and the lane occupancy rate surges from 15% to 40%, meeting the preset traffic event judgment rules. The system then determines that the physical defect has affected traffic operation and further retrieves and integrates "vehicle avoidance" event records reported by the video surveillance system during the same time period, ultimately generating a comprehensive event file containing details of physical facility damage and associated traffic congestion. This method, through cross-system multi-source data fusion analysis, reveals the dynamic causal relationship between facility status and traffic flow, providing management departments with more valuable decision-making support information.

[0060] S5. In response to the anomaly confirmation information, obtain the handling feedback data generated during the handling process, and use the anomaly confirmation information and the handling feedback data to perform data update operations on the knowledge graph and the configuration optimization relationship.

[0061] Optionally, the response to the handling feedback data generated during the abnormality confirmation information acquisition and handling process includes: when the abnormality type is an immediate abnormality, searching for roadside fixed monitoring devices whose geographical distance meets preset proximity conditions based on the location in the abnormality confirmation information; sending control commands containing target orientation and zoom parameters to the roadside fixed monitoring devices to generate a monitoring perspective covering the abnormal location; constructing a synchronous transmission channel containing a video stream from the drone's perspective and a video stream from the monitoring perspective, and pushing the data from the synchronous transmission channel to the handling terminal to obtain handling feedback data.

[0062] Specifically, the anomaly type in the anomaly confirmation information is analyzed. When it is determined to be an immediate anomaly, such as a traffic accident, vehicle fire, or large road obstruction, the emergency response procedure is immediately activated. Based on the precise location coordinates in the anomaly confirmation information, a spatial query is performed in the highway fixed asset database to retrieve all roadside fixed monitoring devices that meet the preset proximity conditions, such as roadside fixed monitoring equipment within 1 kilometer of the anomaly point. These devices are usually high-point monitoring cameras installed on gantry frames or poles.

[0063] Based on the geographic coordinates of the anomaly point and the selected roadside fixed monitoring device, geometric calculations determine the required target orientation for the camera, including the horizontal azimuth and vertical pitch angles. Simultaneously, based on the straight-line distance between the two points, appropriate zoom parameters are automatically calculated and set to ensure the anomaly location is both clear and has a suitable context in the image. For example, for an anomaly point 500 meters away, a 15x optical zoom is set. These calculated parameters are encapsulated into a standard control command, such as the PTZ control command in the ONVIF protocol, and sent over the network to the controller of the roadside fixed monitoring device.

[0064] After sending the control command, two parallel video stream retrieval channels are established. The first channel acquires the real-time video stream captured by the onboard camera of a drone hovering or circling above the anomaly point, representing the drone's perspective. The second channel acquires the adjusted monitoring perspective video stream from the roadside fixed monitoring equipment that has just been remotely controlled. By aligning the frame timestamps of the two video streams, millisecond-level playback synchronization is achieved, and the two video streams are displayed in a split-screen or picture-in-picture layout within a single frame. The data from this merged synchronous transmission channel is encoded as a whole and prepared for transmission to the processing terminal.

[0065] Data from this synchronous transmission channel is pushed to the handling terminal, and a data link is established for receiving handling instructions and status feedback. The handling terminal can be a large video wall in the monitoring center, a desktop computer for command personnel, or a mobile device for field personnel. While pushing video, interactive tools are provided on the terminal interface, allowing handling personnel to input instructions, such as "notify road administration to proceed with cleanup," or mark the event status, such as "confirmed, currently being handled." These manually input instructions, status updates, and handling results constitute the handling feedback data.

[0066] For example, the closed-loop handling and learning module identifies the "crack" anomaly at K351 as requiring manual verification. It quickly retrieves the roadside high-point monitoring camera 150 meters away from the anomaly from the asset database. Based on the high-precision 3D coordinates of both cameras, it performs geometric calculations to generate a PTZ control command containing "azimuth 235 degrees, pitch -12 degrees, zoom 8x" and sends it to the device. The system then establishes a synchronous transmission channel, performing frame-level synchronization and picture-in-picture synthesis between the UAV's aerial view video stream and the roadside monitoring's head-up video stream, pushing the data to the command center's large screen. It also receives real-time handling instructions from personnel via terminals, such as "Maintenance work area has been dispatched," and stores these as handling feedback data in conjunction with the original anomaly information. This method, through automated air-to-ground video linkage, provides command personnel with comprehensive, real-time situational awareness capabilities and achieves a fully digital closed loop from discovery to handling.

[0067] Optionally, the data update operation of the knowledge graph and the configuration optimization relationship using anomaly confirmation information and handling feedback data includes: converting the anomaly confirmation information into historical disease data nodes and writing them into the knowledge graph; extracting post-repair status data from the handling feedback data and using the post-repair status data to correct the facility's health decay parameter; performing correlation statistics on the accumulated anomaly confirmation information and the handling feedback data, and using the statistical results to adjust the mapping weights of scene features and configuration schemes in the configuration optimization relationship.

[0068] Specifically, once the anomaly confirmation information, including the anomaly type, location, and image evidence, is finally confirmed, it is automatically converted into a standardized historical disease data node. This node contains attributes such as disease type, severity level, discovery time, and precise coordinates. Subsequently, a graph database write operation is performed to add this new node to the knowledge graph and establish its association with other relevant nodes. For example, the disease node is connected to its specific facility node, such as "gantry at K105+300," via an edge indicating "occurred at," and to the corresponding road segment node via an edge indicating "located at."

[0069] Key post-repair status data is extracted from the handling feedback data, including qualitative descriptions such as "repair completed" and "new parts replaced," as well as quantitative health scores provided by maintenance personnel or re-inspection equipment, such as "health restored to 98%." The node of the facility in the knowledge graph is then located, and its associated health degradation parameters are invoked. These parameters are typically one or a set of coefficients characterizing the rate of decline in facility performance over time, load, and environmental factors. By comparing the time of the anomaly with the model's predicted failure time, if the actual occurrence time is earlier than the prediction, the facility's health degradation parameter is increased; conversely, it is decreased, thus making the model's predictions closer to actual operating conditions.

[0070] By statistically analyzing large-scale accumulated data, the inspection strategy itself is optimized and upgraded. A comprehensive correlation analysis is periodically performed on all anomaly confirmation information and handling feedback data accumulated in the database. This analysis aims to answer the question, "Under what scenario characteristics, which sensor configuration and flight mode yields the highest success rate in detecting specific types of anomalies?" For example, statistical results might show that in a scenario like a "winter morning with light fog," using a configuration of "visible light zoom camera combined with an infrared thermal imager" to inspect "bridge expansion joints" increases the probability of detecting "foreign object blockage" anomalies by 60% compared to using only a visible light camera. Based on these statistical results, the mapping weights in the pre-stored configuration optimization relationships are quantitatively adjusted. Specifically, the mapping weight values ​​from high-success-rate scenario characteristics to corresponding configuration schemes are increased, while the weights of low-success-rate combinations are decreased. This allows for the automatic recommendation and adoption of the optimal inspection strategy, proven in practice, in future mission planning.

[0071] For example, the system transforms the anomaly confirmation information of "cracking" at K351 into standardized historical defect nodes, writes them into the knowledge graph, and establishes a connection between them and the "K351 section" node. Subsequently, based on the information of "severe actual road surface aging" recorded in the handling feedback data, the system compares and finds that the occurrence time of this defect is significantly earlier than the prediction time of the original health model. Therefore, it uses the feedback data to calibrate the health degradation parameter of this section, increasing its annual degradation rate from 3% to 4.5%. At the same time, the system performs statistical analysis on the quarterly cumulative data and finds that in the scenario of "strong winds and insufficient sunlight," the anomaly detection rate of the "wind-resistant mode + lidar" solution is 55% higher than that of using only visible light cameras. Based on this, the system updates the configuration optimization relationship and increases the recommendation weight of the lidar configuration solution that points to the characteristics of this scenario. This method, through self-learning and model calibration based on real data, realizes the automatic updating of the knowledge base and the adaptive evolution of the inspection strategy, ensuring the long-term vitality and accuracy of the system.

[0072] Based on the same inventive concept, as shown in Figure 4, this invention also provides a UAV-based intelligent highway inspection and abnormal state handling system, comprising: an instruction parsing module, used to perform semantic decomposition and knowledge graph association retrieval on the acquired natural language input to obtain a contextualized inspection instruction containing the geographical range of the target road segment, the object of interest, and the characteristics of the associated scene; and a dynamic planning module, used to receive the contextualized inspection instruction, acquire basic geographic information and real-time dynamic environmental data of the geographical range of the target road segment, and use the contextualized inspection instruction to perform three-dimensional potential field repulsion calculation on the basic geographic information and real-time dynamic environmental data to generate a dynamic safe route defining the airspace where the UAV can fly. The system comprises: a corridor; a task optimization module, used to extract sensor configuration schemes and flight modes from pre-stored configuration optimization relationships based on the associated scene features, and map them to waypoints in the dynamic safety corridor to generate optimized flight inspection tasks; a collaborative detection module, used to perform wide-area anomaly screening and local fine-grained identification on the video stream collected by the UAV during the execution of the optimized flight inspection tasks, generating anomaly confirmation information containing anomaly type and location; and a closed-loop handling and learning module, used to obtain handling feedback data generated during the handling process in response to the anomaly confirmation information, and to update the knowledge graph and the configuration optimization relationships using the anomaly confirmation information and handling feedback data.

[0073] It should be noted that the electrical connections between the various units described above do not necessarily represent direct or indirect connections. Any indirect connection method can be applied to the embodiments of the present invention as long as it achieves the purpose of the present invention. The above descriptions are merely exemplary embodiments of the present invention and should not be construed as limiting the scope of the present invention.

[0074] All equivalent changes and modifications made in accordance with the teachings of this invention are still within the scope of this invention. Those skilled in the art will readily conceive of other embodiments of this invention upon considering the specification and the disclosure of practical truth. This application is intended to cover any variations, uses, or adaptations of this invention that follow the general principles of this invention and include common knowledge or conventional techniques in the art not described herein.

Claims

1. A method for intelligent highway inspection and abnormal condition handling based on unmanned aerial vehicles (UAVs), characterized in that, The method includes: performing semantic decomposition and knowledge graph association retrieval on the acquired natural language input to obtain a contextualized inspection instruction containing the geographical range of the target road segment, the object of interest, and the features of the associated scene; receiving the contextualized inspection instruction, acquiring basic geographic information and real-time dynamic environmental data of the geographical range of the target road segment, and using the contextualized inspection instruction to perform three-dimensional potential field repulsion calculation on the basic geographic information and real-time dynamic environmental data to generate a dynamic safety corridor defining the airspace where the UAV can fly; extracting sensor configuration schemes and flight modes from pre-stored configuration optimization relationships based on the associated scene features, and mapping them to waypoints of the dynamic safety corridor to generate an optimized flight inspection task; performing wide-area anomaly screening and local fine-grained identification on the video stream collected by the UAV during the execution of the optimized flight inspection task to generate anomaly confirmation information containing anomaly type and location; responding to the anomaly confirmation information to acquire the handling feedback data generated during the handling process, and using the anomaly confirmation information and the handling feedback data to perform data update operations on the knowledge graph and the configuration optimization relationship.

2. The method for intelligent highway inspection and abnormal status handling based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The process of obtaining a contextualized inspection instruction containing the geographical range of the target road segment, the object of interest, and the features of related scenarios includes: performing semantic parsing on the natural language input to extract the road segment description text and the demand description text; performing topological matching between the road segment description text and the road segment node data in the knowledge graph to determine the geographical range of the target road segment; performing feature association matching and risk probability calculation between the demand description text and historical disease data, meteorological time series data, and facility archive data associated with the geographical range of the target road segment in the knowledge graph to generate the features of the object of interest and the features of related scenarios; and encapsulating the geographical range of the target road segment, the object of interest, and the features of related scenarios to generate a contextualized inspection instruction.

3. The method for intelligent highway inspection and abnormal status handling based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The process of generating a dynamic safety corridor defining the airspace where UAVs can fly includes: acquiring real-time traffic flow density data and real-time micro-meteorological data of the target road segment from a traffic monitoring platform and a meteorological monitoring network, respectively, to generate real-time dynamic environmental data; acquiring basic geographic information from a map service that provides geographic information as static obstacle parameters, and converting the real-time dynamic environmental data into dynamic repulsive force field parameters; and calculating the spatial virtual repulsive force intensity of the static obstacle parameters and the dynamic repulsive force field parameters based on contextualized inspection commands to define the three-dimensional spatial boundary and generate a dynamic safety corridor defining the airspace where UAVs can fly.

4. The method for intelligent highway inspection and abnormal status handling based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The step of extracting sensor configuration schemes and flight modes from pre-stored configuration optimization relationships based on the associated scene features includes: extracting material features from the associated scene features, indexing sensor type combinations from the configuration optimization relationships; extracting environmental risk features from the associated scene features, indexing mission flight modes including switching logic between cruise scanning mode and fixed-point detailed investigation mode from the configuration optimization relationships; and combining and binding the sensor type combinations with the mission flight modes to generate sensor configuration schemes and flight modes.

5. The method for intelligent highway inspection and abnormal status handling based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The optimized flight inspection task generation includes: extracting an optimized repulsive potential energy path within the three-dimensional boundary of the dynamic safety corridor, and spatially discretizing the path to generate a flight waypoint sequence containing temporal information; parsing the cruise speed parameters and attitude control parameters in the flight mode, assigning the cruise speed parameters and attitude control parameters to each waypoint in the flight waypoint sequence, and generating basic flight path data with kinematic constraints; calculating the field of view coverage of the sensor configuration scheme on the flight waypoint sequence based on the geographical location of the object of interest in the contextualized inspection command, and selecting key observation waypoints for data collection from the flight waypoint sequence; binding the acquisition trigger command and parameters in the sensor configuration scheme to the key observation waypoints, and temporally encapsulating the binding result with the basic flight path data to generate the optimized flight inspection task.

6. The method for intelligent highway inspection and abnormal status handling based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The process of generating anomaly confirmation information containing anomaly type and location includes: performing background subtraction and texture analysis on the video stream to generate a first-level detection result containing suspected anomaly regions and preliminary attribute descriptions; controlling the drone to adjust the optical zoom and shooting angle for each suspected anomaly region in the first-level detection result to acquire a detailed image; and using dedicated recognition parameters matching the object of interest to extract features and classify the detailed image based on the preliminary attribute description to generate anomaly confirmation information containing anomaly type and location.

7. The method for intelligent highway inspection and abnormal status handling based on unmanned aerial vehicles (UAVs) according to claim 6, characterized in that, The method further includes: obtaining historical traffic flow data within the time window of the anomaly occurrence based on the location data and time data in the anomaly confirmation information; performing spatiotemporal correlation analysis on the anomaly confirmation information and the historical traffic flow data to generate a traffic event correlation determination result; if the traffic event correlation determination result indicates a correlation, extracting the relevant traffic event records and appending them to the anomaly confirmation information.

8. The method for intelligent highway inspection and abnormal status handling based on unmanned aerial vehicles (UAVs) according to claim 6, characterized in that, The response to the abnormality confirmation information acquisition and handling process generates the following handling feedback data: when the abnormality type is an immediate abnormality, based on the location in the abnormality confirmation information, search for roadside fixed monitoring devices whose geographical distance meets preset proximity conditions; send control commands containing target orientation and zoom parameters to the roadside fixed monitoring devices to generate a monitoring perspective covering the abnormality location; construct a synchronous transmission channel containing a video stream from the drone's perspective and a video stream from the monitoring perspective, and push the data from the synchronous transmission channel to the handling terminal to obtain handling feedback data.

9. The method for intelligent highway inspection and abnormal status handling based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The data update operation of the knowledge graph and the configuration optimization relationship using anomaly confirmation information and handling feedback data includes: converting the anomaly confirmation information into historical disease data nodes and writing them into the knowledge graph; extracting post-repair status data from the handling feedback data and using the post-repair status data to correct the facility's health decay parameter; performing correlation statistics on the accumulated anomaly confirmation information and the handling feedback data, and using the statistical results to adjust the mapping weights of scene features and configuration schemes in the configuration optimization relationship.

10. A highway intelligent inspection and abnormal status handling system based on unmanned aerial vehicles (UAVs), characterized in that: The system includes: an instruction parsing module, used to perform semantic decomposition and knowledge graph association retrieval on the acquired natural language input to obtain contextualized inspection instructions containing the geographical range of the target road segment, the object of interest, and the characteristics of the associated scene; a dynamic planning module, used to receive the contextualized inspection instructions, acquire basic geographic information and real-time dynamic environmental data of the geographical range of the target road segment, and use the contextualized inspection instructions to perform three-dimensional potential field repulsion calculation on the basic geographic information and real-time dynamic environmental data to generate a dynamic safety corridor that defines the airspace where the UAV can fly; a task optimization module, used to extract sensor configuration schemes and flight modes from pre-stored configuration optimization relationships based on the associated scene characteristics, and map them to waypoints of the dynamic safety corridor to generate optimized flight inspection tasks; a collaborative detection module, used to perform wide-area anomaly screening and local fine-grained identification on the video stream collected by the UAV during the execution of the optimized flight inspection tasks, and generate anomaly confirmation information containing anomaly type and location; and a closed-loop handling and learning module, used to respond to the anomaly confirmation information to acquire handling feedback data generated during the handling process, and use the anomaly confirmation information and handling feedback data to perform data update operations on the knowledge graph and the configuration optimization relationships.

Images