Urban and rural area intelligent inspection method and system based on multi-unmanned aerial vehicle cooperation

Abstract

This invention provides a method and system for intelligent inspection of urban and rural areas based on multi-UAV collaboration, comprising: performing spatial overlay analysis on multi-source heterogeneous data of urban and rural areas to obtain a dynamic risk heat map; converting the dynamic risk heat map into a risk spatial potential energy field and performing adaptive grid clustering to obtain a standard task package set; using the risk spatial potential energy field as a global scheduling constraint, allocating tasks to the standard task package set according to the UAV performance feature set to obtain task allocation results; performing multi-dimensional collaborative constraints and collaborative path planning on the task allocation results to obtain a collaborative inspection track set; controlling the UAV swarm to execute the standard task package set according to the collaborative inspection track set; and performing multi-source data collection, fusion analysis, and event identification in urban and rural areas to obtain inspection event identification results. This invention can improve the accuracy and reliability of inspection event identification.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and image recognition technology, and in particular to an intelligent inspection method and system for urban and rural areas based on multi-UAV collaboration. Background Technology

[0002] In the fields of smart city governance and urban and rural infrastructure security, intelligent inspection technology based on multi-UAV collaboration has become an important development direction for achieving large-scale, high-frequency, and refined monitoring. Especially in application scenarios such as power grid inspection, river flood control, transportation facility monitoring, and public safety assurance, the system needs to fuse and analyze multi-source heterogeneous data in complex urban and rural environments, and combine the collaborative scheduling and path planning of UAV swarms to achieve priority coverage and dynamic response to high-risk areas. This approach is characterized by "complex spatial distribution, strong time-varying risk, and high coupling of multiple tasks." How to construct a unified risk representation model, transform discrete risk information into a continuous spatial representation that can be used for task scheduling and path guidance, and achieve dynamic optimization of task allocation and path planning during multi-UAV collaboration has become a key issue in improving inspection efficiency and intelligence.

[0003] However, existing UAV inspection methods still have significant technical limitations when facing complex urban and rural environments and multi-task collaboration. First, traditional inspection schemes often rely on fixed routes or path planning strategies based on simple rules, lacking the ability to uniformly model multi-source heterogeneous risk data. This makes it difficult to dynamically perceive and efficiently represent risk distribution, leading to uneven allocation of inspection resources and insufficient coverage of high-risk areas. Second, existing multi-UAV task allocation methods typically prioritize distance or energy consumption as optimization objectives, lacking comprehensive consideration of task risk levels, capability matching, and environmental constraints. This makes it difficult to achieve refined scheduling under multi-factor coupling, easily resulting in unreasonable task allocation or low resource utilization efficiency. Furthermore, existing methods often fail to effectively integrate collision avoidance constraints, spatiotemporal occupancy constraints, and communication link constraints during path planning, lacking a unified collaborative optimization mechanism, making it difficult to guarantee the safety and collaboration of multiple UAVs in complex environments. Furthermore, in terms of inspection data processing, traditional methods rely heavily on single visual information for target recognition, lack the integration and utilization of environmental status data, and are difficult to maintain stable recognition performance under complex conditions such as changes in lighting and weather disturbances. At the same time, they lack anomaly discrimination mechanisms based on historical semantic knowledge and spatiotemporal correlation, resulting in insufficient accuracy and reliability of event recognition. Summary of the Invention

[0004] (a) Technical problems to be solved To address the shortcomings of existing technologies, this invention provides a method and system for intelligent inspection of urban and rural areas based on multi-UAV collaboration. It has the advantages of dynamic risk field driving and multi-agent collaborative decision-making, and solves the problems of suboptimal resource scheduling, low collaborative efficiency, high operational safety risks, and high false alarm rate of intelligent identification caused by the lack of dynamic risk fusion modeling, disconnect between task allocation and risk situation, weak collaborative planning under multi-dimensional constraints, and insufficient reliability of multi-modal event recognition in the process of multi-UAV urban and rural inspection.

[0005] (II) Technical Solution To achieve the above objectives, the present invention provides the following technical solution: This invention provides a method for intelligent inspection of urban and rural areas based on multi-UAV collaboration, comprising the following steps: Spatial overlay analysis of multi-source heterogeneous data in urban and rural areas is performed to obtain a dynamic risk heat map. Based on the heterogeneous task density function, the dynamic risk heat map is transformed into a risk spatial potential energy field. The risk spatial potential energy field is then subjected to adaptive grid clustering to obtain a standard task package set. A set of UAV performance characteristics is constructed based on the performance parameters of the UAV swarm. The risk space potential energy field is used as a global scheduling constraint. Based on the improved contract net algorithm, the standard task package set is assigned tasks according to the set of UAV performance characteristics to obtain the task assignment results. Using the risk space potential energy field as the objective function, multi-dimensional collaborative constraints and collaborative path planning are applied to the task allocation results to obtain a collaborative inspection track set. The multi-dimensional collaborative constraints include UAV collision avoidance constraints, spatiotemporal occupancy constraints, and communication link constraints. The drone swarm is controlled to execute the standard task package set according to the collaborative inspection track set, and to collect multi-source data on the urban and rural areas to obtain inspection image data and environmental status data. The inspection image data and environmental status data are fused and analyzed, and event recognition is performed to obtain the inspection event recognition results.

[0006] According to a preferred embodiment of the present invention, the process of spatial overlay analysis of multi-source heterogeneous data in urban and rural areas includes: Acquire multi-source heterogeneous data from urban and rural areas, and perform coordinate system unification, spatial resolution alignment, and format standardization on the multi-source heterogeneous data to obtain a standard data layer set; Spatiotemporal correlation analysis is performed on the standard data layer set to identify spatial co-occurrence and temporal evolution patterns among different standard data layers, and the analysis results are obtained. Based on the analysis results, the risk of the standard data layer set is quantified to obtain a multidimensional risk feature layer set. Based on the preset inspection targets, inspection weights are assigned to the multidimensional risk feature layer set, and the multidimensional risk feature layer set is fused based on the grid weighted overlay algorithm to obtain an initial risk value grid map. Dynamic change features are extracted in real time from the standard data layer set, and the dynamic change features are mapped to the initial risk value raster map based on the spatiotemporal matching algorithm to perform dynamic risk fusion and obtain a dynamic risk value distribution map. The risk values ​​in the dynamic risk value distribution map are normalized and color-mapped to obtain a dynamic risk heat map.

[0007] According to another preferred embodiment of the present invention, the process of converting the dynamic risk heatmap into a risk space potential field based on the heterogeneous task density function includes: Based on the inspection target, an inspection target object set is extracted from the standard data layer set, and a heterogeneous task density function is constructed according to the target object type of the inspection target object set; Configure a task density parameter table for the heterogeneous task density function, wherein the task density parameter table includes the influence radius, spatial decay factor and basic weights; Based on the heterogeneous task density function and the corresponding task density parameter table, the potential energy contribution value of each inspection target object in the inspection target object set is calculated, and all potential energy contribution values ​​are spatially superimposed to obtain the discrete task potential energy field. The normalized risk value of each pixel is extracted from the dynamic risk heat map to obtain the risk value field. The discrete task potential field and the risk value field are then fused to obtain the original fused potential field. The original fused potential field is smoothed by Gaussian convolution and normalized globally to obtain the risk space potential field.

[0008] According to another preferred embodiment of the present invention, the process of adaptively clustering the risk space potential energy field includes: The potential energy gradient field is obtained by calculating the spatial gradient of the risk space potential energy field. Based on the potential energy gradient field, local maxima and high gradient boundary regions are extracted, and a candidate anchor point set is constructed based on the local maxima and high gradient boundary regions. Based on preset potential energy similarity thresholds and spatial distance thresholds, the candidate anchor point set is merged to obtain an initial clustering seed point set; Based on gradient constraints, region growing clustering is performed on each initial clustering seed point in the initial clustering seed point set to obtain an initial cluster set. The initial cluster set is subjected to cluster merging, cluster segmentation, and geometric optimization to obtain a candidate task package set; Add task attributes to each candidate task package in the candidate task package set, and encapsulate them into a standard task package set.

[0009] According to another preferred embodiment of the present invention, the process of allocating tasks to the standard task package set based on the UAV performance feature set using the improved contract network algorithm includes: Using the gradient direction of the potential energy field in the risk space as the inspection guidance direction, the standard task packages in the standard task package set are selected one by one as target task packages in descending order of potential energy. The target task package is parsed to obtain task requirement features, which include space requirement features, performance requirement features, and risk requirement features. Based on the task requirement characteristics, a dynamic task announcement for the target task package is generated, and the task requirement characteristics of all dynamic task announcements are aggregated into a task requirement characteristic set. Based on the UAV performance feature set and mission requirement feature set, the matching relationship between the UAV and the standard mission package is established one by one, the corresponding estimated matching cost is calculated, and the estimated matching cost matrix is ​​constructed. The estimated matching cost includes path cost, capability adaptation cost and energy consumption cost. Based on the estimated matching cost matrix, bidding decisions, conflict resolution, and consistency verification are performed on each dynamic task announcement to obtain the task allocation results.

[0010] According to another preferred embodiment of the present invention, the process of making bidding decisions, resolving conflicts, and verifying consistency for each dynamic task announcement based on the estimated matching cost matrix includes: Each dynamic task announcement is selected as the target task announcement, and the estimated matching cost of each UAV in the UAV swarm for the target task announcement is extracted from the estimated matching cost matrix to obtain the estimated cost set. Extract the state factors of each UAV and calculate the bidding cost set by combining them with the estimated cost set; The drone with the smallest bid cost in the set of bid costs is selected as the winning drone in the target mission announcement, and a primary allocation list is generated based on each target mission announcement and the corresponding winning drone. Spatiotemporal conflict detection is performed on each winning UAV in the primary allocation list according to a preset sliding spatiotemporal window to obtain conflict task packages; Conflict resolution is performed based on the bid cost set of the conflict task package, and the primary allocation list is updated based on the result of conflict resolution to obtain the task allocation list. A global consistency check based on risk priority is performed on the task allocation list to obtain the task allocation result.

[0011] According to another preferred embodiment of the present invention, the process of performing multi-dimensional collaborative constraints and collaborative path planning on the task allocation results includes: Based on the task allocation results, the spatial boundary and potential energy centroid coordinates of the target task package corresponding to each matched UAV are extracted, and an initial guiding vector field is constructed by combining the gradient distribution of the risk space potential energy field. Based on the initial guidance vector field, the initial reference trajectory of each matched UAV is calculated, and the relative displacement and estimated arrival time between each matched UAV are calculated according to the initial reference trajectory, thus constructing a spatiotemporal occupancy prediction model. Based on the aforementioned spatiotemporal occupancy prediction model, UAV collision avoidance constraints and spatiotemporal occupancy constraints are introduced to construct a cooperative obstacle avoidance potential field. Communication link constraints are introduced into the cooperative obstacle avoidance potential field to construct a safe and feasible region; The initial reference trajectories of each matched UAV are dynamically optimized within the safe and feasible domain based on the rolling time-domain optimization algorithm to obtain a collaborative inspection trajectory set.

[0012] According to another preferred embodiment of the present invention, the process of introducing communication link constraints and constructing a safe and feasible region in the cooperative obstacle avoidance potential field includes: A signal gain distribution map of the urban and rural areas is constructed, and spatial interpolation is performed on the signal gain distribution map to obtain a continuous signal gain field; A communication link constraint is introduced, and based on the signal gain field, regions where the signal strength is lower than a preset strength threshold are identified as regions where the communication constraint is violated. The communication constraint violation region is transformed into a repulsive potential energy term and fused with the cooperative obstacle avoidance potential field to obtain a comprehensive spatial potential field. Based on the comprehensive spatial potential field, a set of regions that satisfy the UAV collision avoidance constraints, spatiotemporal occupancy constraints, and communication link constraints and also satisfy spatial continuity are extracted. Then, the spatiotemporal occupancy prediction model is used to filter out regions that satisfy time accessibility from the set of regions to obtain candidate safe regions. Connectivity analysis and reachability verification are performed on the candidate safe regions. Based on the analysis and verification results, isolated regions in the candidate safe regions are eliminated to obtain a safe and feasible region that satisfies collision avoidance constraints, spatiotemporal occupancy constraints, and communication link constraints.

[0013] According to another preferred embodiment of the present invention, the process of fusing and analyzing the inspection image data and environmental status data and identifying events includes: Spatiotemporal alignment is performed based on the collection timestamps and collection coordinates marked in the inspection image data and environmental status data to obtain aligned multimodal data; By combining the environmental state data in the aligned multimodal data, the inspection image data is preprocessed and environmental feature compensation is performed. Then, the compensated inspection image data is extracted to obtain a multidimensional enhanced feature set. By combining the prior knowledge provided by the standard task package set, target detection and semantic segmentation are performed on the multidimensional enhanced feature set to obtain the semantic set of the inspection object; The semantic set of the inspection objects is semantically matched with the preset historical risk semantic database, and abnormal deviations are extracted from the semantic matching results based on the spatiotemporal correlation analysis method to obtain potential inspection events. Based on the multi-criteria decision analysis method, the potential inspection events are subjected to logical consistency verification and confidence verification to obtain the inspection event identification results.

[0014] To achieve at least one of the above-mentioned objectives, the present invention further provides an intelligent inspection system for urban and rural areas based on multi-UAV collaboration, the system comprising a grid division module, a task allocation module, a path planning module, an inspection and data collection module, and a fusion analysis module; The grid partitioning module is used to perform spatial overlay analysis on multi-source heterogeneous data in urban and rural areas to obtain a dynamic risk heat map. Based on the heterogeneous task density function, the dynamic risk heat map is transformed into a risk spatial potential energy field, and the risk spatial potential energy field is subjected to adaptive grid clustering partitioning to obtain a standard task package set. The task allocation module is used to construct a UAV performance feature set based on the performance parameters of the UAV swarm, use the risk space potential energy field as a global scheduling constraint, and allocate tasks to the standard task package set based on the UAV performance feature set using the improved contract net algorithm to obtain the task allocation result. The path planning module is used to perform multi-dimensional collaborative constraints and collaborative path planning on the task allocation results with the risk space potential energy field as the objective function, so as to obtain a collaborative inspection track set. The multi-dimensional collaborative constraints include UAV collision avoidance constraints, spatiotemporal occupancy constraints and communication link constraints. The inspection and data acquisition module is used to control the UAV swarm to execute the standard task package set according to the collaborative inspection track set, and to collect multi-source data on the urban and rural areas to obtain inspection image data and environmental status data. The fusion analysis module is used to perform fusion analysis and event recognition on the inspection image data and environmental status data to obtain the inspection event recognition results.

[0015] (III) Beneficial Effects Compared with existing technologies, this invention provides a method and system for intelligent inspection of urban and rural areas based on multi-UAV collaboration, which has the following beneficial effects: This intelligent inspection method for urban and rural areas based on multi-UAV collaboration achieves a unified quantitative expression of risk distribution in urban and rural areas by integrating multi-source heterogeneous data and constructing a dynamic risk heat map. By introducing a heterogeneous task density function, discrete risk information is transformed into a continuous risk space potential energy field, giving the inspection task a physical guidance mechanism. Key structural features are extracted through potential energy gradient analysis, and combined with a gradient-constrained region growth clustering method, adaptive partitioning of complex spaces is achieved, avoiding the task fragmentation and resource waste problems caused by traditional fixed grid partitioning. By generating a standard task package set, the spatial range, risk intensity, and task attributes are structurally encapsulated, providing a unified and computable task unit for subsequent multi-UAV collaborative scheduling and path planning.

[0016] This intelligent inspection method for urban and rural areas based on multi-UAV collaboration constructs a unified estimated matching cost function by structurally modeling the multi-dimensional performance characteristics of UAVs and the characteristics of task requirements, and introduces a risk space potential energy field to modulate the path cost. This transforms task allocation from being driven by a single distance or energy consumption to a multi-factor coupled decision-making process. Through joint modeling of path potential energy integral, capability adaptability, and endurance constraints, UAVs not only consider physical reachability when performing tasks, but also comprehensively evaluate task risk and execution matching degree. At the same time, combined with an improved contract network mechanism, through dynamic task announcements, distributed bidding, and conflict resolution, adaptive collaborative scheduling of multiple UAVs in complex urban and rural environments is achieved. This effectively improves the coverage priority and overall resource utilization efficiency in high-risk areas, and enhances the robustness and real-time response capability of the system in dynamic environments.

[0017] This intelligent inspection method for urban and rural areas based on multi-UAV collaboration generates an initial guidance vector field using a risk space potential energy field, enabling UAVs to adaptively guide to high-value inspection areas. By introducing collision avoidance constraints based on spatiotemporal occupancy prediction and regional capacity constraints, it effectively avoids spatial conflicts and task congestion among multiple UAVs. By combining communication link constraints, it transforms communication blind spots into high-cost areas, enabling proactive avoidance of communication reliability issues during path planning. By employing a rolling time-domain optimization algorithm to perform dynamic trajectory optimization within the safe and feasible domain, it enables the system to respond to environmental changes in real time. This method achieves a unified improvement in the safety, collaboration, and task coverage efficiency of multiple UAVs in complex urban and rural environments, significantly enhancing the intelligence and robustness of the overall inspection system. Attached Figure Description

[0018] Figure 1 The diagram shown is a flowchart of an intelligent inspection method for urban and rural areas based on multi-UAV collaboration according to the present invention.

[0019] Figure 2 The diagram shows the flowchart of the method for allocating tasks to a standard task package set according to the present invention.

[0020] Figure 3 The diagram shown is a flowchart of the method for multi-dimensional collaborative constraints and collaborative path planning according to the present invention.

[0021] Figure 4 The diagram shown is a structural diagram of an intelligent inspection system for urban and rural areas based on multi-UAV collaboration according to the present invention. Detailed Implementation

[0022] The following description is intended to disclose the present invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious modifications will occur to those skilled in the art. The basic principles of the invention defined in the following description can be applied to other embodiments, modifications, improvements, equivalents, and other technical solutions that do not depart from the spirit and scope of the invention.

[0023] It is understood that the term "a" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element can be one, while in another embodiment, the number of the element can be multiple, and the term "a" should not be understood as a limitation on the number.

[0024] Example 1: Please combine Figure 1 This invention discloses an intelligent inspection method for urban and rural areas based on multi-UAV collaboration, the method comprising the following steps: S1. Perform spatial overlay analysis on multi-source heterogeneous data in urban and rural areas to obtain a dynamic risk heat map. Based on the heterogeneous task density function, transform the dynamic risk heat map into a risk spatial potential energy field. Then, perform adaptive grid clustering on the risk spatial potential energy field to obtain a standard task package set.

[0025] The multi-source heterogeneous data includes geospatial data of urban and rural areas, infrastructure data, socioeconomic and environmental data, historical event data and business data, as well as real-time dynamic data. Geospatial data includes GIS vector layers, satellite remote sensing imagery, and DEM terrain models. Socioeconomic and environmental data includes population density distribution, land use types, building age, and key areas (schools, hospitals, and municipal centers). Infrastructure data includes distribution maps and status monitoring data of power grids, water conservancy networks, and transportation networks (bridges, tunnels). Historical event and business data includes records of past safety accidents, natural disaster archives, and inspection and maintenance logs. Real-time dynamic data includes traffic flow and meteorological information. The dynamic risk heat map is a geospatial visualization layer that uses color gradients to intuitively represent the risk level or inspection priority of different geographical areas. "Dynamic" refers to the evolution of risk level or inspection priority over time. For example, the risk heat map of transportation hubs is extremely high during the morning rush hour and decreases at night; the risk of river embankments increases significantly during the rainy season. The risk is automatically recalculated and updated using a preset model, with color depth representing the probability or severity of the risk, providing intuitive weight guidance for drone swarms.

[0026] In detail, the process of spatial overlay analysis of multi-source heterogeneous data in urban and rural areas includes: Acquire multi-source heterogeneous data from urban and rural areas, and perform coordinate system unification, spatial resolution alignment, and format standardization on the multi-source heterogeneous data to obtain a standard data layer set; Spatiotemporal correlation analysis is performed on the standard data layer set to identify spatial co-occurrence and temporal evolution patterns among different standard data layers, and the analysis results are obtained. Based on the analysis results, the risk of the standard data layer set is quantified to obtain a multidimensional risk feature layer set. Based on the preset inspection targets, inspection weights are assigned to the multidimensional risk feature layer set, and the multidimensional risk feature layer set is fused based on the grid weighted overlay algorithm to obtain an initial risk value grid map. Dynamic change features are extracted in real time from the standard data layer set, and the dynamic change features are mapped to the initial risk value raster map based on the spatiotemporal matching algorithm to perform dynamic risk fusion and obtain a dynamic risk value distribution map. The risk values ​​in the dynamic risk value distribution map are normalized and color-mapped to obtain a dynamic risk heat map.

[0027] The unification of the coordinate system refers to transforming data from different sources, such as GIS vector layers, satellite remote sensing imagery, and DEM terrain models, into a unified geographic coordinate system through affine transformation or projection. Spatial resolution alignment refers to resampling data of different resolutions to ensure consistent pixel size across all data points. Format standardization refers to converting data of different formats, such as vector data, text data, and raster data, into a standardized raster or vector data structure. Each standard data layer in the standard data layer set corresponds to heterogeneous data from a single source, and this can be achieved by calculating the Pearson correlation coefficient or... Spatiotemporal autocorrelation index is used to perform spatiotemporal correlation analysis, thereby obtaining analytical results such as "whether the traffic accident rate increases synchronously in areas with high population density" or "whether the risk of dams increases with delay in areas with high rainfall". The risk quantification refers to quantifying each data feature in the standard data layer into risk features. For example, "building age" is mapped to a value between 0 and 1, with 50-year-old buildings recorded as 0.9 (high risk) and 10-year-old buildings recorded as 0.2 (low risk). A mapping function from data to risk index can be established for each data layer based on business rules, expert knowledge, or a trained machine learning model, thereby realizing risk quantification.

[0028] Specifically, the inspection target refers to the core business orientation of this inspection task, which can be "safety inspection of river embankments during the flood season," "safety assurance of the urban core area during major events," or "daily infrastructure health monitoring." The allocation of inspection weights refers to determining the weights of areas corresponding to the inspection target within the multi-dimensional risk feature layer based on the inspection target. For example, if the target is "fire prevention," the weights of gas stations and old wooden structures are increased; if the target is "flood prevention," the weights of rivers and low-lying areas are increased. This can be achieved using the Analytic Hierarchy Process (AHP). The AHP (Advanced Risk Process) assigns inspection weights by comparing the importance of different factors pairwise (e.g., is power safety more important than road damage), constructing a discrimination matrix and calculating inspection weights (e.g., power 0.5, road 0.2, etc.). The grid weighted overlay algorithm refers to linearly weighting and summing the risk values ​​corresponding to all multi-dimensional risk feature layers pixel by pixel; the dynamic change features refer to the risk features corresponding to real-time dynamic data, and the spatiotemporal matching algorithm refers to assigning spatial coordinates and timestamps to the risk features of real-time dynamic data; for example, a "congestion alarm" from the traffic department includes the intersection name, which is geocoded. Convert to latitude and longitude coordinates; a weather station's "strong wind warning" includes a station ID, which is located by looking up its preset coordinates; on the initial comprehensive risk grid map, find the pixel where the coordinate is located, and confirm whether the event occurred within the time window of this inspection, and use Kalman filtering or Bayesian update methods to perform dynamic risk fusion. Normalization refers to scaling the calculated risk value to a standard space of 0 to 1, and color mapping refers to mapping the risk value in the standard space to the corresponding warm and cool color table. Values ​​close to 1 are mapped to dark red (high-risk area), and values ​​close to 0 are mapped to dark blue (safe area), finally forming a dynamic risk heat map.

[0029] In detail, the process of transforming the dynamic risk heatmap into a risk space potential field based on the heterogeneous task density function includes: Based on the inspection target, an inspection target object set is extracted from the standard data layer set, and a heterogeneous task density function is constructed according to the target object type of the inspection target object set; Configure a task density parameter table for the heterogeneous task density function, wherein the task density parameter table includes the influence radius, spatial decay factor and basic weights; Based on the heterogeneous task density function and the corresponding task density parameter table, the potential energy contribution value of each inspection target object in the inspection target object set is calculated, and all potential energy contribution values ​​are spatially superimposed to obtain the discrete task potential energy field. The normalized risk value of each pixel is extracted from the dynamic risk heat map to obtain the risk value field. The discrete task potential field and the risk value field are then fused to obtain the original fused potential field. The original fused potential field is smoothed by Gaussian convolution and normalized globally to obtain the risk space potential field.

[0030] In this context, each inspection target in the inspection target set is a region object that needs to be inspected, corresponding to the inspection target, and includes the corresponding spatial pixel coordinates. The target object types include point targets, line targets, and area targets. Point targets, such as power towers and fire hydrants, can have their coordinates and semantics directly obtained from point elements in a GIS vector layer, in the format "Transmission Tower," "Voltage Level: 500kV." Line targets, such as highways and rivers, are in the format "Expressway," "Number of Lanes: 6." Area targets, such as farmland and residential areas, are in the format "Basic Farmland," "Crop: Rice." The heterogeneous task density function is a function used to describe the distribution of the "attraction" or "potential energy" generated by a single inspection target on its surrounding space. For example, for inspection target objects of the point target type, the corresponding heterogeneous task density function is: ,in, Let the coordinates of any point in the urban or rural area be the spatial pixel coordinates. It means Targeting and the first The potential energy contribution value of each inspection target object. It refers to the first The basic weight of each inspection target object It is the first The spatial pixel coordinates of the inspection target object yes and Spatial Euclidean distance, It is the first The spatial attenuation factor of each inspection target object , It is the first The influence radius of each inspection target object; the basic weight is allocated according to the semantics of each inspection target object; the normalized risk value refers to the risk value after color mapping planning; the fusion calculation is weighted superposition fusion or product coupling fusion; wherein, the risk space potential energy field is to transform the dynamic risk heat map into a continuous, calculable, and decision-guided spatial scalar field; the UAV is regarded as a point mass moving in the risk space potential energy field; the high potential energy region exerts a gravitational force on the UAV, guiding the swarm to spontaneously converge towards the risk concentration area; through this transformation, the original risk color becomes the physical force that drives the UAV to automatically plan its trajectory.

[0031] Specifically, the process of adaptive grid clustering partitioning of the risk space potential energy field includes: The potential energy gradient field is obtained by calculating the spatial gradient of the risk space potential energy field. Based on the potential energy gradient field, local maxima and high gradient boundary regions are extracted, and a candidate anchor point set is constructed based on the local maxima and high gradient boundary regions. Based on preset potential energy similarity thresholds and spatial distance thresholds, the candidate anchor point set is merged to obtain an initial clustering seed point set; Based on gradient constraints, region growing clustering is performed on each initial clustering seed point in the initial clustering seed point set to obtain an initial cluster set. The initial cluster set is subjected to cluster merging, cluster segmentation, and geometric optimization to obtain a candidate task package set; Add task attributes to each candidate task package in the candidate task package set, and encapsulate them into a standard task package set.

[0032] Specifically, the Sobel operator can be used to perform finite difference calculations of the spatial gradient. Local maxima refer to the maximum values ​​within a local neighborhood, used to determine the core inspection area. High gradient boundaries refer to areas with the most drastic changes in potential energy; local maxima can be extracted using non-maximum suppression or sliding window extremum detection methods, and high gradient regions can be extracted through threshold segmentation. The construction of the candidate anchor point set is a set of candidate anchor points centered on the high gradient boundary regions and the local extrema. Anchor point merging refers to merging anchor points whose spatial distance is less than the spatial distance threshold and whose potential energy... Two candidate anchor points whose absolute difference in potential energy value is less than the potential energy similarity are merged, for example, by taking the coordinates of the one with higher potential energy or by taking a weighted average, to obtain the initial clustering seed point. Region growing clustering refers to using each initial clustering seed point as the starting point and executing a region growing algorithm to include spatially adjacent neighboring pixels that satisfy potential energy continuity and gradient direction consistency into the initial cluster. Here, potential energy continuity means that the absolute value of the difference between the potential energy value of the neighboring pixel and the potential energy value of the initial clustering seed point is less than the potential energy similarity; the gradient direction consistency means that the neighboring pixel and the current growing point are similar. The gradient direction angle of the boundary pixels is less than the preset gradient angle threshold; cluster merging refers to assigning initial clusters with too small an area or too low total potential energy to the nearest neighboring initial clusters; cluster segmentation refers to segmenting initial clusters with extremely irregular shapes (e.g., aspect ratio greater than the preset aspect ratio threshold) or containing potential energy holes according to potential energy contour lines; geometric optimization refers to smoothing the boundaries of each cluster (e.g., using morphological closing operations), filling internal holes, and calculating the minimum circumcircle convex hull or circumcircle rectangle to ensure spatial continuity and shape regularity; adding task attributes refers to adding task attributes to each candidate task... The task package extracts the corresponding spatial description, performs potential energy statistics, extracts geometric attributes, and extracts task metadata, which is then encapsulated as task attributes. The spatial description refers to the polygon boundary of the corresponding cluster's geographic region. The potential energy statistics refer to the extraction of data such as average potential energy, maximum potential energy, potential energy centroid coordinates, and total potential energy. The geometric attributes include data such as area, perimeter, and centroid coordinates. The task metadata includes data such as target inspection type labels and suggested sensor types. Each standard task package in the standard task package set is a structured scheduling unit obtained after dividing the risk spatial potential energy field.

[0033] By integrating multi-source heterogeneous data and constructing a dynamic risk heatmap, a unified quantitative expression of risk distribution in urban and rural areas is achieved. By introducing a heterogeneous task density function, discrete risk information is transformed into a continuous risk spatial potential energy field, enabling inspection tasks to have a physically meaningful guidance mechanism. Key structural features are extracted through potential energy gradient analysis, and combined with a gradient-constrained region growth clustering method, adaptive partitioning of complex spaces is achieved, avoiding the task fragmentation and resource waste problems caused by traditional fixed grid partitioning. By generating a standard task package set, the spatial range, risk intensity, and task attributes are structurally encapsulated, providing a unified and computable task unit for subsequent multi-UAV collaborative scheduling and path planning.

[0034] S2. Based on the performance parameters of the UAV swarm, a UAV performance feature set is constructed. Using the risk space potential energy field as a global scheduling constraint, the standard task package set is assigned tasks according to the UAV performance feature set based on the improved contract network algorithm, and the task assignment result is obtained.

[0035] The performance parameters refer to the parameters related to the working performance of each drone in the drone swarm. These parameters can be entered into the configuration file according to the factory specifications, and the real-time performance status parameters of the drones can be obtained based on real-time communication. The performance characteristics of each drone in the drone performance characteristic set correspond to the performance parameters of the individual drones in the drone swarm, including flight performance characteristics, payload capacity characteristics, perception capability characteristics, communication capability characteristics, and energy consumption and status characteristics. Flight performance characteristics include maximum endurance, maximum range, maximum speed, and wind resistance level; payload capacity characteristics include maximum payload, types of sensors that can be carried (including visible light, infrared, or multispectral), and the upper limit of the number of sensors; perception capability characteristics include camera resolution, night vision capability, infrared capability, and edge computing power; communication capability characteristics include maximum communication distance, whether relay is supported, and bandwidth transmission capability; and energy consumption and status characteristics include current remaining battery power, current task load, and current location information.

[0036] For details, please refer to Figure 2 The process of allocating tasks to the standard task package set based on the improved contract network algorithm according to the UAV performance feature set includes: S21. Using the gradient direction of the potential energy field in the risk space as the inspection guidance direction, select the standard task packages in the standard task package set as target task packages one by one in order of potential energy from large to small. S22. Perform task requirement analysis on the target task package to obtain task requirement characteristics, wherein the task requirement characteristics include space requirement characteristics, performance requirement characteristics and risk requirement characteristics. S23. Generate a dynamic task announcement for the target task package based on the task requirement features, and aggregate the task requirement features of all dynamic task announcements into a task requirement feature set; S24. Based on the UAV performance feature set and mission requirement feature set, establish the matching relationship between the UAV and the standard mission package one by one, calculate the corresponding estimated matching cost, and construct the estimated matching cost matrix, wherein the estimated matching cost includes path cost, capability adaptation cost and energy consumption cost. S25. Based on the estimated matching cost matrix, bid decisions, conflict resolution, and consistency verification are performed on each dynamic task announcement to obtain the task allocation result.

[0037] The task requirement analysis refers to using the lead drone in the drone swarm to analyze the task attributes and spatial boundaries of each standard task package. The task requirement feature set includes spatial requirement features, performance requirement features, and risk requirement features. The spatial requirement features correspond to the boundary vertex sequence of the task package; the performance requirement features are the suggested sensor type, minimum image resolution, and maximum allowable task completion time for the corresponding inspection target type; and the risk requirement features include average potential energy, maximum potential energy, and total potential energy, used to measure the urgency and importance of the task. The estimated matching cost is calculated by each drone upon receiving the corresponding dynamic task announcement, combining its own drone performance characteristics with the task requirement features in the dynamic task announcement, as shown in the following formula: in, It refers to the first The performance characteristics of the first drone and the first The estimated matching cost corresponding to each task requirement feature , , These are weighting coefficients. yes The corresponding path potential energy integral cost, yes The corresponding capability adaptation cost, yes The corresponding cost to the feasibility of extended battery life; in, It refers to the first The potential energy centroid coordinates corresponding to the characteristics of each task requirement It means The current position coordinates corresponding to each drone's performance characteristics It refers to from arrive The potential energy value at any point on the shortest integration path. This refers to the infinitesimal length of the path. It refers to the first Individual drone performance characteristics It refers to the first Individual task requirements characteristics It refers to the first The estimated total energy consumption for each task requirement, including energy consumption for arrival, operation, return, or flight to the next node, can be estimated based on path length, hovering time, and UAV power consumption models. It refers to the first The current remaining battery power in the performance characteristics of each UAV, the generation of the dynamic task announcement of the target task package refers to the lead UAV extracting the task ID, spatial boundary, potential energy centroid coordinates, and task requirement characteristics of the target task package, and generating a dynamic task announcement in combination with the bidding deadline, and broadcasting the dynamic task announcement to each UAV.

[0038] Specifically, the capability adaptation cost The calculations are performed by constructing the performance characteristics of the drone. Feature vectors related to task requirements And calculate its mismatch degree to achieve the performance characteristics of the UAV; With demand feature vector Generates based on preset dimensional rules, and its construction and matching rules are as follows: Sensor type matching dimension: For each sensor (e.g., visible light, infrared, multispectral) in the "Suggested Sensor Types" list of task requirements features, it occupies an independent dimension in the vector. If the drone is equipped with that type of sensor, then it will occupy a different dimension in the vector. The corresponding dimension value is 1 if it is not 1, otherwise it is 0. In this context, the dimension corresponding to the sensor required for the task has a value of 1, otherwise it has a value of 0. The matching contribution value of this dimension is the minimum value (logical AND) of the two vectors in this dimension. Performance metric matching dimensions: A matching degree function is defined for numerical performance requirements such as "minimum image resolution" and "maximum allowed task completion time." For example, regarding resolution requirements, the drone resolution... Resolution required by the task The matching degree can be defined as min(1, A perfect match (value of 1) is defined as meeting or exceeding the requirements; otherwise, the match is reduced proportionally. This value is used as... and The value in the corresponding dimension; for the maximum task completion time, the ratio of the estimated operational time of the UAV to the required task time can be calculated similarly; Basic capability matching dimensions: For graded or Boolean features such as "wind resistance level" and "night vision capability", they can be converted into standard levels (e.g., 0-1). The drone and mission requirements respectively take the corresponding level values ​​as vector dimension values. Vector generation and cost calculation: Based on the above rules, numerical values ​​are generated. and When the drone fully meets all mission capability requirements, the two vectors are aligned. A value close to 0 indicates an extremely low adaptation cost; when one or more capabilities are missing or mismatched, the vector direction deviates. Increasing the value results in an adaptation cost penalty; for example, consider a task... If the requirement vector is encoded as requiring a [visible light, infrared] sensor with a minimum resolution of 1080p, then its vector can be represented as follows: =[1,1,1.0] (assuming a perfect resolution of 1.0). If the drone... Equipped only with a visible light sensor and a resolution of 720p, its capability vector can be expressed as follows: =[1,0,0.67] (720p / 1080p≈0.67), calculating the cosine similarity will be significantly less than 1, thus producing a higher similarity. The cost reflects its inadequacy.

[0039] Specifically, the process of making bidding decisions, resolving conflicts, and verifying consistency for each dynamic task announcement based on the estimated matching cost matrix includes: Each dynamic task announcement is selected as the target task announcement, and the estimated matching cost of each UAV in the UAV swarm for the target task announcement is extracted from the estimated matching cost matrix to obtain the estimated cost set. Extract the state factors of each UAV and calculate the bidding cost set by combining them with the estimated cost set; The drone with the smallest bid cost in the set of bid costs is selected as the winning drone in the target mission announcement, and a primary allocation list is generated based on each target mission announcement and the corresponding winning drone. Spatiotemporal conflict detection is performed on each winning UAV in the primary allocation list according to a preset sliding spatiotemporal window to obtain conflict task packages; Conflict resolution is performed based on the bid cost set of the conflict task package, and the primary allocation list is updated based on the result of conflict resolution to obtain the task allocation list. A global consistency check based on risk priority is performed on the task allocation list to obtain the task allocation result.

[0040] Spatiotemporal conflict detection refers to detecting overlaps in the execution time intervals and spatial regions of UAV tasks in the initial allocation list according to a preset sliding spatiotemporal window, identifying task packages with spatiotemporal resource conflicts as conflicting task packages. Conflict resolution refers to detecting conflicts in the bidding decision (e.g., the same UAV being initially allocated to multiple tasks with overlapping times). For standard conflicting task packages, a fast local auction is initiated among all the corresponding winning UAVs: each winning UAV submits a revised bid price for the conflicting task (the original price plus the time conflict penalty), iterating this process until a conflict-free allocation scheme is obtained. Global consistency verification based on risk priority refers to verifying whether the allocation scheme after conflict resolution satisfies the global constraints of the risk space potential energy field. For example, the allocation result after conflict resolution is mapped and verified with the risk space potential energy field to assess whether the core task package with the highest total potential energy is allocated to the UAV with the strongest perception and payload capabilities. If there are high-priority tasks that are not matched with high-capability UAVs, a forced re-labeling optimization is performed across task packages to finally obtain a stable task allocation result. Through the above distributed task allocation, the computational load of each UAV can be reduced, and the efficiency of task allocation can be improved.

[0041] Specifically, the iterative process of conflict resolution follows the following executable rules: Iteration termination condition: Set the maximum number of iteration rounds. When the number of iterations reaches The iteration terminates if no new spatiotemporal conflict is detected in the current iteration; the penalty mechanism for revising the bidding cost increment is as follows: in each round of local auction, the drone is the conflict mission. Submitted revised bid price The calculation formula is: ,in, For the drone to perform the mission The original bid price, This is the current iteration number (counting from 1). As a basic penalty item based on the severity of the conflict, This can be defined as the estimated temporal overlap or spatial intrusion depth caused by the conflict. This design ensures that the penalty cost for the same conflicting task after each failed bid is borne by the task. The increasing number of iterations forces each drone to more actively adjust its plans in subsequent bidding processes to resolve conflicts. A downgraded handling strategy is employed when conflicts cannot be completely resolved: if the maximum number of iterations is reached... If residual conflicts still exist, a downgrade strategy will be initiated. The system will calculate the priority of each conflicting task package based on the risk space potential energy field, forcibly release the current allocation of the lowest priority task, and mark it as "pending reallocation" to ensure conflict-free allocation of high-priority tasks. The reassigned task will re-participate in the bidding in subsequent scheduling cycles.

[0042] By structurally modeling the multidimensional performance characteristics of UAVs and mission requirements, a unified predicted matching cost function is constructed, and a risk space potential energy field is introduced to modulate the path cost. This transforms task allocation from being driven by a single distance or energy consumption to a multi-factor coupled decision-making process. Through joint modeling of path potential energy integral, capability adaptability, and endurance constraints, UAVs not only consider physical reachability when performing tasks but also comprehensively assess mission risk and execution matching. Furthermore, by combining an improved contract network mechanism, dynamic task announcements, distributed bidding, and conflict resolution enable adaptive collaborative scheduling of multiple UAVs in complex urban and rural environments. This effectively improves coverage priority and overall resource utilization efficiency in high-risk areas and enhances the system's robustness and real-time response capabilities in dynamic environments.

[0043] S3. Using the risk space potential energy field as the objective function, perform multi-dimensional collaborative constraints and collaborative path planning on the task allocation results to obtain a collaborative inspection track set, wherein the multi-dimensional collaborative constraints include UAV collision avoidance constraints, spatiotemporal occupancy constraints and communication link constraints.

[0044] For details, please refer to Figure 3 The process of applying multi-dimensional collaborative constraints and collaborative path planning to the task allocation results includes: S31. Based on the task allocation results, extract the spatial boundary and potential energy centroid coordinates of the target task package corresponding to each matched UAV, and construct the initial guiding vector field by combining the gradient distribution of the risk space potential energy field. S32. Based on the initial guidance vector field, calculate the initial reference trajectory of each matched UAV, and calculate the relative displacement and expected arrival time between each matched UAV according to the initial reference trajectory, and construct a spatiotemporal occupancy prediction model. S33. Based on the aforementioned spatiotemporal occupancy prediction model, introduce UAV collision avoidance constraints and spatiotemporal occupancy constraints to construct a cooperative obstacle avoidance potential field. S34. Introduce communication link constraints into the cooperative obstacle avoidance potential field and construct a safe and feasible domain; S35. Based on the rolling time-domain optimization algorithm, the initial reference trajectories of each matched UAV are dynamically optimized within the safe and feasible domain to obtain a collaborative inspection trajectory set.

[0045] The initial guidance vector field is a vector field defined in the task space of urban and rural areas. It is used to provide each matched UAV with a recommended flight direction pointing to the corresponding target task package and also for global risk detection. The gradient of the potential energy field in the risk space can be used as a directional guide to construct the initial guidance vector field. A smooth curve leading to the task area can be generated by integrating along the direction field of the initial guidance vector field from the current position of the UAV using streamline tracing or gradient ascent methods, thus obtaining the initial reference track. The calculation of relative displacement refers to calculating the position difference of each matched UAV at future time stamps based on the initial reference track, thereby determining the collision risk. The calculation of the estimated arrival time is based on the length of the initial reference track and the performance parameters of the corresponding matched UAV to construct a motion model, thereby estimating the time it takes for the matched UAV to arrive at the corresponding task area, and thus determining the spatiotemporal occupancy conflict. The spatiotemporal occupancy prediction model is based on the initial reference track and the estimated arrival time, and predicts the change of the position of each UAV in space over a period of time in the future, as well as the change of the number of UAVs in the task area over time. The spatiotemporal occupancy prediction model can be a four-dimensional occupancy grid, used to identify when and where UAVs may be present, as well as the UAV density in a specific area. This is the basis for detecting collision avoidance and regional congestion conflicts.

[0046] Specifically, the UAV collision avoidance constraint refers to constraining any two UAVs to maintain a minimum safe distance at any time. Introducing the UAV collision avoidance constraint means that when it is detected that the relative displacement of two UAVs may be less than the minimum safe distance at some future time, a repulsive potential energy term is generated on the line connecting the two UAVs, thereby forcing the optimization algorithm to adjust its trajectory. The spatiotemporal occupancy constraint refers to limiting the number of UAVs operating in the same task area or airspace channel at the same time to prevent efficiency loss, signal interference, or safety risks caused by excessive concentration. Introducing the spatiotemporal occupancy constraint means defining a virtual capacity for each task area that needs to be managed. When the spatiotemporal occupancy prediction model shows that the number of UAVs in the task area will exceed the capacity limit at some future time, a repulsive potential energy term is generated for that area. This repulsive potential energy term increases with the increase of the number of UAVs exceeding the limit, causing subsequent UAVs to tend to adjust their arrival time or detour. The cooperative obstacle avoidance potential field is a resultant potential field formed by the superposition of the initial guidance vector field and each repulsive potential energy term.

[0047] In detail, the process of introducing communication link constraints and constructing a safe and feasible region in the cooperative obstacle avoidance potential field includes: A signal gain distribution map of the urban and rural areas is constructed, and spatial interpolation is performed on the signal gain distribution map to obtain a continuous signal gain field; A communication link constraint is introduced, and based on the signal gain field, regions where the signal strength is lower than a preset strength threshold are identified as regions where the communication constraint is violated. The communication constraint violation region is transformed into a repulsive potential energy term and fused with the cooperative obstacle avoidance potential field to obtain a comprehensive spatial potential field. Based on the comprehensive spatial potential field, a set of regions that satisfy the UAV collision avoidance constraints, spatiotemporal occupancy constraints, and communication link constraints and also satisfy spatial continuity are extracted. Then, the spatiotemporal occupancy prediction model is used to filter out regions that satisfy time accessibility from the set of regions to obtain candidate safe regions. Connectivity analysis and reachability verification are performed on the candidate safe regions. Based on the analysis and verification results, isolated regions in the candidate safe regions are eliminated to obtain a safe and feasible region that satisfies collision avoidance constraints, spatiotemporal occupancy constraints, and communication link constraints.

[0048] Communication link constraints refer to the constraints that ensure each UAV maintains a reliable connection with the communication network (ground station, relay UAV) during flight, i.e., the received signal strength is not lower than the signal strength threshold. The signal gain distribution map is a distribution model constructed based on the location coordinates of the signal transmitter and the signal gain value in urban and rural areas. The process of dynamically optimizing the initial reference trajectory of each matched UAV within the safe and feasible region based on the rolling time-domain optimization algorithm includes: setting a prediction window of several future time steps based on the performance characteristics of each matched UAV, and using the initial reference trajectory as the initial value of the optimization sequence; extracting the potential energy gradient in the comprehensive spatial potential field as the guiding term, and using the path length and energy consumption index as smoothing terms to construct a multi-index weighted cost function; in each decision cycle, using a nonlinear programming algorithm to search for a local trajectory sequence that satisfies the minimization of the cost function within the safe and feasible region, and sliding the prediction window backward according to the time sequence to obtain the collaborative inspection trajectory.

[0049] By utilizing the potential energy field of the risk space to generate an initial guidance vector field, the system enables adaptive guidance of UAVs to high-value inspection areas. By introducing collision avoidance constraints and regional capacity constraints based on spatiotemporal occupancy prediction, spatial conflicts and task congestion among multiple UAVs are effectively avoided. By combining communication link constraints, communication blind spots are transformed into high-cost areas, enabling proactive avoidance of communication reliability during path planning. By employing a rolling time-domain optimization algorithm to perform dynamic trajectory optimization within the safe and feasible domain, the system possesses real-time response capabilities to environmental changes. This achieves a unified improvement in the safety, coordination, and task coverage efficiency of multiple UAVs in complex urban and rural environments, significantly enhancing the intelligence and robustness of the overall inspection system.

[0050] S4. Control the UAV swarm to execute the standard task package set according to the collaborative inspection track set, and collect multi-source data on the urban and rural areas to obtain inspection image data and environmental status data.

[0051] Executing the standard task package set refers to calling the corresponding matching UAV for each standard task package in the set, controlling the matching UAV to fly to the task area of ​​the standard task package according to the corresponding collaborative inspection flight path, extracting task attributes from the standard task package, and performing redundant scanning data acquisition on the task area based on the task attributes. The redundant scanning data acquisition refers to acquiring data on the same target area through different perspectives, different sensors, or multiple repeated scans to improve data integrity and reliability. For example, using a visible light camera for orthophoto shooting, activating an infrared thermal imager for scanning, and using environmental sensors to collect atmospheric quality parameters, and then associating the scan data with the acquisition timestamp and acquisition coordinates. The system stores and obtains inspection image data and environmental status data. Inspection image data refers to visual data acquired by imaging sensors mounted on the UAV, which reflects the appearance and structural features of the target area. This includes visible light images or videos, infrared thermal images, multispectral or hyperspectral images, and depth maps or point clouds generated by lidar, all labeled with acquisition timestamps and coordinates. Environmental status data refers to numerical or time-series data collected by non-imaging sensors or system status modules mounted on the UAV, which reflects the environmental conditions of the inspection area and the UAV's operating status. This includes data such as ambient light intensity, temperature, humidity, wind speed, wind direction, air pressure, and concentration of harmful gases, all labeled with acquisition timestamps and coordinates.

[0052] S5. Perform fusion analysis and event recognition on the inspection image data and environmental status data to obtain the inspection event recognition result.

[0053] In detail, the process of fusing and analyzing the inspection image data and environmental status data, as well as identifying events, includes: Spatiotemporal alignment is performed based on the collection timestamps and collection coordinates marked in the inspection image data and environmental status data to obtain aligned multimodal data; By combining the environmental state data in the aligned multimodal data, the inspection image data is preprocessed and environmental feature compensation is performed. Then, the compensated inspection image data is extracted to obtain a multidimensional enhanced feature set. By combining the prior knowledge provided by the standard task package set, target detection and semantic segmentation are performed on the multidimensional enhanced feature set to obtain the semantic set of the inspection object; The semantic set of the inspection objects is semantically matched with the preset historical risk semantic database, and abnormal deviations are extracted from the semantic matching results based on the spatiotemporal correlation analysis method to obtain potential inspection events. Based on the multi-criteria decision analysis method, the potential inspection events are subjected to logical consistency verification and confidence verification to obtain the inspection event identification results.

[0054] In this process, the inspection image data and environmental status data are uploaded to the cloud for fusion analysis and event recognition. The cloud-based high-performance computing then performs the processing. Image preprocessing involves automatically adjusting image exposure and contrast based on ambient light intensity from the environmental status data, compensating for motion blur using wind speed and direction data, and correcting thermal radiation attenuation in infrared thermal imaging using temperature and humidity data through an atmospheric transmission model. Feature extraction involves extracting low-level features such as color, texture, shape, and spectrum from the compensated inspection image data, combining these features with environmental feature compensation to extract environmental features from the corresponding environmental status data, and fusing them into a multi-dimensional enhanced feature set. Prior knowledge includes the task area type and coordinates of each standard task package in the standard task package set. This prior knowledge can be used to configure Regions of Interest (ROIs) in the multi-dimensional enhanced feature set. The system uses a list of interest regions and expected categories, combined with models such as YOLO, to achieve target detection, and uses models such as U-Net for semantic segmentation. The historical risk semantic database is a structured knowledge base that stores records of previously confirmed risks or abnormal events. Each record includes not only the event type (such as "fire", "illegal dumping", "equipment damage"), but also its spatiotemporal context (location, time), environmental conditions (weather, season), and visual or physical feature descriptions. Semantic matching can be performed by calculating feature similarity or spatiotemporal proximity. The spatiotemporal correlation analysis refers to constructing a spatiotemporal baseline model of normal behavior, and performing anomaly analysis based on a knowledge graph or rule base of spatiotemporal correlation to discover patterns that violate normal spatiotemporal rules. For example, it identifies spatial anomalies of construction machinery in areas where construction is prohibited, detects temporal anomalies of large vehicle activity during non-working hours at night, and identifies spatiotemporal correlation anomalies where humidity should rise in an area after a rainstorm (corresponding to a short circuit or infiltration event), but infrared images show abnormally high temperature points. Taking the potential event of "large vehicles appearing in restricted areas at night" as an example, this paper illustrates the semantic matching, anomaly extraction, and multi-criteria decision-making process: Semantic matching and feature comparison refers to retrieving event templates related to "vehicle violations" from the "historical risk semantic database". Its main features include: visual semantics ("truck", "construction vehicle"), time conditions ("night"), spatial rules ("prohibited areas"), and environmental conditions ("no special requirements"). The semantics of the currently identified inspection objects (e.g., target category "dump truck", time "02:30", location "core park green space") are compared with the template. The matching degree is calculated in three dimensions: visual category, time window, and spatial control rules. The "abnormal deviation" comes from the joint violation of the spatiotemporal baseline model and the prior knowledge of the task. The normal situation of this area in the same historical time period (late at night) is "no large vehicles". The current presence of vehicles constitutes a significant deviation from the time behavior pattern. The "target inspection type label" of this area in the standard task package is "green space protection", which implies that the spatial rules exclude the entry of large engineering vehicles. The current semantics violate this rule. When the matching degree calculation shows that the visual, temporal, and spatial features are highly consistent with the historical risk template and significantly deviate from the baseline model and prior rules, a clear abnormal deviation is extracted, and a potential event of "nighttime vehicle violation" is generated.

[0055] The logical consistency check item verifies whether the semantics of "existence of large vehicles" are logically consistent with other modal data. For example, it checks whether the sound sensor data at this time and location shows an abnormal increase in engine noise, or whether the vibration sensor shows abnormal readings. If there is supporting evidence, the logical consistency is high; if all environmental sensors show no abnormalities, the confidence level needs to be lowered. The confidence level check item comes from the following sources: the original confidence level of target detection (e.g., the confidence level of YOLO or other models in identifying "dump trucks") (e.g., 0.92); the statistical significance of anomalies (calculating the probability of such vehicles appearing in the area at the current time, relative to the Z-score of the historical baseline model); and the historical prior probability (querying the frequency of similar events occurring in the area from the historical risk semantic database). By weightedly fusing the above three items (e.g., weighted average), a comprehensive confidence score for the potential event is obtained. If the score exceeds the preset threshold and the logical consistency check passes, the event is confirmed as a formal inspection event identification result. Multi-Criteria Decision Analysis (MCDA) is a collection of methods based on different principles and ideas to assist in the selection of solutions. Logical consistency check refers to checking whether the logic of a potential inspection event is contradictory. For example, if the visual recognition identifies a potential inspection event as "fire", but the environmental sensor shows that the temperature is extremely low and there is no increase in smoke concentration, then the logical consistency is low and it is judged as a false alarm of light and shadow. Confidence check refers to obtaining the final confidence score by weighting the original confidence of target detection, the statistical significance of abnormal deviation (e.g., Z-score), and the prior probability of the potential inspection event in the historical risk semantic database, and then combining it with the confidence threshold for confidence check.

[0056] By deeply fusing and analyzing inspection image data and environmental status data, a shift from single visual perception to multimodal intelligent cognition was achieved. An environmental perception compensation mechanism was introduced to effectively eliminate the impact of factors such as lighting changes and weather disturbances on image quality, improving target recognition accuracy in complex environments. Prior knowledge from standard task packages was combined to constrain the detection process, enabling precise task-oriented recognition. Furthermore, a historical risk semantic database and spatiotemporal correlation analysis were used to perform semantic-level anomaly detection on the recognition results, giving the system the ability to understand risk evolution and anomaly patterns. Finally, a multi-criteria decision-making method was used for consistency and confidence verification, significantly reducing the false alarm rate.

[0057] Example 2: Please refer to Figure 4 This invention discloses an intelligent inspection system for urban and rural areas based on multi-UAV collaboration. The system includes a grid division module, a task allocation module, a path planning module, an inspection and data collection module, and a fusion analysis module. The grid partitioning module is used to perform spatial overlay analysis on multi-source heterogeneous data in urban and rural areas to obtain a dynamic risk heat map. Based on the heterogeneous task density function, the dynamic risk heat map is transformed into a risk spatial potential energy field, and the risk spatial potential energy field is subjected to adaptive grid clustering partitioning to obtain a standard task package set. The task allocation module is used to construct a UAV performance feature set based on the performance parameters of the UAV swarm, use the risk space potential energy field as a global scheduling constraint, and allocate tasks to the standard task package set based on the UAV performance feature set using the improved contract net algorithm to obtain the task allocation result. The path planning module is used to perform multi-dimensional collaborative constraints and collaborative path planning on the task allocation results with the risk space potential energy field as the objective function, so as to obtain a collaborative inspection track set. The multi-dimensional collaborative constraints include UAV collision avoidance constraints, spatiotemporal occupancy constraints and communication link constraints. The inspection and data acquisition module is used to control the UAV swarm to execute the standard task package set according to the collaborative inspection track set, and to collect multi-source data on the urban and rural areas to obtain inspection image data and environmental status data. The fusion analysis module is used to perform fusion analysis and event recognition on the inspection image data and environmental status data to obtain the inspection event recognition results.

[0058] The processes described above with reference to the flowcharts in the embodiments disclosed in this invention can be implemented as computer software programs. The embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), it performs the functions defined in the methods of this application. It should be noted that the computer-readable medium described above in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wire segments, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless segments, wire segments, optical fibers, RF, etc., or any suitable combination thereof.

[0059] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0060] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are merely examples and do not limit the present invention. The purpose of the present invention has been fully and effectively achieved. The functions and structural principles of the present invention have been shown and explained in the embodiments. Without departing from the stated principles, the implementation of the present invention may have any variations or modifications.

Claims

1. A method for intelligent inspection of urban and rural areas based on multi-UAV collaboration, characterized in that, The method includes: Spatial overlay analysis of multi-source heterogeneous data in urban and rural areas is performed to obtain a dynamic risk heat map. Based on the heterogeneous task density function, the dynamic risk heat map is transformed into a risk spatial potential energy field. The risk spatial potential energy field is then subjected to adaptive grid clustering to obtain a standard task package set. A set of UAV performance characteristics is constructed based on the performance parameters of the UAV swarm. The risk space potential energy field is used as a global scheduling constraint. Based on the improved contract net algorithm, the standard task package set is assigned tasks according to the set of UAV performance characteristics to obtain the task assignment results. Using the risk space potential energy field as the objective function, multi-dimensional collaborative constraints and collaborative path planning are applied to the task allocation results to obtain a collaborative inspection track set. The multi-dimensional collaborative constraints include UAV collision avoidance constraints, spatiotemporal occupancy constraints, and communication link constraints. The drone swarm is controlled to execute the standard task package set according to the collaborative inspection track set, and to collect multi-source data on the urban and rural areas to obtain inspection image data and environmental status data. The inspection image data and environmental status data are fused and analyzed, and event recognition is performed to obtain the inspection event recognition results.

2. The intelligent inspection method for urban and rural areas based on multi-UAV collaboration according to claim 1, characterized in that, The process of spatial overlay analysis of multi-source heterogeneous data in urban and rural areas includes: Acquire multi-source heterogeneous data from urban and rural areas, and perform coordinate system unification, spatial resolution alignment, and format standardization on the multi-source heterogeneous data to obtain a standard data layer set; Spatiotemporal correlation analysis is performed on the standard data layer set to identify spatial co-occurrence and temporal evolution patterns among different standard data layers, and the analysis results are obtained. Based on the analysis results, the risk of the standard data layer set is quantified to obtain a multidimensional risk feature layer set. Based on the preset inspection targets, inspection weights are assigned to the multidimensional risk feature layer set, and the multidimensional risk feature layer set is fused based on the grid weighted overlay algorithm to obtain an initial risk value grid map. Dynamic change features are extracted in real time from the standard data layer set, and the dynamic change features are mapped to the initial risk value raster map based on the spatiotemporal matching algorithm to perform dynamic risk fusion and obtain a dynamic risk value distribution map. The risk values ​​in the dynamic risk value distribution map are normalized and color-mapped to obtain a dynamic risk heat map.

3. The intelligent inspection method for urban and rural areas based on multi-UAV collaboration according to claim 2, characterized in that, The process of transforming the dynamic risk heatmap into a risk space potential field based on the heterogeneous task density function includes: Based on the inspection target, an inspection target object set is extracted from the standard data layer set, and a heterogeneous task density function is constructed according to the target object type of the inspection target object set; Configure a task density parameter table for the heterogeneous task density function, wherein the task density parameter table includes the influence radius, spatial decay factor and basic weights; Based on the heterogeneous task density function and the corresponding task density parameter table, the potential energy contribution value of each inspection target object in the inspection target object set is calculated, and all potential energy contribution values ​​are spatially superimposed to obtain the discrete task potential energy field. The normalized risk value of each pixel is extracted from the dynamic risk heat map to obtain the risk value field. The discrete task potential field and the risk value field are then fused to obtain the original fused potential field. The original fused potential field is smoothed by Gaussian convolution and normalized globally to obtain the risk space potential field.

4. The intelligent inspection method for urban and rural areas based on multi-UAV collaboration according to claim 1, characterized in that, The process of adaptive grid clustering partitioning of the risk space potential energy field includes: The potential energy gradient field is obtained by calculating the spatial gradient of the risk space potential energy field. Based on the potential energy gradient field, local maxima and high gradient boundary regions are extracted, and a candidate anchor point set is constructed based on the local maxima and high gradient boundary regions. Based on preset potential energy similarity thresholds and spatial distance thresholds, the candidate anchor point set is merged to obtain an initial clustering seed point set; Based on gradient constraints, region growing clustering is performed on each initial clustering seed point in the initial clustering seed point set to obtain an initial cluster set. The initial cluster set is subjected to cluster merging, cluster segmentation, and geometric optimization to obtain a candidate task package set; Add task attributes to each candidate task package in the candidate task package set, and encapsulate them into a standard task package set.

5. The intelligent inspection method for urban and rural areas based on multi-UAV collaboration according to claim 1, characterized in that, The process of allocating tasks to the standard task package set based on the UAV performance feature set using the improved contract network algorithm includes: Using the gradient direction of the potential energy field in the risk space as the inspection guidance direction, the standard task packages in the standard task package set are selected one by one as target task packages in descending order of potential energy. The target task package is parsed to obtain task requirement features, which include space requirement features, performance requirement features, and risk requirement features. Based on the task requirement characteristics, a dynamic task announcement for the target task package is generated, and the task requirement characteristics of all dynamic task announcements are aggregated into a task requirement characteristic set. Based on the UAV performance feature set and mission requirement feature set, the matching relationship between the UAV and the standard mission package is established one by one, the corresponding estimated matching cost is calculated, and the estimated matching cost matrix is ​​constructed. The estimated matching cost includes path cost, capability adaptation cost and energy consumption cost. Based on the estimated matching cost matrix, bidding decisions, conflict resolution, and consistency verification are performed on each dynamic task announcement to obtain the task allocation results.

6. The intelligent inspection method for urban and rural areas based on multi-UAV collaboration according to claim 5, characterized in that, The process of making bidding decisions, resolving conflicts, and verifying consistency for each dynamic task announcement based on the estimated matching cost matrix includes: Each dynamic task announcement is selected as the target task announcement, and the estimated matching cost of each UAV in the UAV swarm for the target task announcement is extracted from the estimated matching cost matrix to obtain the estimated cost set. Extract the state factors of each UAV and calculate the bidding cost set by combining them with the estimated cost set; The drone with the smallest bid cost in the set of bid costs is selected as the winning drone in the target mission announcement, and a primary allocation list is generated based on each target mission announcement and the corresponding winning drone. Spatiotemporal conflict detection is performed on each winning UAV in the primary allocation list according to a preset sliding spatiotemporal window to obtain conflict task packages; Conflict resolution is performed based on the bid cost set of the conflict task package, and the primary allocation list is updated based on the result of conflict resolution to obtain the task allocation list. A global consistency check based on risk priority is performed on the task allocation list to obtain the task allocation result.

7. The intelligent inspection method for urban and rural areas based on multi-UAV collaboration according to claim 6, characterized in that, The process of applying multi-dimensional collaborative constraints and collaborative path planning to the task allocation results includes: Based on the task allocation results, the spatial boundary and potential energy centroid coordinates of the target task package corresponding to each matched UAV are extracted, and an initial guiding vector field is constructed by combining the gradient distribution of the risk space potential energy field. Based on the initial guidance vector field, the initial reference trajectory of each matched UAV is calculated, and the relative displacement and estimated arrival time between each matched UAV are calculated according to the initial reference trajectory, thus constructing a spatiotemporal occupancy prediction model. Based on the aforementioned spatiotemporal occupancy prediction model, UAV collision avoidance constraints and spatiotemporal occupancy constraints are introduced to construct a cooperative obstacle avoidance potential field. Communication link constraints are introduced into the cooperative obstacle avoidance potential field to construct a safe and feasible region; The initial reference trajectories of each matched UAV are dynamically optimized within the safe and feasible domain based on the rolling time-domain optimization algorithm to obtain a collaborative inspection trajectory set.

8. The intelligent inspection method for urban and rural areas based on multi-UAV collaboration according to claim 7, characterized in that, The process of introducing communication link constraints and constructing a safe and feasible region in the cooperative obstacle avoidance potential field includes: A signal gain distribution map of the urban and rural areas is constructed, and spatial interpolation is performed on the signal gain distribution map to obtain a continuous signal gain field; A communication link constraint is introduced, and based on the signal gain field, regions where the signal strength is lower than a preset strength threshold are identified as regions where the communication constraint is violated. The communication constraint violation region is transformed into a repulsive potential energy term and fused with the cooperative obstacle avoidance potential field to obtain a comprehensive spatial potential field. Based on the comprehensive spatial potential field, a set of regions that satisfy the UAV collision avoidance constraints, spatiotemporal occupancy constraints, and communication link constraints and also satisfy spatial continuity are extracted. Then, the spatiotemporal occupancy prediction model is used to filter out regions that satisfy time accessibility from the set of regions to obtain candidate safe regions. Connectivity analysis and reachability verification are performed on the candidate safe regions. Based on the analysis and verification results, isolated regions in the candidate safe regions are eliminated to obtain a safe and feasible region that satisfies collision avoidance constraints, spatiotemporal occupancy constraints, and communication link constraints.

9. The intelligent inspection method for urban and rural areas based on multi-UAV collaboration according to claim 1, characterized in that, The process of fusing and analyzing the inspection image data and environmental status data, and identifying events, includes: Spatiotemporal alignment is performed based on the collection timestamps and collection coordinates marked in the inspection image data and environmental status data to obtain aligned multimodal data; By combining the environmental state data in the aligned multimodal data, the inspection image data is preprocessed and environmental feature compensation is performed. Then, the compensated inspection image data is extracted to obtain a multidimensional enhanced feature set. By combining the prior knowledge provided by the standard task package set, target detection and semantic segmentation are performed on the multidimensional enhanced feature set to obtain the semantic set of the inspection object; The semantic set of the inspection objects is semantically matched with the preset historical risk semantic database, and abnormal deviations are extracted from the semantic matching results based on the spatiotemporal correlation analysis method to obtain potential inspection events. Based on the multi-criteria decision analysis method, the potential inspection events are subjected to logical consistency verification and confidence verification to obtain the inspection event identification results.

10. A smart inspection system for urban and rural areas based on multi-UAV collaboration, implementing the method described in any one of claims 1-9, characterized in that, The system includes a grid division module, a task allocation module, a path planning module, an inspection and data collection module, and a fusion analysis module. The grid partitioning module is used to perform spatial overlay analysis on multi-source heterogeneous data in urban and rural areas to obtain a dynamic risk heat map. Based on the heterogeneous task density function, the dynamic risk heat map is transformed into a risk spatial potential energy field, and the risk spatial potential energy field is subjected to adaptive grid clustering partitioning to obtain a standard task package set. The task allocation module is used to construct a UAV performance feature set based on the performance parameters of the UAV swarm, use the risk space potential energy field as a global scheduling constraint, and allocate tasks to the standard task package set based on the UAV performance feature set using the improved contract net algorithm to obtain the task allocation result. The path planning module is used to perform multi-dimensional collaborative constraints and collaborative path planning on the task allocation results with the risk space potential energy field as the objective function, so as to obtain a collaborative inspection track set. The multi-dimensional collaborative constraints include UAV collision avoidance constraints, spatiotemporal occupancy constraints and communication link constraints. The inspection and data acquisition module is used to control the UAV swarm to execute the standard task package set according to the collaborative inspection track set, and to collect multi-source data on the urban and rural areas to obtain inspection image data and environmental status data. The fusion analysis module is used to perform fusion analysis and event recognition on the inspection image data and environmental status data to obtain the inspection event recognition results.

Images