Patrol route self-adjustment method and system based on drone inspection

Abstract

The present invention provides a patrol route self-adjustment method and system based on unmanned aerial vehicle (UAV) inspection. First, a multi-source environmental perception data set of a target area is acquired. Subsequently, feature extraction is performed on the multi-source environmental perception data set and the data is input into a dynamic route planning model to generate an obstacle avoidance correction vector set and an environmental adaptation parameter set. Spatial position offset compensation is performed on a reference route based on the obstacle avoidance correction vector set to generate an initial correction path node set. Path smoothness optimization processing is then performed on the initial correction path node set according to the environmental adaptation parameter set to generate a final dynamic patrol route. Finally, the dynamic patrol route is converted into a waypoint control instruction stream executable by a UAV flight control system and transmitted to an onboard controller in real time to drive the UAV to perform inspection tasks, thereby realizing dynamic self-adjustment of the UAV inspection route and improving the flexibility and safety of the inspection.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle (UAV) control technology, and in particular to a patrol route self-adjustment method and system based on UAV inspection. Background Art

[0002] In the field of drone inspections, the rapid development of drone technology has led to its widespread application in numerous fields, including power inspections, agricultural monitoring, and environmental monitoring. However, traditional drone inspection route planning methods are often based on pre-set fixed routes, which are inadequate in complex and changing natural environments. For example, in mountainous areas, forests, or densely built-up urban areas, fixed routes may not effectively avoid obstacles, increasing the risk of drone collisions. Furthermore, weather conditions (such as wind speed, direction, and visibility) at different times of day can significantly impact drone flight safety and inspection effectiveness. Therefore, dynamically adjusting drone inspection routes based on real-time environmental perception data to accommodate complex environmental and weather changes has become a pressing issue. Summary of the Invention

[0003] In view of the above-mentioned problems, in combination with the first aspect of the present invention, an embodiment of the present invention provides a method for self-adjusting a patrol route based on drone inspection, the method comprising:

[0004] Acquire a multi-source environmental perception data set of a target area, wherein the multi-source environmental perception data set includes a synchronously collected multispectral image sequence, a three-dimensional lidar point cloud data, and real-time meteorological sensor data;

[0005] Extracting features from the multi-source environmental perception data set to obtain multi-source environmental perception features;

[0006] Inputting the multi-source environmental perception features into a dynamic route planning model to generate an obstacle avoidance correction vector set and an environmental adaptation parameter set;

[0007] Performing spatial position offset compensation on the three-dimensional path nodes of the reference route based on the obstacle avoidance correction vector set to generate an initial correction path node set;

[0008] The path smoothness optimization processing is performed on the initial corrected path node set according to the environmental adaptation parameter set to generate a final dynamic patrol route, and the final dynamic patrol route is converted into a waypoint control instruction stream executable by the UAV flight control system, and transmitted to the airborne controller in real time to drive the UAV to perform the inspection task.

[0009] On the other hand, an embodiment of the present invention also provides a patrol route self-adjustment system based on drone inspection, including a processor and a machine-readable storage medium, wherein the machine-readable storage medium is connected to the processor, the machine-readable storage medium is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the machine-readable storage medium to implement the above method.

[0010] Based on the above aspects, by acquiring and fusing multi-source environmental perception data, it is possible to comprehensively and accurately perceive the environmental status of the target area. Through feature extraction and dynamic route planning models, a set of obstacle avoidance correction vectors and a set of environmental adaptation parameters are generated, thereby achieving precise correction and optimization of the reference route. Spatial position offset compensation based on the obstacle avoidance correction vector set effectively avoids obstacles and improves the flight safety of the UAV. The path smoothness optimization processing based on the environmental adaptation parameter set further improves the rationality of the patrol route and the flight efficiency. Finally, the dynamic patrol route is converted into a waypoint control instruction stream executable by the UAV flight control system and transmitted to the onboard controller in real time, realizing the automation and intelligent execution of the UAV inspection task, significantly improving the flexibility and adaptability of the UAV inspection, reducing flight risks, and improving the inspection effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 The figure is a schematic diagram of the execution flow of the patrol route self-adjustment method based on drone inspection provided by an embodiment of the present invention.

[0012] Figure 2 Schematic diagram of exemplary hardware and software components of a patrol route self-adjustment system based on drone inspection provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0013] The present invention will be described in detail below with reference to the accompanying drawings. Figure 1 The present invention provides a flowchart of a method for self-adjusting patrol routes based on drone inspections according to an embodiment of the present invention. The method for self-adjusting patrol routes based on drone inspections is introduced in detail below.

[0014] Step S110: Acquire a multi-source environmental perception data set of the target area, wherein the multi-source environmental perception data set includes a synchronously collected multispectral image sequence, three-dimensional lidar point cloud data, and real-time meteorological sensor data.

[0015] In this example, the scenario is a large-scale warehouse and logistics park, where a drone needs to conduct inspections. In order for the drone to plan patrol routes smoothly and safely, it must first obtain comprehensive and synchronized multi-source environmental perception data for the area.

[0016] Multispectral image sequences are acquired by a specialized multispectral image acquisition device mounted on a drone. Assume the acquisition device captures n1 multispectral images per second, forming an image sequence I1, I2, I3, and so on. These images cover the entire warehouse and logistics park. Image information in different wavelengths can reveal the characteristics of different ground features. For example, some wavelengths are sensitive to metal materials, while others better capture the textures of vegetation or buildings.

[0017] 3D LiDAR point cloud data is collected by a 3D LiDAR device mounted on a drone. The LiDAR emits a laser beam at a constant frequency into the surrounding space and receives the reflected signal. Based on information such as the time and angle of return of the reflected signal, it determines the position of surface points in space, generating point cloud data. The point cloud data generated by each scan is labeled P1, P2, P3, etc. This point cloud data accurately depicts the 3D spatial position and shape of objects such as buildings, cargo stacks, and vehicles within the warehouse and logistics park.

[0018] Real-time meteorological sensor data is acquired by meteorological sensors mounted on drones. These sensors monitor meteorological parameters within the park, such as wind speed, direction, and visibility, in real time. Assuming meteorological data is recorded every t1 interval, this creates a series of meteorological data, M1, M2, M3, and so on. These data reflect the changing weather conditions within the park at different times. A precise synchronization mechanism ensures that the multispectral image sequence, 3D LiDAR point cloud data, and real-time meteorological sensor data are fully synchronized in acquisition time, thus forming a multi-source environmental perception data set for the target area.

[0019] Step S120: extracting features from the multi-source environment perception data set to obtain multi-source environment perception features.

[0020] After acquiring a multi-source environmental perception data set for the warehousing and logistics park, key features must be extracted to effectively utilize this data for dynamic patrol route planning. This process involves performing specialized feature extraction operations on multispectral image sequences, 3D LiDAR point cloud data, and real-time meteorological sensor data, and then aggregating and integrating the extracted features.

[0021] Step S121: performing terrain semantic segmentation processing on the multispectral image sequence to generate a surface cover type distribution map and an obstacle initial position feature set.

[0022] For multispectral image sequences of warehousing and logistics parks, a pre-trained semantic segmentation neural network is used to classify objects within each pixel. This pre-trained semantic segmentation neural network is trained using a large amount of multispectral image data from various scenarios (including but not limited to warehousing and logistics parks, urban areas, and industrial plants) to learn the characteristic patterns of different objects in multispectral images.

[0023] Taking a multispectral image, I, as an example, the neural network classifies each pixel into different categories based on its multi-band spectral information, forming classified pixel regions. These regions include vegetation, water, buildings, and temporary obstacles. For example, pixels in a certain area of image I are classified as buildings based on their reflectance characteristics in multiple bands, including the near-infrared and visible light bands.

[0024] The classified pixel areas are processed by morphological closing operations. Morphological closing operations include a dilation operation followed by an erosion operation. The dilation operation will expand each pixel area to the surrounding area, making it possible to connect adjacent pixel areas of the same type. Assuming that for a certain pixel area A, the dilation operation will be centered on the boundary pixels of the area, search for similar pixels within a certain neighborhood range, and merge them into the area. The subsequent erosion operation will shrink the area and remove some noise or isolated pixels introduced by the dilation. Through this closing operation, a continuous surface cover patch is generated. For example, some originally scattered pixel areas representing small buildings will form a larger, continuous building surface cover patch after the closing operation.

[0025] Extract the geometric center coordinates and boundary contour features of each surface cover patch. For each continuous surface cover patch, determine its geometric center coordinates through a certain geometric calculation method. Assuming that the patch is irregular in shape, the geometric center coordinates are approximated by calculating the average value of the coordinates of all pixel points in the patch. At the same time, extract the boundary contour features of the patch. This can be done by using an edge detection algorithm, such as the Canny edge detection algorithm, to determine the edge pixels of the patch and then obtain the boundary contour features. Use these geometric center coordinates and boundary contour features to construct a surface cover type distribution map. For example, the geometric center coordinates and boundary contour features of different types of surface cover patches are recorded in a certain format to form a data structure as a surface cover type distribution map.

[0026] Match the boundary contour features of the temporary obstacle area with the preset obstacle feature template. The preset obstacle feature template is pre-constructed based on common obstacle shapes, sizes and other features. For example, for common cargo pile obstacles, its approximate shape, size range and other features are constructed as a template. By comparing the similarity between the boundary contour features of the temporary obstacle area and the template features, significant obstacle areas are screened out. The similarity calculation can be based on a variety of methods, such as calculating the shape similarity and size matching of the contours. For areas where the similarity exceeds a certain threshold, they are determined to be significant obstacle areas, and a set of obstacle initial position features is generated. This set records the location, approximate shape and other features of the significant obstacle area.

[0027] Step S122: performing spatial density cluster analysis on the three-dimensional lidar point cloud data, constructing a set of three-dimensional geometric models of obstacles, and extracting the spatial occupancy features of each obstacle.

[0028] In the warehouse and logistics park scenario, the acquired 3D LiDAR point cloud data is first subjected to voxel meshing. Voxel meshing divides the entire 3D space into a number of uniformly sized voxel units. Assuming the 3D space of the warehouse and logistics park is divided into voxel units, each of size v1×v2×v3 (where v1, v2, and v3 represent the dimensions of the voxel along the three coordinate axes, respectively), a uniformly distributed set of voxel units is generated.

[0029] Perform density statistics on the point cloud data within each voxel in the voxel set. Calculate the number of point cloud data points within each voxel and use this as the density of that voxel. Voxels with a density exceeding a preset threshold T1 are selected as candidate obstacle regions. For example, if the number of point cloud data points within a voxel is greater than T1, the voxel is considered to contain an obstacle.

[0030] Connected domain analysis is performed on the candidate obstacle regions. Connected domain analysis searches for adjacent voxel units in space whose density exceeds a threshold and merges them into a single entity. For example, if two adjacent voxel units both meet the condition of density exceeding threshold T1, they are merged into a single obstacle cluster. In this way, adjacent voxel units are merged to form an obstacle cluster.

[0031] A minimum bounding cube (LCC) is fitted for each obstacle cluster. For each obstacle cluster, a specific algorithm is used to find a minimum LC that contains all point cloud data within the cluster. The center coordinates, size parameters, and surface normal direction of the LCC are calculated. The center coordinates can be obtained by averaging the coordinates of all point cloud data points within the cluster. The size parameters are determined by the difference between the maximum and minimum lengths of the LCC along the three coordinate axes. The surface normal direction can be determined by geometric analysis of the LCC surface.

[0032] Based on the center coordinates and size parameters, a 3D geometric model set is generated. This 3D geometric model set records the approximate 3D shape and position of each obstacle. The surface normal direction of each cube is also extracted as a spatial occupancy feature, which can be used to subsequently analyze the obstacle's orientation in space and its relative position to other objects.

[0033] Step S123: performing wind speed gradient modeling and visibility attenuation analysis on the real-time meteorological sensor data to generate a set of meteorological interference influence coefficients.

[0034] In the warehousing and logistics park, real-time meteorological sensor data is processed. First, the real-time meteorological sensor data is sampled in a time series, assuming that sampling occurs every t2 interval. The wind speed vector and visibility measurement values at the current moment are obtained. The wind speed vector contains wind speed magnitude and wind direction information. Assume that the wind speed vector obtained from a particular sampling is W (including wind speed magnitude w1 and wind direction angle θ1), and the visibility measurement value is V1.

[0035] A wind speed change rate prediction model is constructed based on historical wind speed data. Wind speed data from a past period is collected. For example, a series of wind speed data, W1, W2, ..., Wn2, is collected from time t-n2 to time t. Using data analysis methods such as the autoregressive integrated moving average (ARIMA) model used in time series analysis, a wind speed change rate prediction model is constructed. This wind speed change rate prediction model is used to calculate the wind speed gradient curve for the next N sampling periods. For example, based on the changing trends of the historical wind speed data, the wind speed change rate prediction model predicts the wind speed change for each of the next N sampling periods, thereby forming a wind speed gradient curve.

[0036] Based on the visibility measurement value V1 and a preset attenuation coefficient model, a visibility compensation parameter is generated to measure the effect of visibility on LiDAR detection accuracy. The preset attenuation coefficient model is based on experimental data and theoretical analysis, and it describes the relationship between visibility and LiDAR detection accuracy. For example, based on the visibility measurement value V1, by searching the relationship table or calculation formula corresponding to the attenuation coefficient model, the visibility compensation parameter C1 is obtained. This parameter is used to compensate for the impact of visibility changes on LiDAR detection accuracy.

[0037] The wind speed gradient curve is decomposed into a horizontal offset coefficient and a vertical turbulence intensity coefficient. By decomposing the wind speed gradient curve in the horizontal and vertical directions, it is assumed that the wind speed gradient curve can be represented as a vector sequence, with its horizontal component as the horizontal offset coefficient H1 and its vertical component as the vertical turbulence intensity coefficient Vt1.

[0038] The horizontal offset coefficient H1, vertical turbulence intensity coefficient Vt1, and visibility compensation parameter C1 are integrated to generate a set of meteorological interference impact coefficients. This set contains the correlation coefficients of the potential interference of meteorological conditions on drone flight and lidar detection, which are used for subsequent route planning adjustments.

[0039] Step S124: aggregating the surface cover type distribution map, the obstacle initial position feature set, the three-dimensional geometric model set, the space occupancy feature, and the meteorological interference influence coefficient set to obtain the multi-source environment perception feature.

[0040] In the warehousing and logistics park scenario, the various features extracted from multispectral image sequences, 3D LiDAR point cloud data, and real-time meteorological sensor data are aggregated. The surface cover type distribution map, which records the distribution of different surface cover types within the park, is represented by data structure D1; the obstacle initial position feature set, which records the initial position and approximate shape of obstacles identified from multispectral imagery, is represented by set S1; the 3D geometric model set, which contains the 3D geometric models of obstacles constructed from 3D LiDAR point cloud data, is represented by set G1; the spatial occupancy features, which contain the surface normal vector direction corresponding to each 3D obstacle geometric model, are represented by set O1; and the meteorological interference coefficient set, which contains the coefficients of influence of meteorological factors such as wind speed and visibility on drone flight and LiDAR detection, is represented by set M1. By combining these data structures and sets according to certain rules, a unified data structure or set is formed as the multi-source environmental perception feature, denoted as feature set F1. This completes the feature extraction process for multi-source environmental perception data and provides comprehensive environmental feature information for subsequent dynamic route planning.

[0041] Step S130: Input the multi-source environmental perception features into a dynamic route planning model to generate an obstacle avoidance correction vector set and an environmental adaptation parameter set.

[0042] In the warehousing and logistics park scenario, the multi-source environmental perception feature set F1 obtained above is input into the dynamic route planning model. This dynamic route planning model consists of multiple modules, including an encoder, a spatiotemporal evolution prediction module, a path node offset prediction network, an environmental compensation generator, a vector fusion layer, and a fully connected decoder. These modules work together to generate a reasonable set of obstacle avoidance correction vectors and environmental adaptation parameters based on environmental characteristics.

[0043] Step S131: Input the multi-source environmental perception features into the encoder of the dynamic route planning model, perform spatiotemporal alignment on the heterogeneous features through a cross-channel attention mechanism, and generate a multi-source environmental perception tensor with unified dimension.

[0044] The multi-source environmental perception feature set F1 contains heterogeneous features from different data sources, such as a map of land cover types and a feature set of initial obstacle positions. These features are fed into the encoder of the dynamic route planning model. The encoder processes these heterogeneous features so that they can work together in subsequent model processing.

[0045] The cross-channel attention mechanism learns the importance weights between different feature channels in the encoder, reweighting and combining heterogeneous features to achieve spatiotemporal alignment. For different features in the multi-source environmental perception feature set F1, such as the land cover type distribution map D1 and the obstacle initial position feature set S1, the cross-channel attention mechanism analyzes their correlation across different temporal and spatial dimensions. For feature data at a given time t, the attention mechanism calculates the importance weights W1, W2, and so on for each feature element in D1 and S1 across different channels. These weights are then applied to the feature elements, aligning them across time and space. This process generates a multi-source environmental perception tensor T1 with unified dimensions. This multi-source environmental perception tensor has a unified data structure and format, facilitating processing by subsequent model modules.

[0046] Step S132: Input the multi-source environmental perception tensor into the spatiotemporal evolution prediction module of the dynamic route planning model, use the three-dimensional gated convolution layer to extract the spatiotemporal coupling relationship between the dynamic displacement trend of obstacles and meteorological disturbances, and generate the obstacle risk probability distribution and meteorological interference propagation field.

[0047] The dimensionally unified multi-source environmental perception tensor T1 is fed into the spatiotemporal evolution prediction module. This module analyzes how environmental characteristics change over time and space to predict the impact of future environmental conditions on flight routes.

[0048] The 3D gated convolutional layer plays a core role in the spatiotemporal evolution prediction module. It performs a convolution operation on the multi-source environmental perception tensor T1 in three dimensions (two spatial dimensions and one temporal dimension), while introducing a gating mechanism to control the flow of information. For each location (x, y, t) in the multi-source environmental perception tensor T1 (where x and y represent spatial location and t represents time), the 3D gated convolutional layer extracts the spatiotemporal coupling relationship between the dynamic displacement trend of the obstacle and the meteorological disturbance based on the characteristic information of the surrounding locations and the gating signal.

[0049] For example, for obstacles within a warehousing and logistics park, by combining the positional changes of obstacles reflected by multispectral imagery and LiDAR point cloud data at different times, along with concurrent meteorological data, the 3D gated convolutional layer can learn how obstacles may dynamically shift as weather conditions change. This processing generates an obstacle risk probability distribution P1, which represents the probability of an obstacle posing a risk to drone flight at different spatial locations and time points. Simultaneously, a meteorological interference propagation field F2 is generated, describing the spatial propagation of meteorological interference (such as wind speed and direction changes) and the extent of its impact on different areas.

[0050] Step S133: Perform three-dimensional spatial gradient inversion on the obstacle risk probability distribution through the path node offset prediction network of the dynamic route planning model to generate an obstacle avoidance repulsive force vector in the tangential direction of each path node of the reference route.

[0051] After obtaining the obstacle risk probability distribution P1, it is input into the path node offset prediction network. The task of this network is to determine how each path node on the reference route needs to be offset to avoid obstacles based on the obstacle risk probability distribution.

[0052] Three-dimensional spatial gradient inversion is a key operation in the path node offset prediction network. The obstacle risk probability distribution P1 has different probability values distributed in three-dimensional space. By calculating the gradient of the probability value in three-dimensional space, the direction and magnitude of the probability change at each location are determined. For example, for a path node N1 on the reference route, the gradient of the obstacle risk probability distribution P1 is calculated in the three-dimensional spatial region surrounding it. If the probability increases rapidly in a certain direction, it indicates that the risk of approaching obstacles in that direction is increasing.

[0053] Based on the gradient calculation results, an obstacle avoidance repulsion force vector is generated for each path node along the baseline route in the tangential direction. This obstacle avoidance repulsion force vector represents the direction and magnitude of the repulsive force that the path node should experience in the tangential direction to avoid obstacles. For example, for path node N1, the generated obstacle avoidance repulsion force vector R1 points away from high-risk obstacle areas, and its magnitude is determined by the gradient magnitude and preset parameters (such as the safety distance factor). These obstacle avoidance repulsion force vectors will be used to subsequently adjust the position of the path nodes along the baseline route.

[0054] Step S134: input the meteorological interference propagation field into the environmental compensation generator of the dynamic route planning model, extract the horizontal wind speed compensation gradient and the vertical turbulence suppression coefficient through recurrent neural network sequence modeling, and generate a meteorological interference compensation parameter set.

[0055] The meteorological interference propagation field F2 is input into the environmental compensation generator. The function of the environmental compensation generator is to generate parameters that can compensate for the impact of meteorological interference on the flight of the UAV based on the information of the meteorological interference propagation field.

[0056] Recurrent neural network sequence modeling plays a key role in the environmental compensation generator. It analyzes and models the time series information in the meteorological interference propagation field F2. This field contains meteorological interference information at different times and spatial locations, such as how wind speed and direction change over time.

[0057] Through recurrent neural network processing, the horizontal wind speed compensation gradient H2 and the vertical turbulence suppression coefficient Vt2 are extracted. For the horizontal wind speed compensation gradient H2, the recurrent neural network predicts the gradient values required to compensate for the impact of horizontal wind speed on the UAV's flight path at different locations and times based on the changing trend of horizontal wind speed in the meteorological interference propagation field F2. The vertical turbulence suppression coefficient Vt2 determines how to suppress the impact of vertical turbulence on the UAV's flight stability based on the intensity and variation of turbulence in the vertical direction.

[0058] The horizontal wind speed compensation gradient H2 and the vertical turbulence suppression coefficient Vt2 are combined to generate the meteorological interference compensation parameter set C2. The parameters in this set will be used to subsequently adjust the route to adapt to changing meteorological conditions and ensure the stability and safety of the drone flight.

[0059] Step S135: Input the obstacle avoidance repulsion force vector and the weather interference compensation parameter set into the vector fusion layer of the dynamic route planning model, use the learnable weight matrix to perform direction vector synthesis and module length normalization, and generate an obstacle avoidance correction vector set containing three-dimensional offset components and heading angle adjustment weights.

[0060] The obstacle avoidance repulsion force vector (such as R1) and the weather interference compensation parameter set C2 are input into the vector fusion layer. The purpose of the vector fusion layer is to comprehensively consider the obstacle avoidance requirements and the impact of weather conditions on the route, and generate a unified set of obstacle avoidance correction vectors.

[0061] First, the obstacle avoidance repulsion force vectors are unit-vectorized in the three-dimensional coordinate system. Each obstacle avoidance repulsion force vector, such as R1, is converted to a unit vector to obtain the obstacle avoidance direction reference vector U1, and its modulus scaling factor k1 is extracted. This unit-vectorization process ensures that all obstacle avoidance repulsion force vectors have a unified directional representation, facilitating subsequent synthesis operations with other vectors.

[0062] The horizontal wind speed compensation gradient H2 in the meteorological interference compensation parameter set C2 is decomposed into a lateral wind pressure offset vector Hh1 and a longitudinal airflow offset vector Hv1. At the same time, the vertical turbulence suppression coefficient Vt2 is mapped into a vertical fluctuation suppression vector Vv1. This vector decomposition and mapping operation is based on the impact of horizontal wind speed and vertical turbulence on UAV flight in different directions.

[0063] The lateral wind pressure offset vector Hh1, the longitudinal airflow offset vector Hv1, and the vertical fluctuation suppression vector Vv1 are superimposed in three dimensions to generate the meteorological interference compensation vector Mv1. This meteorological interference compensation vector integrates the impact of meteorological conditions on UAV flight in different directions.

[0064] Based on the relevant performance parameters set by the UAV's flight control system, the modulus of the meteorological interference compensation vector Mv1 is dynamically constrained and truncated. The UAV's flight control system specifies parameters related to flight performance, such as maximum flight speed and maneuvering acceleration threshold. Assume that the maximum allowable modulus of the meteorological interference compensation vector corresponding to the maximum flight speed specified by the flight control system is Lmax, and the maximum allowable modulus change rate of the meteorological interference compensation vector corresponding to the maneuvering acceleration threshold is Kmax. By comparing the modulus |Mv1| of the meteorological interference compensation vector Mv1 with Lmax and the modulus change rate with Kmax, if |Mv1| exceeds Lmax or the modulus change rate exceeds Kmax, the meteorological interference compensation vector Mv1 is truncated so that its modulus and modulus change rate meet the UAV's dynamic performance requirements, generating a normalized meteorological compensation vector Mn1 that meets the UAV's dynamic performance.

[0065] The obstacle avoidance reference vector U1 and the normalized weather compensation vector Mn1 are input into a learnable weight matrix W. This learnable weight matrix W is learned from a large amount of training data during model training. Its function is to rationally adjust the weights of obstacle avoidance and weather compensation factors in the final correction vector based on different environmental conditions and flight requirements. Through matrix multiplication and addition operations, each component of the obstacle avoidance reference vector U1 and the corresponding component of the normalized weather compensation vector Mn1 are multiplied by the corresponding weight elements in the learnable weight matrix W. The resulting products are then added together to generate the three-dimensional spatial offset composite vector Sv1.

[0066] A dynamic attenuation factor, D, is used to map the heading angle offset to the composite 3D offset vector, Sv1. This factor is dynamically determined based on the drone's current flight state, environmental conditions, and other factors. For example, when the drone approaches an obstacle or weather conditions are adverse, the value of D is adjusted to more accurately reflect the environmental impact on the heading angle. The heading angle adjustment weight parameter, α1, is extracted by multiplying the composite 3D offset vector, Sv1, by the dynamic attenuation factor, D.

[0067] Dynamically scale the 3D offset composite vector Sv1 according to the modulus scaling factor k1. This factor, extracted when the obstacle avoidance repulsion force vector is unit-vectorized, reflects the relative magnitude of the original obstacle avoidance repulsion force vector. The 3D offset component δ1 is generated by multiplying each component of the 3D offset composite vector Sv1 by the modulus scaling factor k1.

[0068] The three-dimensional offset component δ1 is concatenated with the heading angle adjustment weight parameter α1 to generate the correction vector Rv1 corresponding to each path node in the obstacle avoidance correction vector set. This process comprehensively considers the impact of obstacle avoidance and weather factors on the route, generating an obstacle avoidance correction vector set that contains both spatial position offset information and heading angle adjustment information, providing a basis for subsequent adjustments to the baseline route.

[0069] Step S136: Based on the hidden state features of the spatiotemporal evolution prediction module, the dynamic impact factor of visibility attenuation on the safety distance of path nodes is extracted through the fully connected decoder of the dynamic route planning model, and an environmental adaptation parameter set that matches the dimension of the obstacle avoidance correction vector set is generated.

[0070] When the spatiotemporal evolution prediction module processes the multi-source environmental perception tensor T1, it generates hidden state features Hid1. These features contain rich information about how environmental characteristics change over time and space. These include not only the spatiotemporal coupling between obstacle dynamic displacement trends and meteorological disturbances, but also other information potentially influencing route planning.

[0071] The hidden state feature Hid1 is fed into the fully connected decoder of the dynamic route planning model. The fully connected decoder extracts information related to the safe spacing between path nodes from the hidden state feature, specifically the impact of visibility degradation on the safe spacing between path nodes.

[0072] A fully connected decoder extracts information through a series of neuron connections and weight adjustments. Assume that the fully connected decoder consists of multiple layers of neurons, each connected by a weight matrix. The hidden state feature Hid1 is first input to the first layer of neurons. After weighted summation and activation function processing, the intermediate feature representation I1 is obtained. This intermediate feature representation I1 is then sequentially input to subsequent layers of neurons. After layer-by-layer processing, the final output is the dynamic impact factor β1 of visibility attenuation on the safe spacing between path nodes.

[0073] The calculation of the dynamic impact factor β1 involves a comprehensive analysis of the various elements in the hidden state feature Hid1. For example, the hidden state feature Hid1 may contain visibility information at different times, obstacle locations, and the drone's flight status. Based on the complex relationships between these elements, the fully connected decoder determines how visibility degradation affects the safe spacing between path nodes through weight adjustment and neuron processing.

[0074] Generate an environmental adaptation parameter set E1 that matches the dimensions of the obstacle avoidance correction vector set. The obstacle avoidance correction vector set adjusts the position and heading angle of the baseline route to avoid obstacles and adapt to weather conditions, while the environmental adaptation parameter set E1 ensures that the route meets safety spacing requirements under different environmental conditions. By matching the dimensions of the environmental adaptation parameter set E1 with the obstacle avoidance correction vector set, subsequent route adjustments can comprehensively consider multiple factors such as obstacle avoidance, weather adaptation, and safety spacing, generating a reasonable dynamic patrol route.

[0075] Step S140: performing spatial position offset compensation on the three-dimensional path nodes of the reference route based on the obstacle avoidance correction vector set to generate an initial correction path node set.

[0076] In the warehousing and logistics park scenario, the baseline route is a pre-defined drone inspection route consisting of a series of 3D path nodes. To enable the drone to avoid obstacles and adapt to weather conditions, the 3D path nodes of the baseline route need to be adjusted based on a set of obstacle avoidance correction vectors.

[0077] Step S141: extracting the spatial coordinate set of each three-dimensional path node in the reference route in the original geographic coordinate system.

[0078] The baseline route has clear path node definitions in the geographic coordinate system. Assume that the baseline route consists of path nodes P1, P2, P3, etc. Each path node has corresponding three-dimensional spatial coordinates in the geographic coordinate system, denoted as (X1, Y1, Z1), (X2, Y2, Z2), (X3, Y3, Z3), etc. These coordinates describe the path node's position in real space. Through data reading and parsing operations, the spatial coordinate set C1 = {(X1, Y1, Z1), (X2, Y2, Z2), (X3, Y3, Z3), etc. of these three-dimensional path nodes in the original geographic coordinate system is extracted from the baseline route's data storage structure.

[0079] Step S142: performing a three-dimensional Euclidean space vector superposition operation on the spatial coordinate set according to the three-dimensional offset components corresponding to each path node in the obstacle avoidance correction vector set to generate an adjusted three-dimensional space coordinate set.

[0080] Each path node in the obstacle avoidance correction vector set has a corresponding three-dimensional offset component, such as the three-dimensional offset components δ1, δ2, δ3 generated previously. For each path node coordinate in the reference route spatial coordinate set C1, such as (Xi, Yi, Zi), a three-dimensional Euclidean space vector superposition operation is performed based on the corresponding three-dimensional offset components (δxi, δyi, δzi). That is, the adjusted coordinates are (Xi+δxi, Yi+δyi, Zi+δzi). By performing this operation on all path node coordinates in the spatial coordinate set C1, the adjusted three-dimensional spatial coordinate set C2 = {(X1+δx1, Y1+δy1, Z1+δz1), (X2+δx2, Y2+δy2, Z2+δz2), (X3+δx3, Y3+δy3, Z3+δz3)...} is generated. This process causes the path nodes to be offset in three-dimensional space according to the obstacle avoidance requirements.

[0081] Step S143: Based on the heading angle adjustment weight in the obstacle avoidance correction vector set, the adjusted three-dimensional space coordinate set is compensated for the heading angle deflection of the drone to generate the path node coordinates after the heading angle is corrected.

[0082] The obstacle avoidance correction vector set also includes heading angle adjustment weight parameters, such as α1, α2, α3, etc. For each path node coordinate in the adjusted three-dimensional space coordinate set C2, such as (Xi+δxi, Yi+δyi, Zi+δz1), heading angle deflection compensation is performed based on the corresponding heading angle adjustment weight parameter αi. Assuming the current heading angle of the drone is θi, the heading angle is adjusted based on the heading angle adjustment weight parameter αi. The new heading angle θi'=θi+f(αi), where f(αi) is the heading angle adjustment function determined by the heading angle adjustment weight parameter αi. This process generates the coordinates of the path nodes after the heading angle correction, forming a new coordinate set C3 = {(X1+δx1, Y1+δy1, Z1+δz1, θ1'), (X2+δx2, Y2+δy2, Z2+δz2, θ2'), (X3+δx3, Y3+δy3, Z3+δz3, θ3')...}. This set contains not only the adjusted spatial positions of the path nodes but also the corrected heading angle information.

[0083] Step S144: using the boundary contour features in the multi-source environment perception features, performing collision detection on the path node coordinates after the heading angle correction, and eliminating abnormal path nodes that have spatial overlap with the obstacle three-dimensional geometric model set.

[0084] The multi-source environmental perception feature set F1 includes boundary contour features extracted from multispectral image sequences and a set of 3D obstacle geometric models constructed from 3D lidar point cloud data. For each path node coordinate in the heading-corrected path node coordinate set C3, such as (Xi + δx1, Y1 + δy1, Z1 + δz1, θi'), collision detection is performed using the boundary contour features and the 3D obstacle geometric model set.

[0085] The specific detection process is as follows: First, the drone's position and orientation in space are determined based on the coordinates of the path nodes. Then, a detection area is defined, centered on the drone, based on its size and flight attitude. This detection area is compared with each obstacle model in the set of 3D geometric models to determine whether there is any spatial overlap. For example, for a 3D geometric model of an obstacle O1, if the detection area overlaps with O1 in space, the path node is considered to have a collision risk and is removed from the coordinate set C3. Through this detection and elimination operation, a coordinate set C4 of path nodes is obtained that does not contain any spatial overlap with obstacles.

[0086] Step S145: Perform airspace connectivity verification based on the coordinates of the eliminated path nodes to generate an initial corrected path node set that meets the UAV flight safety distance constraints.

[0087] For the coordinate set C4 after collision detection and elimination of abnormal path nodes, airspace connectivity verification is required to ensure that the UAV can fly smoothly along these path nodes during the flight and meet the flight safety distance constraints.

[0088] Airspace connectivity verification checks whether the spatial connections between path nodes are reasonable, with no discontinuities or inaccessible areas. For example, it checks whether the distance between adjacent path nodes is within the drone's flight range, and whether the heading angle changes between path nodes are within the drone's operational range. Furthermore, according to the drone's flight safety spacing constraints, it ensures that each path node maintains a sufficient safety distance from surrounding obstacles and other path nodes.

[0089] Assume that the safety distance constraint is that the minimum distance from obstacles and other path nodes in the horizontal direction is d1, and in the vertical direction is d2. For each path node in the coordinate set C4, calculate its distance from surrounding obstacles and other path nodes. If the distance between a path node and the surrounding objects does not meet the safety distance constraint, it is adjusted or eliminated. Through such airspace connectivity verification and safety distance check and adjustment operations, an initial corrected path node set C5 that meets the UAV flight safety distance constraint is generated. The path nodes in this initial corrected path node set will serve as the basis for subsequent further optimization to generate the final dynamic patrol route.

[0090] Step S150: Optimize the path smoothness of the initial corrected path node set according to the environmental adaptation parameter set to generate a final dynamic patrol route, and convert the final dynamic patrol route into a waypoint control instruction stream executable by the UAV flight control system, and transmit it to the airborne controller in real time to drive the UAV to perform the inspection task.

[0091] In the warehousing and logistics park scenario, after obtaining the initial corrected path node set C5, in order to enable the drone to fly more smoothly and efficiently, it is necessary to optimize its path smoothness according to the environmental adaptation parameter set E1, and then convert the optimized route into an instruction stream that can be recognized and executed by the drone flight control system to drive the drone to perform inspection tasks.

[0092] Step S151: Calculate the dynamic safety buffer distance threshold between adjacent nodes in the initial correction path node set based on the dynamic influencing factors in the environmental adaptation parameter set, combined with the current flight speed, braking response time and visibility attenuation parameters of the UAV.

[0093] The dynamic influencing factors β1, β2, β3, etc. in the environmental adaptation parameter set E1 reflect the impact of visibility degradation on the safe spacing between path nodes. Furthermore, the current flight speed V, braking response time t, and visibility degradation parameter γ are considered. For each pair of adjacent path nodes in the initial corrected path node set C5, such as nodes Pi and Pi+1, the dynamic safety buffer distance threshold Di between them is calculated.

[0094] The calculation process is as follows: First, the visibility influence coefficient kβi on the safe distance is determined based on the visibility attenuation parameter γ and the dynamic influence factor βi. Then, based on the UAV's current flight speed V and the braking response time t, the distance dVt = V × t that the UAV can travel during this time is calculated. The dynamic safety buffer distance threshold Di = dVt × kβi. Through this calculation, a dynamic safety buffer distance threshold is determined for each pair of adjacent path nodes. This dynamic safety buffer distance threshold changes according to environmental conditions and the UAV's flight status to ensure that the UAV has sufficient safety buffer space during flight.

[0095] Step S152: performing a uniform node spacing adjustment on the initial corrected path node set based on the dynamic safety buffer distance threshold, and generating an intermediate interpolation node sequence under an equidistance constraint.

[0096] For the initial corrected path node set C5, node spacing is uniformly adjusted based on the calculated dynamic safety buffer distance threshold Di between each pair of adjacent path nodes. For adjacent path nodes Pi and Pi+1, if the actual distance between them is greater than the dynamic safety buffer distance threshold Di, several intermediate interpolation nodes are inserted between them to ensure that the distance between adjacent nodes is as close to the dynamic safety buffer distance threshold Di as possible.

[0097] Assuming the actual distance between Pi and Pi+1 is Li, the number of intermediate interpolation nodes ni required is determined by calculating the proportional relationship between Li and Di. Then, using a specific interpolation algorithm, such as linear interpolation, ni intermediate interpolation nodes are uniformly inserted between Pi and Pi+1. For example, in a linear interpolation algorithm, for node Pi with coordinates (Xi, Yi, Zi) and node Pi+1 with coordinates (Xi+1, Yi+1, Zi+1), the coordinates of the inserted intermediate interpolation nodes (Xij, Yij, Zij) can be calculated using the formulas Xij = Xi+j × (Xi+1-Xi) / (ni+1), Yij = Yi+j × (Yi+1-Yi) / (ni+1), and Zij = Zi+j × (Zi+1-Zi) / (ni+1), where j = 1, 2, ..., ni. By performing this operation on all adjacent path nodes in the initial corrected path node set C5, a sequence of intermediate interpolation nodes S1 under the equidistant constraint is generated. The node spacing in the intermediate interpolation node sequence is more uniform and meets the requirements of the dynamic safety buffer distance threshold.

[0098] Step S153: using a cubic B-spline curve to perform spatial trajectory fitting on the intermediate interpolation node sequence to generate a continuous and smooth three-dimensional track curve.

[0099] Cubic B-spline curves are a commonly used curve fitting method that can generate a smooth curve from a series of discrete points. For the intermediate interpolation node sequence S1 under the equidistant constraint, a cubic B-spline curve is used to fit the spatial trajectory.

[0100] The fitting process of a cubic B-spline curve involves assigning corresponding weights and parameters to each node in the knot sequence. Assuming that the intermediate interpolation knot sequence S1 contains n knots P1, P2, ..., Pn, a smooth curve passing through these knots can be generated by selecting appropriate knot parameters t1, t2, ..., tn and using the cubic B-spline curve calculation formula. The calculation formula for a cubic B-spline curve involves weighted summation of knot coordinates and the operation of basis functions. For example, the coordinates (X, Y, Z) of a point on the curve can be calculated using the formula X = Σ(i=1ton)Ni,3(t) × Xi, Y = Σ(i=1ton)Ni,3(t) × Yi, Z = Σ(i=1ton)Ni,3(t) × Zi, where Ni,3(t) are the cubic B-spline basis functions, Xi, Yi, Zi are the coordinates of the knot Pi, and t is the parameter value corresponding to the point on the curve. Through such fitting operations, a continuous and smooth three-dimensional trajectory curve T1 is generated, so that the UAV can fly smoothly along this curve during the flight.

[0101] Step S154: Based on the horizontal direction offset coefficient in the multi-source environment perception feature, the horizontal plane wind pressure resistance curvature of the three-dimensional track curve is optimized to generate an anti-interference smooth track.

[0102] The multi-source environmental perception feature set F1 includes the horizontal offset coefficient H1 extracted from real-time meteorological sensor data. The horizontal offset coefficient H1 reflects the impact of horizontal wind pressure on the UAV's flight path. For the continuous and smooth three-dimensional trajectory curve T1, the wind pressure-resistant curvature is optimized on the horizontal plane based on the horizontal offset coefficient H1.

[0103] The specific optimization process is as follows: first, the three-dimensional trajectory curve T1 is projected onto the horizontal plane to obtain the horizontal trajectory curve Th. For each point on the horizontal trajectory curve Th, the amount of adjustment to the curve curvature is calculated based on the horizontal offset coefficient H1. Assuming that the value of the horizontal offset coefficient H1 in a certain area is h1, the curvature of the horizontal trajectory curve Th in this area needs to be increased or decreased based on the size and direction of h1. By performing such a curvature adjustment operation on all points on the horizontal trajectory curve Th, an anti-interference smooth trajectory Ta is generated. This anti-interference smooth trajectory Ta can better resist the influence of wind pressure on the horizontal plane, ensuring that the drone can fly stably even in windy environments.

[0104] Step S155: Based on the maximum steering angle, minimum turning radius, and maximum centripetal acceleration parameters of the UAV flight control system, the anti-disturbance smooth trajectory is subjected to segmented heading angle continuity verification and curvature smoothing optimization to generate a final dynamic patrol route that satisfies both kinematic and dynamic constraints.

[0105] The UAV flight control system specifies several parameters related to flight performance, such as the maximum steering angle θmax, the minimum turning radius rmin, and the maximum centripetal acceleration amax. For a disturbance-free smooth trajectory Ta, these parameters are used to perform segmented heading angle continuity verification and curvature smoothing optimization.

[0106] The anti-disturbance smoothed trajectory Ta is divided into several segments. For each segment, the heading angle change is checked to ensure it is within the maximum steering angle θmax. If the heading angle change for a segment exceeds θmax, the curvature of that segment is adjusted. The curvature of each segment is also checked and optimized based on the minimum turning radius rmin and the maximum centripetal acceleration amax.

[0107] Assume that the current curvature of a certain track segment is κ. Based on the velocity v at a point on the track and the centripetal acceleration formula a = v² × κ, calculate the centripetal acceleration a corresponding to the current curvature. If a exceeds the maximum centripetal acceleration amax, the curvature of the track segment is reduced to meet the dynamic constraints. Curvature adjustment can be achieved by fine-tuning the coordinates of various points on the track. For example, for point P on the track segment, according to a certain rule, such as moving the point a certain distance in a direction that reduces curvature, the curvature of the entire track segment can be changed.

[0108] During the curvature adjustment process, the continuity of the heading angle must be maintained. If the curvature adjustment results in discontinuous changes in the heading angle, further track optimization is required. For example, at the junction of two adjacent tracks, the coordinates near the junction point can be fine-tuned to ensure a smooth transition in the heading angle. After verifying the heading angle continuity and optimizing the curvature smoothness for each track segment, the final dynamic patrol route L is generated, satisfying both kinematic and dynamic constraints.

[0109] Step S156: Convert the final dynamic patrol route into a waypoint control instruction stream executable by the UAV flight control system, and transmit it to the airborne controller in real time to drive the UAV to perform the inspection task.

[0110] The final dynamic patrol route L is the ideal flight path for the UAV, represented by a continuous, smooth curve. However, the UAV's flight control system requires discrete waypoint control commands to drive the UAV. Therefore, the final dynamic patrol route L needs to be converted into a stream of waypoint control commands that the UAV's flight control system can execute.

[0111] First, the final dynamic patrol route L is sampled. The sampling interval is determined based on the UAV's flight accuracy requirements and the processing power of the flight control system. For example, route L is sampled at a certain distance d or time interval t, resulting in a series of discrete waypoints P1', P2', P3', etc. The spatial coordinates of these waypoints and their corresponding time information constitute a waypoint sequence.

[0112] For each waypoint Pi', the control instructions required to control the drone from its current position to that waypoint are calculated based on its position coordinates and the drone's current flight status (such as current position and heading angle). These control instructions include the drone's speed, steering, and altitude adjustment instructions. Assume the drone's current position is P0, its heading angle is θ0, and the waypoint to be reached is Pi', with coordinates (Xi', Yi', Zi'). By calculating the distance and direction between the two points, the speed instruction v is determined, enabling the drone to reach the waypoint within a certain timeframe. Simultaneously, based on the difference between the direction between the two points and the current heading angle, the steering instruction Δθ is determined, enabling the drone to adjust its course toward the waypoint. The altitude adjustment instruction ΔZ is determined based on the difference between the waypoint's altitude Zi' and the drone's current altitude Z0.

[0113] These control instructions for each waypoint are arranged in the order of the waypoints to form a waypoint control instruction stream I. For example, the waypoint control instruction stream I can be expressed as [(v1 instruction, Δθ1 instruction, ΔZ1 instruction), (v2 instruction, Δθ2 instruction, ΔZ2 instruction), (v3 instruction, Δθ3 instruction, ΔZ3 instruction) ...].

[0114] The waypoint control command stream I is then transmitted to the onboard controller in real time. This transmission can be achieved through a wireless communication module, such as using radio signals in a specific frequency band. Upon receiving the waypoint control command stream I, the onboard controller parses and executes these commands sequentially, driving the drone to perform its inspection mission along the final dynamic patrol route L. During the drone's mission, the onboard controller continuously compares and adjusts the actual flight status against the command requirements to ensure the drone accurately follows the planned route.

[0115] Furthermore, for the dynamic route planning model, its training steps refer to the following embodiment.

[0116] The dynamic route planning model needs to be trained before application so that it can accurately generate a reasonable set of obstacle avoidance correction vectors and environmental adaptation parameter sets based on multi-source environmental perception features.

[0117] First, a large amount of multi-source environmental perception data from different scenarios, along with the corresponding desired obstacle avoidance correction vectors and environmental adaptation parameter sets, is collected. These scenarios can include, but are not limited to, different types of areas such as warehousing and logistics parks, industrial parks, and urban areas.

[0118] The multi-source environmental perception data set, as obtained in the previous embodiment, includes synchronously acquired multispectral image sequences, 3D LiDAR point cloud data, and real-time meteorological sensor data. For example, multispectral image sequences I_a1, I_a2, etc., 3D LiDAR point cloud data P_a1, P_a2, etc., and real-time meteorological sensor data M_a1, M_a2, etc., are collected from multiple different warehousing and logistics parks. This data is organized into a specific format to form the input for the training samples.

[0119] The desired set of obstacle avoidance correction vectors and environmental adaptation parameters are determined through manual annotation or based on known reasonable route planning results. For example, for a multi-source environmental perception data set in a specific warehousing and logistics park scenario, the corresponding set of obstacle avoidance correction vectors R_a1, R_a2, etc. and environmental adaptation parameter sets E_a1, E_a2, etc. are calculated based on a pre-planned ideal route that avoids obstacles and adapts to the environment. This data is organized into the output of the training sample.

[0120] The input part and the output part are combined accordingly to form a training sample set T=[(I_a1, P_a1, M_a1, R_a1, E_a1), (I_a2, P_a2, M_a2, R_a2, E_a2)……].

[0121] The dynamic route planning model consists of an encoder, a spatiotemporal evolution prediction module, a path node offset prediction network, an environmental compensation generator, a vector fusion layer, and a fully connected decoder. Before training begins, each module of the model is initialized.

[0122] Initialize the encoder's internal parameters, such as the weight matrix in the cross-channel attention mechanism. Assuming the encoder has a weight matrix W_enc, assign it an initial value through random initialization. At the same time, perform corresponding initialization on other encoder parameters, such as the bias term.

[0123] Initialize the convolution kernel parameters and gating weight parameters of the 3D gated convolution layer in the spatiotemporal evolution prediction module. For example, the convolution kernel parameter K_conv of the 3D gated convolution layer is initialized using a specific initialization method (such as random initialization or initialization based on a certain distribution).

[0124] Modules such as the path node offset prediction network, environmental compensation generator, vector fusion layer, and fully connected decoder also initialize their internal parameters. For example, the weight matrix W_pno in the path node offset prediction network, the recurrent neural network parameters RNN_params in the environmental compensation generator, the learnable weight matrix W_vf in the vector fusion layer, and the weight matrix W_fc in the fully connected decoder are all assigned initial values using the corresponding initialization methods.

[0125] The samples in the training sample set T are subjected to feature extraction to obtain the sample multi-source environmental perception features, which are then sequentially input into the dynamic route planning model. Taking a training sample (I_ai, P_ai, M_ai, R_ai, E_ai) as an example, the multi-source environmental perception data portion (I_ai, P_ai, M_ai) is first subjected to feature extraction to obtain the sample multi-source environmental perception features and input them into the encoder.

[0126] The encoder uses a cross-channel attention mechanism to align the multi-source environmental perception features of the sample in time and space, and outputs a multi-source environmental perception tensor T ai with unified dimensions. During this process, the encoder's internal parameters, such as the weight matrix W enc, can be adjusted based on the input data to optimize the processing of heterogeneous features.

[0127] The multi-source environmental perception tensor, T_ai, is fed into the spatiotemporal evolution prediction module. A three-dimensional gated convolutional layer extracts the spatiotemporal coupling between the dynamic displacement trends of obstacles and meteorological disturbances, generating the obstacle risk probability distribution P_ai' and the meteorological interference propagation field F_ai'. During this process, parameters such as K_conv of the three-dimensional gated convolutional layer are adjusted based on the training data, ensuring that the generated obstacle risk probability distribution and meteorological interference propagation field are closer to reality.

[0128] The path node deviation prediction network generates the tangential obstacle avoidance repulsion force vector R_ai' for each path node on the reference route based on the obstacle risk probability distribution P_ai'. The environmental compensation generator generates the meteorological interference compensation parameter set C_ai' based on the meteorological interference propagation field F_ai'. During the processing of these two modules, internal parameters such as the weight matrix W_pno in the path node deviation prediction network and the recurrent neural network parameters RNN_params in the environmental compensation generator are continuously adjusted.

[0129] The obstacle avoidance repulsion force vector R_ai' and the weather interference compensation parameter set C_ai' are input into the vector fusion layer, generating the obstacle avoidance correction vector set Rv_ai', which includes three-dimensional offset components and heading angle adjustment weights. During this process, the learnable weight matrix W_vf in the vector fusion layer is learned and adjusted based on training data to better integrate obstacle avoidance and weather factors to generate a reasonable obstacle avoidance correction vector.

[0130] Based on the hidden state features of the spatiotemporal evolution prediction module, the fully connected decoder extracts the dynamic impact of visibility attenuation on the safe spacing between path nodes and generates the environmental adaptation parameter set E_ai'. The weight matrix W_fc in the fully connected decoder is also adjusted during training to ensure that the generated environmental adaptation parameter set is closer to the desired result.

[0131] The model-generated obstacle avoidance correction vector set Rv_ai' and environment adaptation parameter set E_ai' are compared with the expected results R_ai and E_ai from the training examples to calculate the loss. Loss can be calculated using various methods, such as the mean squared error (MSE) loss function. Assuming the MSE loss function is used, the loss for the obstacle avoidance correction vector set is L_Rv = Σ((Rv_ai' - R_ai)²), and the loss for the environment adaptation parameter set is L_E = Σ((E_ai' - E_ai)²). The total loss is L = L_Rv + L_E.

[0132] Based on the loss value, all model parameters are updated using the backpropagation algorithm. The backpropagation algorithm calculates the gradient of the loss value with respect to each parameter, and then adjusts the parameter value based on the gradient, gradually reducing the loss value. For example, for the weight matrix W_enc in the encoder, according to the gradient descent algorithm, the update formula can be expressed as W_enc=W_enc-η×∂L / ∂W_enc, where η is the learning rate and ∂L / ∂W_enc is the gradient of the loss value L with respect to the weight matrix W_enc. The parameters of other modules such as K_conv, W_pno, RNN_params, W_vf, W_fc, etc. are updated in the same way.

[0133] Repeat the above process, input all samples in the training sample set T into the model for training in turn, and continuously adjust the model parameters until the loss value converges to a smaller value, indicating that the model has achieved good training results.

[0134] Figure 2 The following diagram illustrates exemplary hardware and software components of a system 100 for self-adjusting patrol routes based on drone inspections, which can implement the concepts of the present application, according to some embodiments of the present application. For example, the processor 120 can be used in the system 100 for self-adjusting patrol routes based on drone inspections and perform the functions described in the present application.

[0135] The patrol route self-adjustment system 100 based on drone inspections can be a general-purpose server or a special-purpose server, both of which can be used to implement the patrol route self-adjustment method based on drone inspections of this application. Although only one server is shown in this application, for convenience, the functions described in this application can be implemented in a distributed manner on multiple similar platforms to balance the processing load.

[0136] For example, the patrol route self-adjustment system 100 based on drone inspections may include a network port 110 connected to a network, one or more processors 120 for executing program instructions, a communication bus 130, and various forms of storage media 140, such as a disk, ROM, or RAM, or any combination thereof. For example, the patrol route self-adjustment system 100 based on drone inspections may also include program instructions stored in ROM, RAM, or other types of non-transitory storage media, or any combination thereof. The method of the present application may be implemented based on these program instructions. The patrol route self-adjustment system 100 based on drone inspections may also include an I / O interface 150 between the computer and other input and output devices.

[0137] For ease of explanation, only one processor is described in the patrol route self-adjustment system 100 based on drone inspections. However, it should be noted that the patrol route self-adjustment system 100 based on drone inspections in this application may also include multiple processors, so the steps performed by one processor described in this application may also be performed jointly or individually by multiple processors. For example, if the processor of the patrol route self-adjustment system 100 based on drone inspections performs steps A and B, it should be understood that steps A and B may also be performed jointly by two different processors or individually in one processor. For example, the first processor performs step A and the second processor performs step B, or the first processor and the second processor perform steps A and B together.

[0138] In addition, an embodiment of the present invention further provides a readable storage medium, in which computer-executable instructions are preset. When a processor executes the computer-executable instructions, the above-mentioned patrol route self-adjustment method based on drone inspection is implemented.

[0139] It should be noted that in order to simplify the description of the present invention and thus help understand one or more embodiments of the invention, in the foregoing description of the embodiments of the present invention, multiple features are sometimes combined into one embodiment, figure or description thereof.

Claims

1. A patrol route self-adjustment method based on drone inspection, characterized in that: The method comprises: Acquire a multi-source environmental perception data set of a target area, wherein the multi-source environmental perception data set includes a synchronously collected multispectral image sequence, a three-dimensional lidar point cloud data, and real-time meteorological sensor data; Extracting features from the multi-source environmental perception data set to obtain multi-source environmental perception features; Inputting the multi-source environmental perception features into a dynamic route planning model to generate an obstacle avoidance correction vector set and an environmental adaptation parameter set; Performing spatial position offset compensation on the three-dimensional path nodes of the reference route based on the obstacle avoidance correction vector set to generate an initial correction path node set; Performing path smoothness optimization processing on the initial corrected path node set according to the environmental adaptation parameter set to generate a final dynamic patrol route, converting the final dynamic patrol route into a waypoint control instruction stream executable by the UAV flight control system, and transmitting the command to the onboard controller in real time to drive the UAV to perform the inspection task; The extracting features from the multi-source environment perception data set to obtain multi-source environment perception features includes: Performing terrain semantic segmentation processing on the multispectral image sequence to generate a surface cover type distribution map and an obstacle initial position feature set; Performing spatial density cluster analysis on the three-dimensional lidar point cloud data to construct a set of three-dimensional geometric models of obstacles and extracting spatial occupancy features of each obstacle; Performing wind speed gradient modeling and visibility attenuation analysis on the real-time meteorological sensor data to generate a set of meteorological interference impact coefficients; The surface cover type distribution map, the obstacle initial position feature set, the three-dimensional geometric model set, the space occupancy feature, and the meteorological interference influence coefficient set are aggregated to obtain the multi-source environment perception feature.

2. The patrol route self-adjustment method based on drone inspection according to claim 1 is characterized in that: The performing terrain semantic segmentation processing on the multispectral image sequence to generate a surface cover type distribution map and an obstacle initial position feature set includes: Calling a pre-trained semantic segmentation neural network to classify each pixel in the multispectral image to obtain a classified pixel area, wherein the classified pixel area includes a vegetation area, a water area, a building area, and a temporary obstacle area; Perform morphological closing operations on the classified pixel areas to generate continuous surface cover patches; Extracting the geometric center coordinates and boundary contour features of each surface cover patch to construct the surface cover type distribution map; The boundary contour features of the temporary obstacle area are matched with a preset obstacle feature template to screen out the significant obstacle area and generate the obstacle initial position feature set.

3. The patrol route self-adjustment method based on drone inspection according to claim 1 is characterized in that: The performing of spatial density cluster analysis on the three-dimensional lidar point cloud data, constructing a set of three-dimensional geometric models of obstacles, and extracting spatial occupancy features of each obstacle, includes: Performing voxel grid division on the three-dimensional lidar point cloud data to generate a set of voxel units with uniform spatial distribution; Performing density statistics on the point cloud data in each voxel unit in the voxel unit set, and screening voxel units with a density exceeding a preset threshold as candidate obstacle areas; Performing a connected domain analysis on the candidate obstacle area and merging spatially adjacent voxel units to form an obstacle cluster; Perform minimum circumscribed cube fitting on each obstacle cluster and calculate the center coordinates, size parameters and surface normal vector direction of the minimum circumscribed cube; The three-dimensional geometric model set is generated based on the center coordinates and size parameters, and the surface normal vector direction of each cube is extracted as the space occupancy feature.

4. The patrol route self-adjustment method based on drone inspection according to claim 1 is characterized in that: The performing of wind speed gradient modeling and visibility attenuation analysis on the real-time meteorological sensor data to generate a set of meteorological interference influence coefficients includes: Performing time series sampling on the real-time meteorological sensor data to obtain wind speed vector and visibility measurement values at the current moment; Build a wind speed change rate prediction model based on historical wind speed data and calculate the wind speed gradient change curve within the next N sampling periods; Generating a visibility compensation parameter for visibility to lidar detection accuracy based on the visibility measurement value and a preset attenuation coefficient model; Decomposing the wind speed gradient change curve into a horizontal offset coefficient and a vertical turbulence intensity coefficient; The horizontal direction offset coefficient, the vertical direction turbulence intensity coefficient and the visibility compensation parameter are integrated to generate the meteorological interference influence coefficient set.

5. The patrol route self-adjustment method based on drone inspection according to any one of claims 1 to 4, characterized in that: Inputting the multi-source environmental perception features into a dynamic route planning model to generate an obstacle avoidance correction vector set and an environmental adaptation parameter set includes: Inputting the multi-source environmental perception features into the encoder of the dynamic route planning model, performing spatiotemporal alignment of heterogeneous features through a cross-channel attention mechanism, and generating a multi-source feature tensor with unified dimension; Inputting the multi-source feature tensor into the spatiotemporal evolution prediction module of the dynamic route planning model, using a three-dimensional gated convolutional layer to extract the spatiotemporal coupling relationship between the dynamic displacement trend of obstacles and meteorological disturbances, and generating an obstacle risk probability distribution and a meteorological interference propagation field; Performing a three-dimensional spatial gradient inversion on the obstacle risk probability distribution through the path node offset prediction network of the dynamic route planning model to generate an obstacle avoidance repulsive force vector in the tangential direction for each path node of the reference route; Inputting the meteorological interference propagation field into the environmental compensation generator of the dynamic route planning model, extracting the horizontal wind speed compensation gradient and the vertical turbulence suppression coefficient through recurrent neural network sequence modeling, and generating a meteorological interference compensation parameter set; Inputting the obstacle avoidance repulsion force vector and the set of weather interference compensation parameters into the vector fusion layer of the dynamic route planning model, performing direction vector synthesis and modulus normalization using a learnable weight matrix, and generating an obstacle avoidance correction vector set including a three-dimensional offset component and a heading angle adjustment weight; Based on the hidden state characteristics of the spatiotemporal evolution prediction module, the dynamic impact factor of visibility attenuation on the safe distance between path nodes is extracted through the fully connected decoder of the dynamic route planning model, and an environmental adaptation parameter set that matches the dimension of the obstacle avoidance correction vector set is generated.

6. The patrol route self-adjustment method based on drone inspection according to claim 5 is characterized in that: The obstacle avoidance repulsion force vector and the weather interference compensation parameter set are input into the vector fusion layer of the dynamic route planning model, and a learnable weight matrix is used to perform direction vector synthesis and modulus normalization to generate an obstacle avoidance correction vector set including a three-dimensional offset component and a heading angle adjustment weight, including: Performing unit vector processing on the obstacle avoidance repulsive force vector in a three-dimensional space coordinate system, and extracting the obstacle avoidance direction reference vector and its modulus scaling coefficient corresponding to each path node; Decomposing the horizontal wind speed compensation gradient in the meteorological interference compensation parameter set into a lateral wind pressure offset vector and a longitudinal airflow offset vector, and mapping the vertical turbulence suppression coefficient into a vertical fluctuation suppression vector; Performing three-dimensional space vector superposition of the lateral wind pressure offset vector, the longitudinal airflow offset vector, and the vertical fluctuation suppression vector to generate a meteorological interference compensation vector; According to the maximum flight speed and maneuvering acceleration threshold set by the UAV flight control system, the modulus of the weather interference compensation vector is dynamically constrained and truncated to generate a normalized weather compensation vector that meets the dynamic performance of the UAV; Inputting the obstacle avoidance direction reference vector and the normalized weather compensation vector into the learnable weight matrix, and generating a three-dimensional space offset synthetic vector through matrix multiplication and addition operations; Using a dynamic attenuation factor to map the three-dimensional space offset synthetic vector to a heading angle offset, and extracting a heading angle adjustment weight parameter; Dynamically scaling the three-dimensional spatial offset synthesis vector according to the modulus scaling coefficient to generate a three-dimensional offset component; The three-dimensional offset component is spliced with the heading angle adjustment weight parameter to generate a correction vector corresponding to each path node in the obstacle avoidance correction vector set.

7. The patrol route self-adjustment method based on drone inspection according to claim 5 is characterized in that: The performing spatial position offset compensation on the three-dimensional path nodes of the reference route based on the obstacle avoidance correction vector set to generate an initial correction path node set includes: Extracting a set of spatial coordinates of each three-dimensional path node in the reference route in the original geographic coordinate system; performing a three-dimensional Euclidean space vector superposition operation on the spatial coordinate set according to the three-dimensional offset components corresponding to each path node in the obstacle avoidance correction vector set to generate an adjusted three-dimensional spatial coordinate set; Based on the heading angle adjustment weight in the obstacle avoidance correction vector set, the adjusted three-dimensional space coordinate set is compensated for the heading angle deflection of the drone to generate the path node coordinates after the heading angle is corrected; Using the boundary contour features in the multi-source environment perception features, collision detection is performed on the coordinates of the path nodes after the heading angle correction, and abnormal path nodes that have spatial overlap with the obstacle three-dimensional geometric model set are eliminated; The airspace connectivity is verified based on the coordinates of the eliminated path nodes, and an initial corrected path node set that meets the UAV flight safety distance constraints is generated.

8. The patrol route self-adjustment method based on drone inspection according to claim 5 is characterized in that: The step of performing path smoothness optimization processing on the initial corrected path node set according to the environmental adaptation parameter set to generate a final dynamic patrol route includes: Calculate the dynamic safety buffer distance threshold between adjacent nodes in the initial correction path node set based on the dynamic influencing factors in the environmental adaptation parameter set, combined with the current flight speed, braking response time and visibility attenuation parameters of the UAV; Based on the dynamic safety buffer distance threshold, the node spacing of the initial corrected path node set is uniformly adjusted to generate an intermediate interpolation node sequence under an equidistance constraint; Using a cubic B-spline curve to perform spatial trajectory fitting on the intermediate interpolation node sequence to generate a continuous and smooth three-dimensional track curve; Based on the horizontal direction deviation coefficient in the multi-source environmental perception characteristics, the horizontal plane anti-wind pressure curvature of the three-dimensional track curve is optimized to generate an anti-interference smooth track; According to the maximum steering angle, minimum turning radius and maximum centripetal acceleration parameters of the UAV flight control system, the anti-disturbance smooth trajectory is subjected to segmented heading angle continuity verification and curvature smoothing optimization to generate the final dynamic patrol route that satisfies both kinematic and dynamic constraints.

9. A patrol route self-adjustment system based on drone inspection, characterized in that: It includes a processor and a memory, the memory is connected to the processor, the memory is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the memory to implement the patrol route self-adjustment method based on drone inspection as described in any one of claims 1 to 8.

Images