Patrol route self-adjusting method and system based on unmanned aerial vehicle inspection

Abstract

The invention provides a patrol route self-adjusting method and system based on unmanned aerial vehicle patrol, and the method comprises the steps: firstly obtaining a multi-source environment perception data set of a target area, then carrying out the feature extraction of the multi-source environment perception data set, inputting a dynamic route planning model, generating an obstacle avoidance correction vector set and an environment adaptation parameter set, and carrying out the feature extraction of the multi-source environment perception data set; and carrying out spatial position offset compensation on the reference route based on the obstacle avoidance correction vector set to generate an initial correction path node set, carrying out path smoothness optimization processing on the initial correction path node set according to the environment adaptation parameter set to generate a final dynamic patrol route, and finally, carrying out dynamic patrol. The dynamic patrol route is converted into a waypoint control instruction stream which can be executed by the unmanned aerial vehicle flight control system, and the waypoint control instruction stream is transmitted to the airborne controller in real time to drive the unmanned aerial vehicle to execute the patrol task, so that dynamic self-adjustment of the unmanned aerial vehicle patrol route is realized, and the patrol flexibility and safety are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle (UAV) control, and more particularly, to a method and system for self-adjusting a patrol route based on UAV patrol inspection. Background Art

[0002] In the field of UAV patrol inspection, with the rapid development of UAV technology, it has been widely used in many fields such as power inspection, agricultural monitoring, and environmental monitoring. However, traditional UAV patrol route planning methods often rely on pre-set fixed routes, which are ineffective in the face of complex and changing natural environments. For example, in mountainous areas, forests, or densely built-up urban areas, fixed routes may not be able to effectively avoid obstacles, increasing the risk of UAV collisions; at the same time, meteorological conditions (such as wind speed, wind direction, visibility, etc.) at different times may also have a significant impact on the flight safety and inspection effect of UAVs. Therefore, how to dynamically adjust the UAV patrol route according to real-time environmental perception data to cope with changes in complex environments and meteorological conditions has become an urgent problem to be solved. Summary of the Invention

[0003] In view of the above-mentioned problems, in combination with the first aspect of the present invention, embodiments of the present invention provide a method for self-adjusting a patrol route based on UAV patrol inspection, the method comprising: Obtaining a multi-source environmental perception data set of a target area, the multi-source environmental perception data set including a multi-spectral image sequence, three-dimensional lidar point cloud data, and real-time meteorological sensor data collected synchronously; Performing feature extraction on the multi-source environmental perception data set to obtain multi-source environmental perception features; Inputting the multi-source environmental perception features into a dynamic route planning model to generate an obstacle avoidance correction vector set and an environment adaptation parameter set; Performing spatial position offset compensation on three-dimensional path nodes of a reference route based on the obstacle avoidance correction vector set to generate an initial corrected path node set; Performing path smoothness optimization processing on the initial corrected path node set according to the environment adaptation parameter set to generate a final dynamic patrol route, and converting the final dynamic patrol route into a waypoint control instruction stream executable by a UAV flight control system, and transmitting it to an on-board controller in real time to drive the UAV to perform a patrol inspection task.

[0004] In another aspect, embodiments of the present invention further provide a system for self-adjusting a patrol route based on UAV patrol inspection, including a processor and a machine-readable storage medium, the machine-readable storage medium being connected to the processor, the machine-readable storage medium being used to store programs, instructions, or codes, and the processor being used to execute the programs, instructions, or codes in the machine-readable storage medium to implement the above method.

[0005] Based on the above aspects, by acquiring and integrating multi-source environmental perception data, the environmental state of the target area can be comprehensively and accurately perceived. Through feature extraction and dynamic flight route planning models, an obstacle avoidance correction vector set and an environmental adaptation parameter set are generated, realizing the precise correction and optimization of the reference flight route. Based on the spatial position offset compensation of the obstacle avoidance correction vector set, obstacles are effectively avoided, improving the flight safety of the UAV. And according to the path smoothness optimization process of the environmental adaptation parameter set, the rationality and flight efficiency of the patrol flight route are further improved. Finally, the dynamic patrol flight route is converted into a waypoint control instruction stream executable by the UAV flight control system and transmitted to the on-board controller in real time, realizing the automatic and intelligent execution of the UAV inspection task, significantly improving the flexibility and adaptability of the UAV inspection, reducing the flight risk, and enhancing the inspection effect. Description of the Drawings

[0006] Figure 1 is a schematic execution flowchart of a patrol flight route self-adjustment method based on UAV inspection provided by an embodiment of the present invention.

[0007] Figure 2 is a schematic diagram of exemplary hardware and software components of a patrol flight route self-adjustment system based on UAV inspection provided by an embodiment of the present invention. Detailed Embodiments

[0008] The present invention will be specifically described below in conjunction with the accompanying drawings of the specification. Figure 1 is a schematic flowchart of a patrol flight route self-adjustment method based on UAV inspection provided by an embodiment of the present invention. The patrol flight route self-adjustment method based on UAV inspection will be introduced in detail below.

[0009] Step S110: Obtain a multi-source environmental perception data set of the target area, where the multi-source environmental perception data set includes a multi-spectral image sequence, three-dimensional lidar point cloud data, and real-time meteorological sensor data collected synchronously.

[0010] In the scenario of this embodiment, it is set as a large-scale warehousing and logistics park, and the UAV needs to inspect the park. In order to enable the UAV to plan the patrol flight route smoothly and safely, it is first necessary to obtain comprehensive and synchronous multi-source environmental perception data of this area.

[0011] For the multi-spectral image sequence, it is obtained by a specific multi-spectral image acquisition device installed on the UAV. Assume that the acquisition device takes n1 multi-spectral images per second, forming an image sequence I1, I2, I3... These images cover the entire range of the warehousing and logistics park, and the image information in different bands can reflect the characteristics of different ground objects. For example, certain bands are sensitive to metal materials, and some have a better presentation of the texture of vegetation or buildings.

[0012] The 3D lidar point cloud data is collected by a 3D lidar device mounted on a drone. The lidar emits laser beams into the surrounding space at a certain frequency and receives the reflected signals. Based on information such as the time and angle of the reflected signals, the position information of the points on the surface of objects in space is determined, thereby generating point cloud data. Let the point cloud data generated by each scan be P1, P2, P3... These point cloud data accurately depict the three-dimensional spatial positions and shapes of buildings, cargo stacks, vehicles, etc. in the warehousing and logistics park.

[0013] The real-time meteorological sensor data is obtained by meteorological sensors installed on the drone. The meteorological sensors continuously monitor the meteorological parameters in the park, such as wind speed, wind direction, visibility, etc. Suppose the meteorological data is recorded every t1 time interval, forming a meteorological data sequence M1, M2, M3... These data can reflect the changes in meteorological conditions in the park at different times. Through an accurate synchronization mechanism, it is ensured that the multi-spectral image sequence, 3D lidar point cloud data, and real-time meteorological sensor data are completely synchronized in terms of acquisition time, thus forming a multi-source environmental perception data set for the target area.

[0014] Step S120: Extract features from the multi-source environmental perception data set to obtain multi-source environmental perception features.

[0015] After obtaining the multi-source environmental perception data set of the warehousing and logistics park, in order to effectively utilize these data for the dynamic planning of the patrol route, key features need to be extracted from it. This process involves performing specialized feature extraction operations on the multi-spectral image sequence, 3D lidar point cloud data, and real-time meteorological sensor data respectively, and then integrating the extracted various features.

[0016] Step S121: Perform terrain semantic segmentation processing on the multi-spectral image sequence to generate a surface cover type distribution map and an initial obstacle position feature set.

[0017] For the multi-spectral image sequence of the warehousing and logistics park, a pre-trained semantic segmentation neural network is called to classify the ground objects for each pixel in the multi-spectral image. This pre-trained semantic segmentation neural network uses a large amount of multi-spectral image data from different scenarios (including but not limited to warehousing and logistics parks, urban areas, industrial plants, etc.) during the training phase to learn the feature patterns of different ground objects in the multi-spectral image.

[0018] Taking one of the multi - spectral images I as an example, the neural network classifies each pixel into different categories based on its multi - band spectral information, forming classified pixel regions, which include vegetation regions, water regions, building regions, and temporary obstacle regions. For example, for the pixels in a certain area of image I, according to their reflectance and other characteristics in multiple bands such as the near - infrared band and the visible light band, it is determined that this area is a building region.

[0019] Perform morphological closing operation on the classified pixel regions. Morphological closing operation includes first performing a dilation operation and then an erosion operation. The dilation operation expands each pixel region towards the surrounding, making it possible for adjacent homogeneous pixel regions to be connected. Suppose for a certain pixel region A, the dilation operation takes the boundary pixels of this region as the center, searches for homogeneous pixels within a certain neighborhood range, and merges them into this region. The subsequent erosion operation then shrinks the region, removing some noise or isolated pixel points introduced by dilation. Through this closing operation, continuous land - cover patches are generated. For example, some originally scattered pixel regions representing small buildings form a larger and continuous building land - cover patch after the closing operation.

[0020] Extract the geometric center coordinates and boundary contour features of each land - cover patch. For each continuous land - cover patch, determine its geometric center coordinates through a certain geometric calculation method. Suppose the patch is of an irregular shape, and the geometric center coordinates are approximately obtained by calculating the average value of the coordinates of all pixel points within the patch. At the same time, extract the boundary contour features of the patch, which can be determined by an edge - detection algorithm, such as the Canny edge - detection algorithm, to identify the edge pixels of the patch and thus obtain the boundary contour features. Using these geometric center coordinates and boundary contour features, construct a land - cover type distribution map. For example, record the geometric center coordinates and boundary contour features of different types of land - cover patches in a certain format to form a data structure as the land - cover type distribution map.

[0021] Match the boundary contour features of the temporary obstacle region with a preset obstacle feature template. The preset obstacle feature template is pre - constructed according to the characteristics of common obstacle shapes, sizes, etc. For example, for a common cargo - pile obstacle, construct its approximate shape, size range, etc. as the template. By comparing the similarity between the boundary contour features of the temporary obstacle region and the template features, filter out the significant obstacle regions. The calculation of similarity can be based on various methods, such as calculating the shape similarity of the contour, size matching degree, etc. For regions with similarity exceeding a certain threshold, determine them as significant obstacle regions and generate an initial obstacle position feature set. This set records the position, approximate shape, etc. of the significant obstacle regions.

[0022] Step S122: Conduct spatial density clustering analysis on the 3D lidar point cloud data, construct a set of 3D geometric models of obstacles, and extract the spatial occupancy characteristics of each obstacle.

[0023] In the scenario of a warehousing and logistics park, for the obtained 3D lidar point cloud data, first perform voxel grid division on it. Voxel grid division is to divide the entire 3D space into many voxel units of uniform size. Assume that the 3D space where the warehousing and logistics park is located is divided into voxel units, and the size of each voxel unit is v1×v2×v3 (where v1, v2, and v3 respectively represent the dimensions of the voxel unit in the three coordinate axis directions), generating a set of voxel units with a uniform spatial distribution.

[0024] Perform density statistics on the point cloud data within each voxel unit in the set of voxel units. Calculate the number of point cloud data points within each voxel unit, and use this as the density of the voxel unit. Screen out the voxel units with a density exceeding the preset threshold T1 as candidate obstacle regions. For example, if the number of point cloud data points within a voxel unit is greater than T1, it is considered that the voxel unit may contain an obstacle.

[0025] Conduct connected component analysis on the candidate obstacle regions. Connected component analysis is to find voxel units that are adjacent in space and have densities exceeding the threshold, and merge them into a whole. For example, for two adjacent voxel units, if they both meet the condition that the density exceeds the threshold T1, they are merged into an obstacle clustering cluster. In this way, merge the spatially adjacent voxel units to form obstacle clustering clusters.

[0026] Perform minimum bounding cube fitting on each obstacle clustering cluster. For each obstacle clustering cluster, use a certain algorithm to find a minimum cube such that all the point cloud data within the clustering cluster is contained within the cube. Calculate the center coordinates, size parameters, and surface normal vector directions of the minimum bounding cube. The center coordinates can be obtained by calculating the average value of the coordinates of all point cloud data points within the clustering cluster. The size parameters are determined based on the difference between the maximum length and the minimum length of the cube in the three coordinate axis directions. The surface normal vector direction can be obtained through geometric analysis of the cube surface.

[0027] Generate a set of 3D geometric models based on the center coordinates and size parameters. This set of 3D geometric models records the approximate 3D shape and position of each obstacle. At the same time, extract the surface normal vector directions of each cube as the spatial occupancy characteristics, which can be used for subsequent analysis of the direction characteristics of the obstacles in space and the relative position relationship with other objects.

[0028] Step S123: Conduct wind speed gradient modeling and visibility attenuation analysis on the real-time meteorological sensor data, and generate a set of meteorological interference influence coefficients.

[0029] In a warehousing and logistics park, real-time meteorological sensor data is processed. First, time series sampling is performed on the real-time meteorological sensor data. Assuming that sampling is carried out every t2 time interval, the wind speed vector and visibility measurement value at the current moment are obtained. The wind speed vector includes the wind speed magnitude and wind direction information. Assuming that the wind speed vector obtained from a certain sampling is W (including the wind speed magnitude w1 and the wind direction angle θ1), and the visibility measurement value is V1.

[0030] Based on historical wind speed data, a wind speed change rate prediction model is constructed. Wind speed data over a past period is collected. Assuming that the wind speed data sequence W1, W2... Wn2 from time t - n2 to time t is collected, and through data analysis methods, such as using the autoregressive moving average model (ARIMA) in time series analysis, etc., a wind speed change rate prediction model is constructed. Using this wind speed change rate prediction model, the wind speed gradient change curve within the next N sampling periods is calculated. For example, based on the change trend of the historical wind speed data, the wind speed change amount for each period within the next N sampling periods is predicted by the wind speed change rate prediction model, thus forming the wind speed gradient change curve.

[0031] According to the visibility measurement value V1 and the preset attenuation coefficient model, the visibility compensation parameter for the lidar detection accuracy due to visibility is generated. The preset attenuation coefficient model is established based on experimental data and theoretical analysis, which describes the relationship between visibility and lidar detection accuracy. For example, according to the visibility measurement value V1, by looking up the corresponding relationship table or calculation formula of the attenuation coefficient model, the visibility compensation parameter C1 is obtained, and this parameter is used to compensate for the impact of visibility changes on lidar detection accuracy.

[0032] The wind speed gradient change curve is decomposed into a horizontal direction offset coefficient and a vertical direction turbulence intensity coefficient. Through vector decomposition of the wind speed gradient change curve in the horizontal and vertical directions, assuming that the wind speed gradient change curve can be represented as a vector sequence, its component in the horizontal direction is used as the horizontal direction offset coefficient H1, and the component in the vertical direction is used as the vertical direction turbulence intensity coefficient Vt1.

[0033] Integrate the horizontal direction offset coefficient H1, the vertical direction turbulence intensity coefficient Vt1, and the visibility compensation parameter C1 to generate a set of meteorological interference influence coefficients. This set contains the correlation coefficients of the meteorological conditions that may interfere with the flight of the unmanned aerial vehicle and the lidar detection, and is used for subsequent route planning adjustment.

[0034] Step S124: Converge the surface cover type distribution map, the set of initial obstacle position characteristics, the set of three-dimensional geometric models, the spatial occupancy characteristics, and the set of meteorological interference influence coefficients to obtain the multi-source environmental perception characteristics.

[0035] In the scenario of a warehousing and logistics park, various features previously extracted from multi-spectral image sequences, 3D lidar point cloud data, and real-time meteorological sensor data are aggregated. The surface cover type distribution map, which records the distribution information of different surface cover types within the park, is represented by the data structure D1; the initial obstacle position feature set, which records features such as the initial positions and approximate shapes of obstacles identified from multi-spectral images, is represented by the set S1; the 3D geometric model set, which is the 3D geometric models of obstacles constructed from 3D lidar point cloud data, is represented by the set G1; the space occupancy feature, which includes features such as the surface normal vector directions corresponding to each 3D geometric model of an obstacle, is represented by the set O1; the meteorological interference influence coefficient set, which contains the influence coefficients of meteorological factors such as wind speed and visibility on the flight of unmanned aerial vehicles and lidar detection, is represented by the set M1. By combining these data structures and sets according to certain rules, a unified data structure or set is formed as the multi-source environmental perception feature, denoted as the feature set F1, thus completing the feature extraction work of multi-source environmental perception data and providing comprehensive environmental feature information for subsequent dynamic route planning.

[0036] Step S130: Input the multi-source environmental perception feature into the dynamic route planning model to generate an obstacle avoidance correction vector set and an environmental adaptation parameter set.

[0037] In the scenario of a warehousing and logistics park, the multi-source environmental perception feature set F1 obtained previously is input into the dynamic route planning model. This dynamic route planning model consists of multiple modules such as an encoder, a spatio-temporal evolution prediction module, a path node offset prediction network, an environmental compensation generator, a vector fusion layer, and a fully connected decoder. These modules cooperate with each other to generate a reasonable obstacle avoidance correction vector set and an environmental adaptation parameter set according to environmental features.

[0038] Step S131: Input the multi-source environmental perception feature into the encoder of the dynamic route planning model, and perform spatio-temporal alignment on heterogeneous features through a cross-channel attention mechanism to generate a multi-source environmental perception tensor with unified dimensions.

[0039] The multi-source environmental perception feature set F1 contains heterogeneous features from different data sources, such as the surface cover type distribution map and the initial obstacle position feature set. It is input into the encoder of the dynamic route planning model. The role of the encoder is to process these heterogeneous features so that they can work together in subsequent model processing.

[0040] The cross-channel attention mechanism re-weights and combines heterogeneous features by learning the importance weights between different feature channels in the encoder, thus achieving spatio-temporal alignment. For different features in the multi-source environmental perception feature set F1, such as the land cover type distribution map D1 and the obstacle initial position feature set S1, the cross-channel attention mechanism analyzes their correlations in different time and space dimensions. Assuming the feature data at a certain moment t, the attention mechanism calculates the importance weights W1, W2... of each feature element in D1 and S1 on different channels. The feature elements are weighted by these weights so that different features can be aligned spatio-temporally. After such processing, a multi-source environmental perception tensor T1 with a unified dimension is generated. This multi-source environmental perception tensor has a unified dimension and format in terms of data structure, facilitating subsequent processing by model modules.

[0041] Step S132: Input the multi-source environmental perception tensor into the spatio-temporal evolution prediction module of the dynamic route planning model, and use a three-dimensional gated convolutional layer to extract the spatio-temporal coupling relationship between the dynamic displacement trend of obstacles and meteorological disturbances, generating an obstacle risk probability distribution and a meteorological interference propagation field.

[0042] Input the multi-source environmental perception tensor T1 with a unified dimension into the spatio-temporal evolution prediction module. This module aims to analyze the variation relationship of environmental features over time and space to predict the impact of future environmental conditions on the route.

[0043] The three-dimensional gated convolutional layer plays a core role in the spatio-temporal evolution prediction module. It performs a convolutional operation on the multi-source environmental perception tensor T1 in a three-dimensional space (including two spatial dimensions and one time dimension), and at the same time introduces a gating mechanism to control the flow of information. For each position (x, y, t) in the multi-source environmental perception tensor T1 (where x and y represent spatial positions and t represents time), the three-dimensional gated convolutional layer extracts the spatio-temporal coupling relationship between the dynamic displacement trend of obstacles and meteorological disturbances based on the feature information of surrounding positions and the gating signal.

[0044] For example, for obstacles in a warehousing and logistics park, combining the changes in the positions of obstacles reflected by multi-spectral images and lidar point cloud data at different times, as well as the meteorological data at the same time, the three-dimensional gated convolutional layer can learn how the obstacles may have dynamic displacements as the meteorological conditions change. Through such processing, an obstacle risk probability distribution P1 is generated, which represents the probability of obstacles causing risks to the flight of drones at different spatial positions and time points. At the same time, a meteorological interference propagation field F2 is generated, which describes the propagation of meteorological interferences (such as changes in wind speed and direction) in space and the degree of influence on different regions.

[0045] Step S133: Perform three-dimensional spatial gradient inversion on the obstacle risk probability distribution through the path node offset prediction network of the dynamic route planning model to generate the obstacle avoidance repulsive force vectors of each path node of the reference route in the tangential direction.

[0046] After obtaining the obstacle risk probability distribution P1, input it into the path node offset prediction network. The task of this network is to determine how each path node on the reference route needs to be offset to avoid obstacles according to the obstacle risk probability distribution.

[0047] Three-dimensional spatial gradient inversion is a key operation in the path node offset prediction network. For the obstacle risk probability distribution P1, it has different probability value distributions in three-dimensional space. By calculating the gradient of the probability values in three-dimensional space, the direction and magnitude of the probability change at each position are obtained. For example, for a certain path node N1 on the reference route, calculate the gradient of the obstacle risk probability distribution P1 in the three-dimensional space region around it. If the probability increases rapidly in a certain direction, it indicates that the risk of approaching the obstacle in that direction increases.

[0048] According to the gradient calculation results, generate the obstacle avoidance repulsive force vectors of each path node of the reference route in the tangential direction. The obstacle avoidance repulsive force vector represents the direction and magnitude of the repulsive force that the path node should receive in the tangential direction to avoid obstacles. For example, for path node N1, the generated obstacle avoidance repulsive force vector R1, its direction points to the direction away from the high-risk obstacle area, and its magnitude is determined according to the gradient magnitude and preset relevant parameters (such as the safety distance coefficient, etc.). These obstacle avoidance repulsive force vectors will be used for subsequent position adjustment of the path nodes of the reference route.

[0049] Step S134: Input the meteorological interference propagation field into the environment compensation generator of the dynamic route planning model, extract the horizontal wind speed compensation gradient and vertical turbulence suppression coefficient through recurrent neural network sequence modeling, and generate a set of meteorological interference compensation parameters.

[0050] Input the meteorological interference propagation field F2 into the environment compensation generator. The role of the environment compensation generator is to generate parameters that can compensate for the impact of meteorological interference on the flight of the UAV according to the information of the meteorological interference propagation field.

[0051] Recurrent neural network sequence modeling plays an important role in the environment compensation generator. It analyzes and models the time series information in the meteorological interference propagation field F2. The meteorological interference propagation field F2 contains meteorological interference information at different times and different spatial positions, such as the changes in wind speed, wind direction, etc. over time.

[0052] Through the processing of the recurrent neural network, the horizontal wind speed compensation gradient H2 and the vertical turbulence suppression coefficient Vt2 are extracted. For the horizontal wind speed compensation gradient H2, the recurrent neural network predicts the gradient values for compensating the flight path of the unmanned aerial vehicle at different positions and times according to the change trend of the horizontal wind speed in the meteorological interference propagation field F2, in order to offset the influence of the horizontal wind speed on the flight path of the unmanned aerial vehicle. The vertical turbulence suppression coefficient Vt2 determines how to suppress the influence of vertical turbulence on the flight stability of the unmanned aerial vehicle according to the intensity and change of turbulence in the vertical direction.

[0053] Integrate the horizontal wind speed compensation gradient H2 and the vertical turbulence suppression coefficient Vt2 together to generate the meteorological interference compensation parameter set C2. The parameters in this set will be used to adjust the flight path subsequently to adapt to the change of meteorological conditions and ensure the flight stability and safety of the unmanned aerial vehicle.

[0054] Step S135: Input the obstacle avoidance repulsive force vector and the meteorological interference compensation parameter set into the vector fusion layer of the dynamic flight path planning model, and use the learnable weight matrix for direction vector synthesis and norm normalization to generate an obstacle avoidance correction vector set containing three-dimensional offset components and heading angle adjustment weights.

[0055] Input the obstacle avoidance repulsive force vector (such as R1, etc.) and the meteorological interference compensation parameter set C2 into the vector fusion layer. The purpose of the vector fusion layer is to comprehensively consider the obstacle avoidance requirements and the influence of meteorological conditions on the flight path and generate a unified obstacle avoidance correction vector set.

[0056] First, perform unit vectorization processing on the obstacle avoidance repulsive force vector in the three-dimensional space coordinate system. For each obstacle avoidance repulsive force vector, such as R1, convert it into a unit vector to obtain the obstacle avoidance direction reference vector U1, and extract its modulus scaling coefficient k1. The unit vectorization processing enables all obstacle avoidance repulsive force vectors to have a unified direction representation, facilitating subsequent synthesis operations with other vectors.

[0057] Decompose the horizontal wind speed compensation gradient H2 in the meteorological interference compensation parameter set C2 into a lateral wind pressure offset vector Hh1 and a longitudinal air flow offset vector Hv1, and at the same time map the vertical turbulence suppression coefficient Vt2 to a vertical fluctuation suppression vector Vv1. This is a vector decomposition and mapping operation based on the influence of horizontal wind speed and vertical turbulence on the flight of the unmanned aerial vehicle in different directions.

[0058] Perform three-dimensional space vector superposition on the lateral wind pressure offset vector Hh1, the longitudinal air flow offset vector Hv1, and the vertical fluctuation suppression vector Vv1 to generate a meteorological interference compensation vector Mv1. This meteorological interference compensation vector comprehensively considers the influence of meteorological conditions on the flight of the unmanned aerial vehicle in different directions.

[0059] According to the relevant performance parameters set by the UAV flight control system, dynamic constraint truncation processing is performed on the magnitude of the meteorological interference compensation vector Mv1. The UAV flight control system will stipulate some parameters related to flight performance, such as the maximum flight speed, the threshold of maneuvering acceleration, etc. Assume that the maximum allowable magnitude of the meteorological interference compensation vector corresponding to the maximum flight speed stipulated by the flight control system is Lmax, and the maximum allowable change rate of the magnitude of the meteorological interference compensation vector corresponding to the maneuvering acceleration threshold is Kmax. By comparing the relationship between the magnitude |Mv1| of the meteorological interference compensation vector Mv1 with Lmax and the change rate of the magnitude with Kmax, if |Mv1| exceeds Lmax, or the change rate of the magnitude exceeds Kmax, then truncation processing is performed on the meteorological interference compensation vector Mv1 to make its magnitude and the change rate of the magnitude meet the requirements of the UAV dynamic performance, and a normalized meteorological compensation vector Mn1 that conforms to the UAV dynamic performance is generated.

[0060] The obstacle avoidance direction reference vector U1 and the normalized meteorological compensation vector Mn1 are input into the learnable weight matrix W. The learnable weight matrix W is obtained by learning a large amount of training data during the model training process. Its role is to reasonably adjust the weights of the obstacle avoidance and meteorological compensation factors in the final correction vector according to different environmental conditions and flight requirements. Through matrix multiplication and addition operations, that is, each component of the obstacle avoidance direction reference vector U1 and the corresponding components of the normalized meteorological compensation vector Mn1 are multiplied by the corresponding weight elements in the learnable weight matrix W respectively, and then the product results are added to generate a three-dimensional space offset synthesis vector Sv1.

[0061] The heading angle offset mapping is performed on the three-dimensional space offset synthesis vector Sv1 using the dynamic attenuation factor D. The dynamic attenuation factor D is dynamically determined according to factors such as the current flight state of the UAV and environmental conditions. For example, when the UAV approaches an obstacle or the meteorological conditions are relatively bad, the value of the dynamic attenuation factor D will be adjusted accordingly to more accurately reflect the influence of the environment on the heading angle. By multiplying the three-dimensional space offset synthesis vector Sv1 by the dynamic attenuation factor D, the heading angle adjustment weight parameter α1 is extracted.

[0062] The dynamic proportional scaling is performed on the three-dimensional space offset synthesis vector Sv1 according to the magnitude scaling coefficient k1. The magnitude scaling coefficient k1 is extracted during the unit vectorization process of the obstacle avoidance repulsive force vector, and it reflects the relative size of the original obstacle avoidance repulsive force vector. By multiplying each component of the three-dimensional space offset synthesis vector Sv1 by the magnitude scaling coefficient k1 respectively, a three-dimensional offset component δ1 is generated.

[0063] The three-dimensional offset component δ1 is concatenated with the heading angle adjustment weight parameter α1 to generate the correction vector Rv1 corresponding to each path node in the obstacle avoidance correction vector set. Through such processing, the impacts of obstacle avoidance and meteorological factors on the route are comprehensively considered, and an obstacle avoidance correction vector set that contains both spatial position offset information and heading angle adjustment information is generated, providing a basis for subsequent adjustment of the reference route.

[0064] Step S136: Based on the hidden state features of the spatio-temporal evolution prediction module, the fully connected decoder of the dynamic route planning model extracts the dynamic influence factor of visibility attenuation on the safety distance between path nodes, and generates an environment adaptation parameter set that matches the dimension of the obstacle avoidance correction vector set.

[0065] During the process of the spatio-temporal evolution prediction module processing the multi-source environment perception tensor T1, the hidden state feature Hid1 will be generated. The hidden state feature Hid1 contains rich information about the changes of environmental features over time and space, including not only the spatio-temporal coupling relationship between the dynamic displacement trend of obstacles and meteorological disturbances, but also some other potential information that affects route planning.

[0066] The hidden state feature Hid1 is input into the fully connected decoder of the dynamic route planning model. The role of the fully connected decoder is to extract the information related to the safety distance between path nodes from the hidden state feature, especially the impact of visibility attenuation on the safety distance between path nodes.

[0067] The fully connected decoder realizes information extraction through a series of neuron connections and weight adjustment operations. Assume that the fully connected decoder consists of multiple layers of neurons, and each layer of neurons is connected by a weight matrix. The hidden state feature Hid1 is first input into the first layer of neurons. After weighted summation and activation function processing, the intermediate feature representation I1 is obtained. The intermediate feature representation I1 is then successively input into the subsequent layers of neurons. After layer-by-layer processing, the dynamic influence factor β1 of visibility attenuation on the safety distance between path nodes is finally output.

[0068] The calculation process of the dynamic influence factor β1 involves the comprehensive analysis of different elements in the hidden state feature Hid1. For example, the hidden state feature Hid1 may contain visibility information, obstacle position information, and UAV flight state information at different times, etc. The fully connected decoder will determine how visibility attenuation affects the safety distance between path nodes according to the complex relationships between these information through weight adjustment and neuron processing.

[0069] Generate an environmental adaptation parameter set E1 that matches the dimension of the obstacle avoidance correction vector set. The obstacle avoidance correction vector set is a vector set for adjusting the position and heading angle of the reference route to avoid obstacles and adapt to meteorological conditions, while the environmental adaptation parameter set E1 is to ensure that the route meets the safety distance requirements under different environmental conditions. By making the environmental adaptation parameter set E1 match the dimension of the obstacle avoidance correction vector set, it is ensured that when adjusting the route subsequently, multiple factors such as obstacle avoidance, meteorological adaptation, and safety distance can be comprehensively considered to generate a reasonable dynamic patrol route.

[0070] Step S140: Perform spatial position offset compensation on the three-dimensional path nodes of the reference route based on the obstacle avoidance correction vector set to generate an initial corrected path node set.

[0071] In the scenario of a warehousing and logistics park, the reference route is a pre-set UAV inspection route, which consists of a series of three-dimensional path nodes. In order to enable the UAV to avoid obstacles and adapt to meteorological conditions, it is necessary to adjust the three-dimensional path nodes of the reference route according to the obstacle avoidance correction vector set.

[0072] Step S141: Extract the set of spatial coordinates of each three-dimensional path node in the original geographic coordinate system of the reference route.

[0073] The reference route has a clear definition of path nodes in the geographic coordinate system. Assume that the reference route consists of path nodes P1, P2, P3... Each path node has corresponding three-dimensional spatial coordinates in the geographic coordinate system, denoted as (X1, Y1, Z1), (X2, Y2, Z2), (X3, Y3, Z3)... respectively. These coordinates describe the positions of the path nodes in the actual space. Through data reading and parsing operations, the set of spatial coordinates C1 = {(X1, Y1, Z1), (X2, Y2, Z2), (X3, Y3, Z3)...} of these three-dimensional path nodes in the original geographic coordinate system is extracted from the data storage structure of the reference route.

[0074] Step S142: Perform a three-dimensional Euclidean space vector superposition operation on the set of spatial coordinates according to the three-dimensional offset components corresponding to each path node in the obstacle avoidance correction vector set to generate an adjusted set of three-dimensional spatial coordinates.

[0075] Each path node in the obstacle avoidance correction vector set has a corresponding three-dimensional offset component, such as the three-dimensional offset components δ1, δ2, δ3 generated previously... For each path node coordinate in the reference route space coordinate set C1, such as (Xi, Yi, Zi), a three-dimensional Euclidean space vector superposition operation is performed according to the corresponding three-dimensional offset component (δxi, δyi, δzi). That is, the adjusted coordinates are (Xi+δxi, Yi+δyi, Zi+δzi). By performing such operations on all path node coordinates in the space coordinate set C1 in turn, the adjusted three-dimensional space coordinate set C2={(X1+δx1, Y1+δy1, Z1+δz1), (X2+δx2, Y2+δy2, Z2+δz2), (X3+δx3, Y3+δy3, Z3+δz3)...} is generated. This process causes the path node to be offset in three-dimensional space according to the obstacle avoidance requirements.

[0076] Step S143: Based on the heading angle adjustment weight in the obstacle avoidance correction vector set, the adjusted three-dimensional space coordinate set is compensated for the heading angle deflection of the drone to generate the path node coordinates after the heading angle is corrected.

[0077] The obstacle avoidance correction vector set also includes heading angle adjustment weight parameters, such as α1, α2, α3... For each path node coordinate in the adjusted three-dimensional space coordinate set C2, such as (Xi+δxi, Yi+δyi, Zi+δz1), the heading angle deflection compensation is performed according to the corresponding heading angle adjustment weight parameter αi. Assuming that the current heading angle of the drone is θi, the heading angle is adjusted according to the heading angle adjustment weight parameter αi, and the new heading angle θi'=θi+f(αi), where f(αi) is the heading angle adjustment function determined according to the heading angle adjustment weight parameter αi. Through such processing, the path node coordinates after the heading angle correction are generated, forming a new coordinate set C3={(X1+δx1, Y1+δy1, Z1+δz1, θ1'), (X2+δx2, Y2+δy2, Z2+δz2, θ2'), (X3+δx3, Y3+δy3, Z3+δz3, θ3')...}. This set contains not only the adjusted spatial position of the path node, but also the corrected heading angle information.

[0078] Step S144: using the boundary contour features in the multi-source environment perception features, performing collision detection on the path node coordinates after the heading angle correction, and eliminating abnormal path nodes that have spatial overlap with the obstacle three-dimensional geometric model set.

[0079] The multi-source environmental perception feature set F1 includes the boundary contour features extracted from the multi-spectral image sequence and the set of three-dimensional geometric models of obstacles constructed from the three-dimensional lidar point cloud data. For each path node coordinate in the set C3 of path node coordinates after heading angle correction, such as (Xi + δx1, Y1 + δy1, Z1 + δz1, θi'), collision detection is performed using the boundary contour features and the set of three-dimensional geometric models of obstacles.

[0080] The specific detection process is as follows. First, the position and orientation of the UAV in space are determined according to the path node coordinates. Then, with the UAV as the center, a detection area is determined according to its size and flight attitude. The detection area is compared with each obstacle model in the set of three-dimensional geometric models of obstacles to determine whether there is spatial overlap. For example, for a certain three-dimensional geometric model O1 of an obstacle, if the detection area overlaps with O1 in space, it is considered that there is a risk of collision conflict for this path node, and it is removed from the coordinate set C3. Through such detection and removal operations, a set C4 of path node coordinates that does not overlap with obstacles in space is obtained.

[0081] Step S145: Based on the remaining path node coordinates, perform airspace connectivity verification to generate an initial set of corrected path nodes that meet the UAV flight safety spacing constraints.

[0082] For the coordinate set C4 after abnormal path node removal through collision detection, airspace connectivity verification is required to ensure that the UAV can fly smoothly along these path nodes during flight and meet the flight safety spacing constraints.

[0083] Airspace connectivity verification is to check whether the spatial connection between path nodes is reasonable, without discontinuity or inability to fly through. For example, check whether the distance between adjacent path nodes is within the flyable range of the UAV, and whether the change in heading angle between path nodes is within the operable range of the UAV. At the same time, according to the UAV flight safety spacing constraints, ensure that there is a sufficient safety distance between each path node and surrounding obstacles and other path nodes.

[0084] Assume that the safety spacing constraint is a minimum distance of d1 from obstacles and other path nodes in the horizontal direction and d2 in the vertical direction. For each path node in the coordinate set C4, calculate its distance from surrounding obstacles and other path nodes. If the distance of a certain path node from surrounding objects does not meet the safety spacing constraint, adjust or remove it. Through such airspace connectivity verification and safety spacing check and adjustment operations, an initial set C5 of corrected path nodes that meet the UAV flight safety spacing constraints is generated. The path nodes in this initial set of corrected path nodes will serve as the basis for further optimization to generate the final dynamic patrol route.

[0085] Step S150: Optimize the path smoothness of the initial corrected path node set according to the environment adaptation parameter set, generate a final dynamic patrol route, convert the final dynamic patrol route into a waypoint control instruction stream executable by the UAV flight control system, and transmit it to the on-board controller in real time to drive the UAV to perform the inspection task.

[0086] In the scenario of a warehousing and logistics park, after obtaining the initial corrected path node set C5, in order to enable the UAV to fly more smoothly and efficiently, it is necessary to optimize the path smoothness according to the environment adaptation parameter set E1, and then convert the optimized route into an instruction stream that can be recognized and executed by the UAV flight control system to drive the UAV to perform the inspection task.

[0087] Step S151: Calculate the dynamic safety buffer distance threshold between adjacent nodes in the initial corrected path node set according to the dynamic influence factors in the environment adaptation parameter set, combined with the current flight speed, braking response time and visibility attenuation parameter of the UAV.

[0088] The dynamic influence factors β1, β2, β3... in the environment adaptation parameter set E1 reflect the influence of visibility attenuation on the safety distance between path nodes. At the same time, consider the current flight speed V, braking response time t and visibility attenuation parameter γ of the UAV. For each pair of adjacent path nodes in the initial corrected path node set C5, such as node Pi and Pi+1, calculate the dynamic safety buffer distance threshold Di between them.

[0089] The calculation process is as follows. First, determine the influence coefficient kβi of visibility on the safety distance according to the visibility attenuation parameter γ and the dynamic influence factor βi. Then, according to the current flight speed V and braking response time t of the UAV, calculate the distance dVt = V×t that the UAV can travel during this period. The dynamic safety buffer distance threshold Di = dVt×kβi. Through such calculations, a dynamic safety buffer distance threshold is determined for each pair of adjacent path nodes, and this dynamic safety buffer distance threshold will change according to the changes in environmental conditions and the flight state of the UAV to ensure that the UAV has sufficient safety buffer space during flight.

[0090] Step S152: Adjust the node spacing of the initial corrected path node set uniformly based on the dynamic safety buffer distance threshold to generate an intermediate interpolation node sequence under equal-distance constraints.

[0091] For the initial set of corrected path nodes C5, the node spacing is adjusted uniformly according to the dynamically calculated safety buffer distance threshold Di between each pair of adjacent path nodes. For adjacent path nodes Pi and Pi+1, if the actual distance between them is greater than the dynamically calculated safety buffer distance threshold Di, several intermediate interpolation nodes are inserted between them so that the distance between adjacent nodes is as close as possible to the dynamically calculated safety buffer distance threshold Di.

[0092] Suppose the actual distance between Pi and Pi+1 is Li. By calculating the ratio of Li to Di, the number of intermediate interpolation nodes ni to be inserted is determined. Then, according to a certain interpolation algorithm, such as the linear interpolation algorithm, ni intermediate interpolation nodes are evenly inserted between Pi and Pi+1. For example, in the linear interpolation algorithm, for the node Pi with coordinates (Xi, Yi, Zi) and the node Pi+1 with coordinates (Xi+1, Yi+1, Zi+1), the coordinates (Xij, Yij, Zij) of the inserted intermediate interpolation nodes can be calculated by the formulas Xij = Xi + j×(Xi+1 - Xi) / (ni + 1), Yij = Yi + j×(Yi+1 - Yi) / (ni + 1), Zij = Zi + j×(Zi+1 - Zi) / (ni + 1), where j = 1, 2, ……, ni. By performing such operations on all adjacent path nodes in the initial set of corrected path nodes C5, an intermediate interpolation node sequence S1 under equal-distance constraints is generated. The node spacing in this intermediate interpolation node sequence is more uniform and meets the requirements of the dynamically calculated safety buffer distance threshold.

[0093] Step S153: Use a cubic B-spline curve to fit the spatial trajectory of the intermediate interpolation node sequence to generate a continuous and smooth three-dimensional flight path curve.

[0094] The cubic B-spline curve is a method commonly used for curve fitting, which can generate a smooth curve through a series of discrete points. For the intermediate interpolation node sequence S1 under equal-distance constraints, a cubic B-spline curve is used for spatial trajectory fitting.

[0095] The fitting process of the cubic B-spline curve involves assigning corresponding weights and parameters to each node in the node sequence. Suppose there are n nodes P1, P2, ……, Pn in the intermediate interpolation node sequence S1. By selecting appropriate node parameters t1, t2, ……, tn and using the calculation formula of the cubic B-spline curve, a smooth curve passing through these nodes is generated. The calculation formula of the cubic B-spline curve involves weighted summation of node coordinates and operations of basis functions. For example, for the coordinates (X, Y, Z) of a point on the curve, they can be calculated by the formulas X = Σ(i = 1 to n)Ni,3(t)×Xi, Y = Σ(i = 1 to n)Ni,3(t)×Yi, Z = Σ(i = 1 to n)Ni,3(t)×Zi, where Ni,3(t) is the cubic B-spline basis function, Xi, Yi, and Zi are the coordinates of node Pi, and t is the parameter value corresponding to a point on the curve. Through such fitting operations, a continuous and smooth three-dimensional flight path curve T1 is generated, enabling the UAV to fly smoothly along this curve during flight.

[0096] Step S154: Based on the horizontal direction offset coefficient in the multi-source environmental perception features, optimize the curvature of the three-dimensional flight path curve against wind pressure on the horizontal plane to generate a disturbance-resistant smooth flight path.

[0097] The multi-source environmental perception feature set F1 includes the horizontal direction offset coefficient H1 extracted from real-time meteorological sensor data. The horizontal direction offset coefficient H1 reflects the influence of wind pressure in the horizontal direction on the flight path of the UAV. For the continuous and smooth three-dimensional flight path curve T1, perform anti-wind pressure curvature optimization on the horizontal plane according to the horizontal direction offset coefficient H1.

[0098] The specific optimization process is as follows. First, project the three-dimensional flight path curve T1 onto the horizontal plane to obtain the horizontal trajectory curve Th. For each point on the horizontal trajectory curve Th, calculate the amount by which the curve curvature needs to be adjusted according to the horizontal direction offset coefficient H1. Suppose the value of the horizontal direction offset coefficient H1 in a certain area is h1. Determine whether the curvature of the horizontal trajectory curve Th needs to be increased or decreased in this area according to the magnitude and direction of h1. By performing such curvature adjustment operations on all points on the horizontal trajectory curve Th, a disturbance-resistant smooth flight path Ta is generated. This disturbance-resistant smooth flight path Ta can better resist the influence of wind pressure on the horizontal plane, ensuring that the UAV can fly stably in a windy environment.

[0099] Step S155: According to the maximum steering angle, minimum turning radius, and maximum centripetal acceleration parameters of the UAV flight control system, perform sectional heading angle continuity verification and curvature smoothing optimization on the disturbance-resistant smooth flight path to generate the final dynamic patrol route that simultaneously meets kinematic and dynamic constraints.

[0100] The UAV flight control system stipulates some parameters related to flight performance, such as the maximum steering angle θmax, the minimum turning radius rmin, and the maximum centripetal acceleration amax. For the disturbance-resistant smooth flight path Ta, it is necessary to perform segmented heading angle continuity verification and curvature smoothing optimization based on these parameters.

[0101] Divide the disturbance-resistant smooth flight path Ta into several segments. For each segment of the flight path, check whether the change in the heading angle is within the range of the maximum steering angle θmax. If the change in the heading angle of a certain segment of the flight path exceeds θmax, then adjust the curvature of that segment of the flight path. At the same time, check and optimize the curvature of each segment of the flight path according to the minimum turning radius rmin and the maximum centripetal acceleration amax.

[0102] Suppose that on a certain segment of the flight path, the current curvature is κ. According to the velocity v at a certain point on the flight path and the centripetal acceleration formula a = v²×κ, calculate the centripetal acceleration a corresponding to the current curvature. If a exceeds the maximum centripetal acceleration amax, then reduce the curvature of that segment of the flight path to meet the dynamic constraints. The method of adjusting the curvature can be achieved by slightly adjusting the coordinates of each point on the flight path. For example, for the point P on that segment of the flight path, according to a certain rule, such as moving the point a certain distance in the direction that reduces the curvature, thus changing the curvature of the entire segment of the flight path.

[0103] During the process of adjusting the curvature, it is necessary to ensure the continuity of the heading angle at the same time. If the change in the heading angle is discontinuous due to the curvature adjustment, it is necessary to further optimize the flight path. For example, at the connection of two adjacent segments of the flight path, by slightly adjusting the coordinates near the connection point, the heading angle can be smoothly transitioned. After such operations of checking the continuity of the heading angle and smoothing the curvature optimization for each segment of the flight path, the final dynamic patrol route L that simultaneously meets the kinematic and dynamic constraints is generated.

[0104] Step S156: Convert the final dynamic patrol route into a waypoint control instruction stream executable by the UAV flight control system and transmit it to the on-board controller in real time to drive the UAV to perform the inspection task.

[0105] The final dynamic patrol route L is the ideal flight path of the UAV represented in the form of a continuous and smooth curve. However, the UAV flight control system requires discrete waypoint control instructions to drive the UAV to fly. Therefore, it is necessary to convert the final dynamic patrol route L into a waypoint control instruction stream executable by the UAV flight control system.

[0106] First, sample the final dynamic patrol route L. According to the flight accuracy requirements of the UAV and the processing ability of the flight control system, determine the sampling interval. For example, sample the route L at regular intervals of a certain distance d or time interval t to obtain a series of discrete waypoints P1', P2', P3'... The position coordinates and corresponding time information of these waypoints in space constitute the waypoint sequence.

[0107] For each waypoint Pi', based on its position coordinates and the current flight state of the UAV (such as the current position, heading angle, etc.), calculate the control instructions required to control the UAV to reach this waypoint from the current position. These control instructions include the speed instruction, steering instruction, altitude adjustment instruction, etc. of the UAV. Assume the current position of the UAV is P0, the heading angle is θ0, the waypoint to be reached is Pi', and the coordinates are (Xi', Yi', Zi'). By calculating the distance and direction between the two points, determine the speed instruction v instruction so that the UAV can reach this waypoint within a certain time. At the same time, according to the difference between the direction between the two points and the current heading angle, determine the steering instruction Δθ instruction so that the UAV can adjust its heading and fly towards the waypoint. According to the difference between the altitude Zi' of the waypoint and the current altitude Z0 of the UAV, determine the altitude adjustment instruction ΔZ instruction.

[0108] Arrange these control instructions for each waypoint in the order of the waypoints to form a waypoint control instruction stream I. For example, the waypoint control instruction stream I can be expressed as [(v1 instruction, Δθ1 instruction, ΔZ1 instruction), (v2 instruction, Δθ2 instruction, ΔZ2 instruction), (v3 instruction, Δθ3 instruction, ΔZ3 instruction) ……].

[0109] Then, transmit the waypoint control instruction stream I to the on-board controller in real time. The transmission method can be realized through a wireless communication module, such as using radio signals in a specific frequency band for data transmission. After receiving the waypoint control instruction stream I, the on-board controller parses and executes these instructions in sequence according to the order of the instructions, so as to drive the UAV to perform the inspection task along the final dynamic patrol route L. During the process of the UAV performing the task, the on-board controller will continuously compare and adjust according to the actual flight state and the instruction requirements to ensure that the UAV accurately flies according to the predetermined route.

[0110] Furthermore, for the dynamic route planning model, the training steps are referred to the following embodiments.

[0111] The dynamic route planning model needs to be trained before application so that it can accurately generate a reasonable set of obstacle avoidance correction vectors and a set of environment adaptation parameters according to the multi-source environment perception features.

[0112] First, collect a large number of multi-source environment perception data sets from different scenarios, as well as the corresponding expected obstacle avoidance correction vector sets and environment adaptation parameter sets. These scenarios can include, but are not limited to, different types of areas such as warehousing and logistics parks, industrial parks, and urban areas.

[0113] For the multi-source environmental perception data set, as obtained in the previous embodiments, it includes a multi-spectral image sequence collected synchronously, 3D lidar point cloud data, and real-time meteorological sensor data. For example, multi-spectral image sequences I_a1, I_a2,... are collected from multiple different warehousing and logistics parks, 3D lidar point cloud data P_a1, P_a2,... and real-time meteorological sensor data M_a1, M_a2,... These data are organized into the input part of the training samples in a certain format.

[0114] For the set of desired obstacle avoidance correction vectors and the set of environment adaptation parameters, they are determined through manual annotation or based on known reasonable route planning results. For example, for the multi-source environmental perception data set in a certain warehousing and logistics park scenario, according to the pre-planned ideal route that can avoid obstacles and adapt to the environment, the corresponding set of obstacle avoidance correction vectors R_a1, R_a2,... and the set of environment adaptation parameters E_a1, E_a2,... are calculated. These data are organized into the output part of the training samples.

[0115] The input part and the output part are correspondingly combined to form a training sample set T = [(I_a1, P_a1, M_a1, R_a1, E_a1), (I_a2, P_a2, M_a2, R_a2, E_a2),...].

[0116] The dynamic route planning model consists of modules such as an encoder, a spatio-temporal evolution prediction module, a path node offset prediction network, an environment compensation generator, a vector fusion layer, and a fully connected decoder. Before the start of training, each module of the model is initialized.

[0117] For the encoder, its internal parameters are initialized, such as the weight matrix in the cross-channel attention mechanism. Assume there is a weight matrix W_enc in the encoder, and it is given an initial value through random initialization. At the same time, other parameters in the encoder, such as the bias term, are also initialized accordingly.

[0118] For the 3D gated convolutional layer in the spatio-temporal evolution prediction module, its convolutional kernel parameters, weight parameters in the gating mechanism, etc. are initialized. For example, the convolutional kernel parameters K_conv of the 3D gated convolutional layer are set with an initial value through a certain initialization method (such as random initialization or initialization based on a certain distribution).

[0119] Modules such as the path node offset prediction network, environment compensation generator, vector fusion layer, and fully connected decoder also initialize their respective internal parameters. For example, the weight matrix W_pno in the path node offset prediction network, the recurrent neural network parameters RNN_params in the environment compensation generator, the learnable weight matrix W_vf in the vector fusion layer, and the weight matrix W_fc in the fully connected decoder are all given initial values through corresponding initialization methods.

[0120] Extract the features of the samples in the training sample set T to obtain the multi-source environment perception features of the samples and input them into the dynamic route planning model in sequence. Taking a training sample (I_ai, P_ai, M_ai, R_ai, E_ai) as an example, first extract the features of the multi-source environment perception data part (I_ai, P_ai, M_ai) to obtain the multi-source environment perception features of the sample and input them into the encoder.

[0121] The encoder performs spatio-temporal alignment on the multi-source environment perception features of the sample through the cross-channel attention mechanism and outputs a multi-source environment perception tensor T_ai with a unified dimension. In this process, the internal parameters of the encoder, such as the weight matrix W_enc, can be adjusted according to the input data to optimize the processing effect of heterogeneous features.

[0122] Input the multi-source environment perception tensor T_ai into the spatio-temporal evolution prediction module, use the three-dimensional gated convolutional layer to extract the spatio-temporal coupling relationship between the dynamic displacement trend of obstacles and meteorological disturbances, and generate the obstacle risk probability distribution P_ai' and the meteorological interference propagation field F_ai'. In this process, the parameters K_conv, etc. of the three-dimensional gated convolutional layer will be adjusted according to the training data, so that the generated obstacle risk probability distribution and meteorological interference propagation field are closer to the real situation.

[0123] The path node offset prediction network generates the obstacle avoidance repulsive force vector R_ai' of each path node of the reference route in the tangential direction according to the obstacle risk probability distribution P_ai', and the environment compensation generator generates the meteorological interference compensation parameter set C_ai' according to the meteorological interference propagation field F_ai'. In the process of processing these two modules, the internal parameters, such as the weight matrix W_pno in the path node offset prediction network and the recurrent neural network parameters RNN_params in the environment compensation generator, will also be continuously adjusted.

[0124] The obstacle avoidance repulsive force vector R_ai' and the meteorological interference compensation parameter set C_ai' are input into the vector fusion layer to generate an obstacle avoidance correction vector set Rv_ai' containing three-dimensional offset components and heading angle adjustment weights. The learnable weight matrix W_vf in the vector fusion layer will be learned and adjusted according to the training data in this process to better synthesize the obstacle avoidance and meteorological factors to generate reasonable obstacle avoidance correction vectors.

[0125] Based on the hidden state features of the spatio-temporal evolution prediction module, the fully connected decoder extracts the dynamic influence factors of visibility attenuation on the safe spacing of path nodes, and generates the set of environment adaptation parameters E_ai'. The weight matrix W_fc in the fully connected decoder will also be adjusted during the training process to make the generated set of environment adaptation parameters closer to the desired results.

[0126] Compare the set of obstacle avoidance correction vectors Rv_ai' and the set of environment adaptation parameters E_ai' generated by the model with the desired results R_ai and E_ai in the training samples, and calculate the loss value. There are various methods for calculating the loss value, such as the mean squared error loss function, etc. Assuming the mean squared error loss function is used, for the set of obstacle avoidance correction vectors, the loss value L_Rv = Σ((Rv_ai' - R_ai)²), and for the set of environment adaptation parameters, the loss value L_E = Σ((E_ai' - E_ai)²). The total loss value L = L_Rv + L_E.

[0127] According to the loss value, all parameters of the model are updated through the backpropagation algorithm. The backpropagation algorithm calculates the gradient of the loss value with respect to each parameter, and then adjusts the value of the parameter according to the gradient to gradually reduce the loss value. For example, for the weight matrix W_enc in the encoder, according to the gradient descent algorithm, the update formula can be expressed as W_enc = W_enc - η × ∂L / ∂W_enc, where η is the learning rate and ∂L / ∂W_enc is the gradient of the loss value L with respect to the weight matrix W_enc. Update the parameters of other modules such as K_conv, W_pno, RNN_params, W_vf, W_fc, etc. in the same way.

[0128] Repeat the above process, input all samples in the training sample set T into the model for training in sequence, and continuously adjust the model parameters until the loss value converges to a smaller value, indicating that the model has achieved a better training effect.

[0129] Figure 2 FIG. shows a schematic diagram of exemplary hardware and software components of a patrol route self-adjustment system 100 based on UAV patrol inspection provided by some embodiments of the present application that can implement the idea of the present application. For example, the processor 120 can be used on the patrol route self-adjustment system 100 based on UAV patrol inspection and is used to execute the functions in the present application.

[0130] The patrol route self-adjustment system 100 based on UAV patrol inspection can be a general-purpose server or a special-purpose server, both of which can be used to implement the patrol route self-adjustment method based on UAV patrol inspection of the present application. Although only one server is shown in the present application, for the sake of convenience, the functions described in the present application can be implemented in a distributed manner on multiple similar platforms to balance the processing load.

[0131] For example, the patrol route self - adjustment system 100 based on drone patrol inspection may include a network port 110 connected to a network, one or more processors 120 for executing program instructions, a communication bus 130, and different forms of storage media 140, such as disks, ROM, or RAM, or any combination thereof. Exemplarily, the patrol route self - adjustment system 100 based on drone patrol inspection may also include program instructions stored in ROM, RAM, or other types of non - transitory storage media, or any combination thereof. The method of the present application can be implemented according to these program instructions. The patrol route self - adjustment system 100 based on drone patrol inspection further includes an I / O interface 150 between the computer and other input / output devices.

[0132] For ease of explanation, only one processor is described in the patrol route self - adjustment system 100 based on drone patrol inspection. However, it should be noted that the patrol route self - adjustment system 100 in the present application may also include multiple processors. Therefore, the steps performed by one processor described in the present application can also be jointly executed or separately executed by multiple processors. For example, if the processor of the patrol route self - adjustment system 100 based on drone patrol inspection executes step A and step B, it should be understood that step A and step B can also be jointly executed by two different processors or separately executed in one processor. For example, the first processor executes step A, the second processor executes step B, or the first processor and the second processor jointly execute steps A and B.

[0133] In addition, an embodiment of the present invention further provides a readable storage medium, in which computer - executable instructions are preset. When the processor executes the computer - executable instructions, the above - mentioned patrol route self - adjustment method based on drone patrol inspection is implemented.

[0134] It should be noted that, in order to simplify the description of the present invention disclosure and thus help the understanding of one or more embodiments of the invention, in the previous description of the embodiments of the present invention, sometimes multiple features are merged into one embodiment, drawing, or description thereof.

Claims

1. A self - adjusting method for patrol routes based on UAV inspection, characterized in that, The method includes: Obtaining a multi-source environmental perception data set of a target area, where the multi-source environmental perception data set includes a multi-spectral image sequence, three-dimensional lidar point cloud data, and real-time meteorological sensor data collected synchronously; Performing feature extraction on the multi-source environmental perception data set to obtain multi-source environmental perception features; Inputting the multi-source environmental perception features into a dynamic route planning model to generate an obstacle avoidance correction vector set and an environmental adaptation parameter set; Performing spatial position offset compensation on the three-dimensional path nodes of the reference route based on the obstacle avoidance correction vector set to generate an initial corrected path node set; Performing path smoothness optimization processing on the initial corrected path node set according to the environmental adaptation parameter set to generate a final dynamic patrol route, converting the final dynamic patrol route into a waypoint control instruction stream executable by a UAV flight control system, and transmitting it to an on-board controller in real time to drive the UAV to perform an inspection task.

2. The self - adjusting method of the patrol route based on drone patrol according to claim 1, wherein, The performing feature extraction on the multi-source environmental perception data set to obtain multi-source environmental perception features includes: Performing topographic semantic segmentation processing on the multi-spectral image sequence to generate a surface cover type distribution map and an initial obstacle position feature set; Performing spatial density clustering analysis on the three-dimensional lidar point cloud data to construct a three-dimensional geometric model set of obstacles and extract the spatial occupancy features of each obstacle; Performing wind speed gradient modeling and visibility attenuation analysis on the real-time meteorological sensor data to generate a meteorological interference influence coefficient set; Converging the surface cover type distribution map, the initial obstacle position feature set, the three-dimensional geometric model set, the spatial occupancy features, and the meteorological interference influence coefficient set to obtain the multi-source environmental perception features.

3. The patrol route self-adjustment method based on UAV patrol inspection according to claim 2, characterized in that The performing topographic semantic segmentation processing on the multi-spectral image sequence to generate a surface cover type distribution map and an initial obstacle position feature set includes: Invoking a pre-trained semantic segmentation neural network to classify each pixel in the multi-spectral image to obtain classified pixel regions, where the classified pixel regions include vegetation regions, water regions, building regions, and temporary obstacle regions; Performing morphological closing operation processing on the classified pixel regions to generate continuous surface cover patches; Extracting the geometric center coordinates and boundary contour features of each surface cover patch to construct the surface cover type distribution map; Matching the boundary contour features of the temporary obstacle regions with a preset obstacle feature template to screen out significant obstacle regions and generate the initial obstacle position feature set.

4. The patrol route self-adjustment method based on drone patrol inspection according to claim 2, wherein The performing spatial density clustering analysis on the three-dimensional lidar point cloud data to construct a three-dimensional geometric model set of obstacles and extract the spatial occupancy features of each obstacle includes: Performing voxel grid division on the three-dimensional lidar point cloud data to generate a set of voxel units with uniform spatial distribution; Performing density statistics on the point cloud data in each voxel unit in the set of voxel units, and screening out voxel units with a density exceeding a preset threshold as candidate obstacle regions; Performing connected component analysis on the candidate obstacle regions to merge spatially adjacent voxel units to form an obstacle clustering cluster; Perform minimum circumscribed cube fitting on each obstacle clustering cluster, and calculate the central coordinates, size parameters, and surface normal vector directions of the minimum circumscribed cube; Generate the three-dimensional geometric model set based on the central coordinates and size parameters, and extract the surface normal vector directions of each cube as the spatial occupancy features.

5. The patrol route self-adjustment method based on UAV inspection according to claim 2, characterized in that The wind speed gradient modeling and visibility attenuation analysis are performed on the real-time meteorological sensor data to generate a set of meteorological interference influence coefficients, including: Perform time series sampling on the real-time meteorological sensor data to obtain the wind speed vector and visibility measurement value at the current moment; Construct a wind speed change rate prediction model based on historical wind speed data, and calculate the wind speed gradient change curve within the next N sampling periods; Generate visibility compensation parameters for the lidar detection accuracy according to the visibility measurement value and the preset attenuation coefficient model; Decompose the wind speed gradient change curve into a horizontal direction offset coefficient and a vertical direction turbulence intensity coefficient; Integrate the horizontal direction offset coefficient, the vertical direction turbulence intensity coefficient, and the visibility compensation parameters to generate the set of meteorological interference influence coefficients.

6. The patrol route self-adjustment method based on drone inspection according to any one of claims 1-5, characterized in that The multi-source environmental perception features are input into the dynamic route planning model to generate an obstacle avoidance correction vector set and an environmental adaptation parameter set, including: Input the multi-source environmental perception features into the encoder of the dynamic route planning model, and perform spatio-temporal alignment on heterogeneous features through a cross-channel attention mechanism to generate a multi-source feature tensor with unified dimensions; Input the multi-source feature tensor into the spatio-temporal evolution prediction module of the dynamic route planning model, and use a three-dimensional gated convolutional layer to extract the spatio-temporal coupling relationship between the dynamic displacement trend of obstacles and meteorological disturbances, and generate an obstacle risk probability distribution and a meteorological interference propagation field; Perform three-dimensional spatial gradient inversion on the obstacle risk probability distribution through the path node offset prediction network of the dynamic route planning model to generate an obstacle avoidance repulsive force vector in the tangential direction of each path node of the reference route; Input the meteorological interference propagation field into the environmental compensation generator of the dynamic route planning model, and extract the horizontal wind speed compensation gradient and the vertical turbulence suppression coefficient through recurrent neural network sequence modeling to generate a set of meteorological interference compensation parameters; Input the obstacle avoidance repulsive force vector and the set of meteorological interference compensation parameters into the vector fusion layer of the dynamic route planning model, and use a learnable weight matrix for direction vector synthesis and norm normalization to generate an obstacle avoidance correction vector set including three-dimensional offset components and heading angle adjustment weights; Based on the hidden state features of the spatio-temporal evolution prediction module, extract the dynamic influence factor of visibility attenuation on the safe spacing of path nodes through the fully connected decoder of the dynamic route planning model to generate an environmental adaptation parameter set with the same dimension as the obstacle avoidance correction vector set.

7. The self - adjustment method of patrol route based on UAV inspection according to claim 6, characterized in that, Input the obstacle avoidance repulsive force vector and the set of meteorological interference compensation parameters into the vector fusion layer of the dynamic route planning model, and use a learnable weight matrix for direction vector synthesis and norm normalization to generate an obstacle avoidance correction vector set including three-dimensional offset components and heading angle adjustment weights, including: Performing unit vector processing of the obstacle avoidance repulsion force vector in a three-dimensional space coordinate system, and extracting the obstacle avoidance direction reference vector and its modulus length scaling coefficient corresponding to each path node; Decomposing the horizontal wind speed compensation gradient in the meteorological interference compensation parameter set into a lateral wind pressure offset vector and a longitudinal airflow offset vector, and mapping the vertical turbulence suppression coefficient into a vertical fluctuation suppression vector; Performing three-dimensional space vector superposition of the lateral wind pressure offset vector, the longitudinal airflow offset vector and the vertical fluctuation suppression vector to generate a meteorological interference compensation vector; According to the maximum flight speed and maneuvering acceleration threshold set by the UAV flight control system, the modulus length of the weather interference compensation vector is dynamically constrained and truncated to generate a normalized weather compensation vector that meets the dynamic performance of the UAV; Inputting the obstacle avoidance direction reference vector and the normalized meteorological compensation vector into the learnable weight matrix, and generating a three-dimensional space offset synthetic vector through matrix multiplication and addition operation; Using a dynamic attenuation factor to map the heading angle offset of the three-dimensional space offset synthesis vector, and extracting a heading angle adjustment weight parameter; Dynamically scaling the three-dimensional spatial offset synthesis vector according to the modulus scaling coefficient to generate a three-dimensional offset component; The three-dimensional offset component is concatenated with the heading angle adjustment weight parameter to generate a correction vector corresponding to each path node in the obstacle avoidance correction vector set.

8. The patrol route self-adjustment method based on UAV inspection according to claim 6, characterized in that The performing spatial position offset compensation on the three-dimensional path nodes of the reference route based on the obstacle avoidance correction vector set to generate an initial correction path node set includes: Extracting a set of spatial coordinates of each three-dimensional path node in the reference route in the original geographic coordinate system; According to the three-dimensional offset components corresponding to each path node in the obstacle avoidance correction vector set, performing a three-dimensional Euclidean space vector superposition operation on the space coordinate set to generate an adjusted three-dimensional space coordinate set; Based on the heading angle adjustment weight in the obstacle avoidance correction vector set, the adjusted three-dimensional space coordinate set is compensated for the heading angle deflection of the drone to generate the path node coordinates after the heading angle is corrected; Using the boundary contour features in the multi-source environment perception features, collision detection is performed on the path node coordinates after the heading angle correction, and abnormal path nodes that overlap with the obstacle three-dimensional geometric model set in space are eliminated; The airspace connectivity is verified based on the coordinates of the eliminated path nodes, and an initial corrected path node set that meets the UAV flight safety spacing constraints is generated.

9. The self-adjusting method of the patrol route based on drone inspection according to claim 6, characterized in that The step of performing path smoothness optimization processing on the initial corrected path node set according to the environmental adaptation parameter set to generate a final dynamic patrol route includes: According to the dynamic influencing factors in the environmental adaptation parameter set, combined with the current flight speed, braking response time and visibility attenuation parameters of the UAV, a dynamic safety buffer distance threshold between adjacent nodes in the initial correction path node set is calculated; Based on the dynamic safety buffer distance threshold, the node spacing of the initial corrected path node set is uniformly adjusted to generate an intermediate interpolation node sequence under an equidistance constraint; Use a cubic B-spline curve to fit the spatial trajectory of the intermediate interpolation node sequence to generate a continuous and smooth three-dimensional flight path curve; Based on the horizontal direction offset coefficient in the multi-source environment perception features, optimize the curvature of the three-dimensional flight path curve against wind pressure on the horizontal plane to generate a disturbance-resistant smooth flight path; According to the maximum steering angle, minimum turning radius, and maximum centripetal acceleration parameters of the UAV flight control system, perform piecewise heading angle continuity verification and curvature smoothing optimization on the disturbance-resistant smooth flight path to generate a final dynamic patrol route that simultaneously satisfies kinematic and dynamic constraints.

10. A patrol route self-adjusting system based on drone patrol, characterized in that, It includes a processor and a memory. The memory is connected to the processor. The memory is used to store programs, instructions, or codes, and the processor is used to execute the programs, instructions, or codes in the memory to implement the patrol route self-adjustment method based on UAV inspection as described in any one of claims 1-9 above.

Images