An unmanned aerial vehicle dynamic path planning method based on lightweight transform space-time fusion and energy-safety collaborative optimization

Abstract

This invention discloses a dynamic path planning method for unmanned aerial vehicles (UAVs) based on lightweight Transformer spatiotemporal fusion and energy consumption-safety co-optimization, belonging to the field of UAV autonomous navigation and intelligent path planning technology. The method first performs spatiotemporal alignment of multi-source sensor data, and uses a lightweight Transformer with M layers of cross-modal multi-head attention to achieve deep fusion of multimodal features, generating a unified environmental state vector. A dynamic risk field R(p) is constructed using the uncertainty (position standard deviation) of dynamic obstacle trajectory prediction as the confidence weight. A path planning method incorporating battery health status (SOH) into the energy consumption cost function is proposed, using extended Kalman filtering to estimate SOH online, overcoming the range estimation bias caused by battery aging. The rate of change of information entropy ΔH of the environmental state probability distribution is used as the adaptive replanning trigger index, replacing the fixed threshold method. A three-level path planning architecture—macro-meta-micro—is adopted. Simulation results (2300 m task, 7 contingency scenarios): Path correction response latency is reduced by approximately 94% under medium threat conditions (from 2.1 s to 0.12 s); macroscopic replanning frequency is reduced by approximately 23%; energy consumption estimation error is reduced from approximately 14.9% to approximately 1.0% after introducing SOH perception (50,000 Monte Carlo simulations); the lightweight Transformer has a single-frame inference latency of approximately 14.6 ms, meeting the 20 ms real-time constraint.

Description

Technical Field

[0001] This invention belongs to the field of UAV autonomous navigation and intelligent path planning technology, specifically involving a UAV dynamic path planning method based on lightweight Transformer spatiotemporal fusion and energy consumption-safety collaborative optimization. Background Technology

[0002] Path planning for unmanned aerial vehicles (UAVs) in complex dynamic environments is a current research hotspot and challenge. Existing path planning methods mainly suffer from the following four shortcomings: First, the depth of multi-source information fusion is insufficient. Existing methods typically employ a hierarchical architecture that uses Kalman filtering to process data from the same sensor source, convolutional neural networks to extract features, and DS evidence theory for decision fusion. However, this architecture has limited ability to model temporal correlations in heterogeneous sensor data and struggles to fully capture long-range dependencies between different modalities, resulting in insufficient accuracy in environmental state representation.

[0003] Second, the path cost function does not consider the battery health status. Existing methods typically optimize path length or coarse-grained energy consumption estimation, neglecting the nonlinear changes in lithium battery performance caused by the decrease in actual capacity and increase in internal resistance with increasing cycle count. Path planning based on rated capacity will lead to deviations in range estimation. Simulation analysis shows that for a battery with a rated capacity of 22 Ah and SOH=0.87, the actual usable energy is only 19.14 Ah. Without introducing SOH correction, the energy consumption estimation error is approximately 14.9%, which may lead to mission interruption.

[0004] Third, the replanning triggering mechanism lacks adaptability. Existing methods typically use a fixed threshold to trigger path replanning. When the threshold is set too low, even slight environmental disturbances can trigger a costly global replanning (taking about 2.1 seconds), significantly increasing computational overhead. When the threshold is set too high, moderate-intensity threats cannot be responded to in a timely manner, and the threat exposure time can be as long as 2.1 seconds, posing a security risk.

[0005] Fourth, the path planning granularity is limited. Most methods employ a single-level planning structure, using the same planning tool regardless of threat level, making it difficult to balance global optimality with local real-time performance. Even for small path deviations, single-level methods still require a complete global replanning operation, resulting in wasted computational resources. Summary of the Invention

[0006] The purpose of this invention is to provide a dynamic path planning method for unmanned aerial vehicles (UAVs) based on lightweight Transformer spatiotemporal fusion and energy consumption-safety co-optimization, which solves the shortcomings of existing methods in terms of multi-source spatiotemporal feature extraction depth, battery health perception, adaptive replanning triggering, and multi-granularity planning.

[0007] To achieve the above objectives, the technical solutions adopted by the present invention are detailed in claims 1 to 5.

[0008] Compared with the prior art, the beneficial effects of the present invention are as follows: First, a lightweight cross-modal Transformer is used for spatiotemporal feature fusion. Compared with the hierarchical architecture of convolutional neural network plus Kalman filter plus DS evidence theory, it can more effectively capture the long-range temporal dependencies of multimodal information. After structured pruning, the number of parameters does not exceed 30% of the standard Transformer. The single-frame inference latency on the Jetson AGX Xavier platform is about 14.6 ms, which meets the 20 ms real-time constraint.

[0009] Second, the battery health status (SOH) is introduced into the path cost function, and the SOH is estimated online by extended Kalman filter. After 50,000 Monte Carlo simulations, the energy consumption estimation error is reduced from about 14.9% to about 1.0% after introducing SOH sensing (EKF estimation standard deviation is about 1.2%, and the error does not exceed 2.4% within the 95% confidence interval).

[0010] Third, using the rate of change of information entropy As a replanning trigger indicator, it can distinguish threat levels and match corresponding planning levels. Simulation verification (2300 m mission, 7 contingency events, 2875 control cycles): For medium threat scenarios, the path correction response delay was reduced from about 2.1 s to about 0.12 s, a reduction of about 94%; the number of macro replanning cycles was reduced by about 23%.

[0011] Fourth, a three-tiered path planning architecture of macro-meso-micro is adopted to achieve "tool-task matching": the micro-level smoothing takes about 0.005 s, the meso-level local replanning takes about 0.12 s, and the macro-level global replanning takes about 2.1 s. Each level is triggered only under the corresponding threat level to avoid over-computation. Attached Figure Description

[0012] Figure 1 This is an overall flowchart of the method of the present invention; Figure 2 A schematic diagram of the lightweight cross-modal Transformer spatiotemporal fusion module structure; Figure 3 A schematic diagram illustrating the construction of a confidence-weighted dynamic risk field; Figure 4 The diagram shows the online estimation of battery SOH, where (a) is the RC equivalent circuit model and (b) is the extended Kalman filter estimation process. Figure 5 A schematic diagram of a three-tiered adaptive replanning architecture: macro-meta-micro. Figure 6 The figure shows a comparison of simulation results between the method of this invention and the baseline method in three typical scenarios. Detailed Implementation

[0013] The method of the present invention includes steps S1 to S6, and the overall process is as follows: Figure 1 As shown.

[0014] The multi-source input information includes six categories: (a) UAV angular velocity output by the inertial measurement unit (rad / s) and linear acceleration (m / s²), sampling rate 200Hz; (b) Location output by the Global Navigation Satellite System (Latitude, Longitude, Ellipsoidal Altitude) and Speed (m / s), update rate 10 Hz, differential positioning accuracy approximately ±0.1 m; (c) 3D point cloud output by lidar Scanning frequency 10 Hz, 16-line harness; (d) Target range output by millimeter-wave radar (m) and radial velocity (m / s), operating frequency 77 GHz, refresh rate 20 Hz; (e) Wind speed output by the meteorological sensor (m / s), wind direction (rad) and temperature and humidity, sampling rate 1 Hz; (f) The sequence of mission target points and mission type instructions issued by the ground station.

[0015] Spatiotemporal alignment using lidar frame timestamps Based on this, the following strategy is adopted: For inertial measurement unit data, linear interpolation is performed within a 20 ms window to obtain the angular velocity and acceleration at the corresponding time of the lidar frame; for global navigation satellite system data, linear extrapolation is performed using the position and velocity of the nearest neighbor frame; for millimeter-wave radar data, the extrinsic parameter matrix obtained from calibration is used. A rigid body coordinate transformation was performed to convert it from the millimeter-wave radar coordinate system to the lidar coordinate system. After the above processing, the measured maximum time alignment deviation was approximately 3.2 ms, which meets the 5 ms constraint.

[0016] The goal of this step is to fuse heterogeneous sensor data from different modalities into a unified environmental state vector. ,like Figure 2 As shown.

[0017] The structure of each modal independent encoder is as follows: (a) LiDAR 3D point cloud encoder: A lightweight PointPillar voxel encoder is adopted, with a voxel size of 0.2 m and a maximum of 32 points per voxel. Voxel features are obtained through a point-by-point multilayer perceptron (pooling after the point dimension is increased to 64 dimensionality), and then spatial features are extracted through a 2D convolutional network. Finally, global average pooling is used to obtain a 128-dimensional feature vector. ; (b) Millimeter-wave radar encoder: The target range-velocity list of each frame is progressively upgraded layer by layer through a four-layer perceptron (input dimension 2, hidden layer dimensions 16, 32, 64, output 128), and then global average pooling is used to obtain the output. ; (c) Inertial Measurement Unit (IMU) Timing Encoder: Ten consecutive frames of IMU data (10×6 shape) are fed into a three-layer one-dimensional convolutional network (kernel size 5, stride 2, and channel numbers 32, 64, and 128 respectively), and global max pooling is performed to obtain... ; (d) Global Navigation Satellite System and Meteorological Scalar Encoder: The 9-dimensional scalar data, including position, velocity, wind speed, and wind direction, are upscaled to 128 dimensions via a linear projection layer to obtain... .

[0018] The computation process of the multi-head attention sublayer across modalities is as follows: In each layer, for any two modalities... and ( ),calculate , , ,in , , For dimension Learnable projection matrix, (Attention count) Dimension of each key vector Cross-modal attention values ; The results from each head are spliced ​​together and then linearly projected back to 128 dimensions; the feedforward network sublayer contains two fully connected layers (hidden layer dimension 512), and residual connections and layer normalization are used.

[0019] After structured channel pruning (pruning ratio of 60%) and knowledge distillation to restore accuracy, the fusion module has approximately 1.8 M parameters (compared to approximately 6 M for a standard 3-layer Transformer). On the Jetson AGX Xavier platform, the single-frame inference time is approximately 14.6 ms, meeting the 20 ms real-time constraint, with an update frequency of 10 Hz.

[0020] Fusion output ; Generated by global average pooling, satisfying .

[0021] like Figure 3 As shown, dynamic risk field It consists of three superimposed components: dynamic obstacle risk component, terrain risk component, and wind field risk component.

[0022] Dynamic obstacle risk component: A Long Short-Term Memory (LSTM) network with attention mechanism is used to predict the trajectory of dynamic obstacles detected by millimeter-wave radar in 3 seconds. The prediction output is a Gaussian distribution (mean) of the position at each future time step. Standard deviation Confidence coefficient ; in The reference standard deviation is used. The greater the forecast uncertainty ( When the larger the value, The smaller the value, the lower the corresponding risk weight, reflecting a conservative design.

[0023] Dynamic obstacle risk components: ; in , Based on the safe distance, For speed coefficient, The speed of the obstacle is (m / s). In urban low-altitude delivery scenarios, the pedestrian speed is approximately 1.2 m / s. Approximately 2.36 m; when the vehicle speed is approximately 10 m / s Approximately 5 m.

[0024] Terrain risk component According to the terrain slope Segmented values: Slope No more than Time: 0 (flat terrain, no additional risk); Slope: Take a slope of 0.5 (medium gradient, requiring appropriate increase in flight altitude); slope Take 1.0 (steep slope, requires detour or significant elevation gain).

[0025] Wind field risk components ; in Location obtained based on meteorological sensors and prediction models Predicted wind speed (m / s). This is the maximum wind-resistant speed for the drone.

[0026] like Figure 4 As shown, an RC equivalent circuit model is used to model the lithium battery. The state variables are... ,in It represents the state of charge (dimensionless, with a value ranging from 0 to 1). For Ohm internal resistance ( ), For polarization internal resistance ( ), The polarization capacitor is F.

[0027] The state equation for the extended Kalman filter is as follows: ; Polarization voltage equation: ; Terminal voltage observation equation: ; in This is the discharge current (A, positive for discharge). The polarization voltage is (V). The terminal voltage is (V). To use the pre-calibrated open-circuit voltage-state-of-charge lookup table, This refers to the battery's rated capacity (Ah).

[0028] Extended Kalman filter parameters: process noise covariance Observation noise covariance SOH passes through the ohmic internal resistance. Drift estimation: ; in , This is the reference value for the ohmic internal resistance of the new battery.

[0029] Verified through 50,000 Monte Carlo simulations (EKF estimate relative standard deviation approximately 1.2%): without SOH sensing, the energy consumption estimation error is approximately 14.9%; with SOH sensing, the average energy consumption estimation error is approximately 1.0%, not exceeding 2.4% within the 95% confidence interval. Taking a 22 Ah rated capacity battery with SOH=0.87 as an example, the corrected actual usable energy estimate is... Compared to directly using the rated capacity, the error is reduced by approximately 93.3%.

[0030] Path planning cost function ; satisfy

[0031] Among them, energy consumption cost ; For drone flight power model; safety costs ; The risk field constructed for step S3; task cost This represents the negative value of the path-to-target point sequence matching degree. Weight setting: For urban delivery scenarios... , , (Safety is the top priority).

[0032] Solving the path optimization problem using an improved covariance matrix adaptive evolution strategy: path parameter dimension ( (Number of path points, 3D coordinates of each path point), population size Initial step size The covariance matrix is ​​initialized to the identity matrix, the maximum number of iterations is 100, and the convergence condition is that the change in function value is less than 1 / 3. Global planning takes approximately 2.8 seconds (offline pre-calculation).

[0033] like Figure 5 As shown, in each control cycle Perform the following operations within: First, calculate the probability distribution of the current fusion environment state. The environment state vector The probability distribution is fitted by a Gaussian mixture model, and the number of components in the mixture model is... The parameters are estimated using the expectation-maximization algorithm. The information entropy is estimated using the Monte Carlo method. Number of sampling points .

[0034] Calculate the rate of change of information entropy (Unit: bit / s) according to The value triggers path correction at the corresponding level according to the following strategy: (a) Microscopic layer ( (Stable environment): Refit the current planned path with a fifth-order spline curve to eliminate curvature jumps on the path and ensure curvature continuity. The process takes approximately 0.005 seconds. This level is suitable for situations where the drone is flying normally along its path without any new threats, ensuring trajectory quality through continuous smoothing.

[0035] (b) Mesoscopic level ( Moderate environmental change): With the current location of the drone as the center and a radius... In a local area ( Take 3-5 s, with 3 s being the typical value. Approximately 8 m / s, then The local path replanning is performed using a covariance matrix adaptive evolution strategy (approximately 24 m), with a rolling step size of 10 m and a time consumption of approximately 0.12 s. This layer is suitable for situations where there are moderate-intensity dynamic obstacles or brief wind speed changes ahead. Simulation results show that compared to the single-layer method which waits 2.1 s for global replanning, the meso-level reduces the response latency in moderate threat scenarios by approximately 94% (from 2.1 s to 0.12 s), significantly reducing threat exposure time.

[0036] (c) Macro level ( (For large-scale or high-intensity environmental changes): Re-execute the global Dijkstra search on a sparse path map with a spacing of 50 m to generate a new globally optimal path, which takes approximately 2.1 seconds. This level is suitable for situations such as sudden strong gusts of wind, the appearance of a large number of new obstacles, or changes in mission objectives.

[0037] Simulation results from a 2300 m mission (7 sudden events, 2875 control cycles): Compared to the single-level fixed threshold method (threshold 0.10 bit / s, triggering 65 macroscopic replanning cycles), the three-level method triggers only 50 macroscopic replanning cycles, a reduction of approximately 23% (the remaining medium threats are handled by the meso-level, taking only 0.12 s instead of 2.1 s, resulting in faster threat response). The micro-level quintic spline curve smoothing ensures continuous output path curvature. ); sends path point coordinate sequences, corresponding suggested speed values, and energy consumption estimates to the flight control system at a period of 0.1 s; and records path execution deviations (position errors). and speed error The data is fed back to the multi-source information fusion module in real time to update the fusion status and form a closed-loop optimization.

Claims

1.S1: Acquire multi-source input information and perform spatiotemporal alignment preprocessing; the multi-source input information includes: The UAV's angular velocity ω (rad / s) and linear acceleration a (m / s²) output by the inertial measurement unit; the position p output by the global navigation satellite system. gnss With velocity v gnss ; 3D point cloud output by lidar; target distance d output by millimeter-wave radar k (m) and radial velocity v k (m / s); Wind speed v output by the meteorological sensor wind (m / s) and wind direction θ wind (rad); and task instructions; the spatiotemporal alignment is based on the lidar frame timestamp, and timestamp interpolation and unified coordinate transformation are performed on the data of each sensor, with the maximum time alignment deviation not exceeding 5 ms. 2.S2: A lightweight cross-modal Transformer is used to perform spatiotemporal fusion of multi-source information; the Transformer contains M cross-modal multi-head attention sublayers, where M is 2, 3, or 4; each modal data is mapped to a d-dimensional feature vector f by an independent encoder. i (i=1,…,N, where N is the number of modes, d=128), the cross-modal attention of the l-th layer is calculated as follows: ； In the formula: ,in, For cross-modal multi-head attention output; Let i be the query vector for the i-th modality. Let j be the key vector of the j-th mode. Let j be the value vector of the j-th mode; , These are the 128-dimensional feature vectors of the i-th and j-th modes, respectively. , , For learnable projection matrix; The dimension of the key vector; The activation function is a normalized exponential function. After structured pruning, the number of parameters in the fusion module does not exceed 30% of that of the standard Transformer, and the single-frame inference latency does not exceed 20 ms; the fusion output is: ； In the formula, It is a 128-dimensional unified environment state vector; The adaptive fusion weights for each modality are adaptively generated by global average pooling, satisfying the following conditions: ; The modal features output by the Lth layer Transformer; Modal index ( , (Total number of modes). 3.S3: Environmental state vector based on the output of step S2 Constructing a confidence-weighted dynamic risk field: ； In the formula, For spatial location Total risk field value at the location; These are three-dimensional spatial coordinates; For dynamic obstacle indexing; Let be the confidence coefficient of the i-th dynamic obstacle; For the first One dynamic obstacle risk component; This is the first dynamic obstacle; For terrain risk components; This represents the risk component of the wind farm. in, The confidence coefficient is: ； In the formula, It is a natural exponential function; For the first Standard deviation of the predicted position of each obstacle trajectory; For reference standard deviation, take . The dynamic obstacle risk component is: ； In the formula, For the first Predicted location of each obstacle; It is the 2-norm (spatial Euclidean distance); For the first An adaptive safe distance for obstacles. The adaptive safety distance is: ； In the formula, Basic safe distance; The velocity coefficient; For the first The speed of the obstacle's movement. Terrain risk component According to the terrain slope Segmented values: Slope Take 0 at time, Take 0.5 at a time. The value is 1.0, where This refers to the slope of the terrain. The wind farm risk components are: ； In the formula, For position Predicted wind speed; This is the maximum wind-resistant speed for the drone. 4.S4: Extended Kalman filter is used for online estimation of battery state of health (SOH); EKF state variables are... ,in In a charged state, For ohmic internal resistance, For polarization internal resistance, For polarized capacitors; the state equation is: ； In the formula, For the first Constant battery state of charge; For the first Discharge current at any time (A, discharge is positive); The sampling time interval; This refers to the battery's rated capacity. This indicates the battery's health status. The terminal voltage observation equation is: ； In the formula, This refers to the battery terminal voltage. This is the open-circuit voltage corresponding to the state of charge; Polarization voltage; It is the internal resistance of the Ohm. Process noise covariance Observation noise covariance SOH is estimated online by drift amount: ； In the formula, This represents the ohmic internal resistance drift. This is the reference value for the ohmic internal resistance of the new battery. Constructing an energy consumption-security collaborative cost function: ； In the formula, For path Total cost; The path scheme to be optimized; As a weight for energy consumption cost, Weighted by safety cost, As the task cost weight, satisfying ; This is the cost of energy consumption along the path; This comes at the cost of path security. The cost of the path task. in, The energy cost is: ； In the formula, This refers to the instantaneous flight power of the drone. For flight speed; For flight acceleration; The slope of the terrain along the flight path; Let be the path element length. The security cost is: ； In the formula, For path The spatial location of the population is determined using an improved covariance matrix adaptive evolution strategy. , For path parameter dimensions. 5.S5: In each control cycle Internal calculation of information entropy change rate: ； In the formula, The rate of change of information entropy (a replanning trigger indicator); for Probability distribution of environmental state at any given time; for Probability distribution of environmental state at any given time; To control the cycle. in, Information entropy is: ； In the formula, Entropy is the information of the probability distribution of environmental states. For the first The probability of each state; It is the natural logarithm. probability distribution From the fusion environment state vector Gaussian mixture model (components) The information entropy was obtained through fitting; it was estimated using the Monte Carlo method. ； In the formula, The number of sampling points in Monte Carlo is taken as... ; For the first Each sampling point. When No more than At that time, only the fifth-order spline curve trajectory smoothing of the micro-layer is performed to ensure that the curvature meets the requirements. ,in For path curvature, For the maximum permissible curvature; when At that time, the adaptive evolution strategy of the local covariance matrix of the meso-level is triggered to dynamically reprogram the window, with a rolling step size of 10 m and a local programming range. ,in For the local planning radius at the meso-level, The current flight speed of the drone. For prediction in the time domain (3–5 s); when At that time, initiate global Dijkstra sparse graph replanning at the macro level, with a node spacing of 50 m. 6.S6: Output the optimal path, send the path point sequence and speed suggestions to the flight control system at fixed intervals, and feed back the path execution deviation to the multi-source information fusion module in real time to form a closed-loop optimization.

Images