A deep learning-based unmanned aerial vehicle route intelligent planning and obstacle avoidance method

Abstract

The application relates to the technical field of unmanned aerial vehicle autonomous control, in particular to a method for unmanned aerial vehicle route intelligent planning and obstacle avoidance based on deep learning, which comprises the following steps: environment sensing and multi-scale feature fusion, data collection through an unmanned aerial vehicle on-board multi-sensor, input of specific convolutional neural network, generation of a dense grid map containing static obstacle elevation information and dynamic obstacle predicted trajectory probability, initial path generation, weighted fusion of obstacle information, construction of a cost function, generation of a flyable initial route by an improved A algorithm, closed-loop deep reinforcement learning control, realization of route tracking and dynamic obstacle avoidance by an Actor-Critic network, and online meta-reinforcement learning for adjusting strategies by a meta-learner to realize self-adaptation. The application improves the environment sensing precision of the unmanned aerial vehicle, the controllability and stability of the route and the environment adaptability, and is suitable for complex dynamic environments.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle autonomous control, in particular to an unmanned aerial vehicle route intelligent planning and obstacle avoidance method based on deep learning. BACKGROUND

[0002] In the technical field of unmanned aerial vehicle autonomous control, the current unmanned aerial vehicle route planning and obstacle avoidance method still has many deficiencies in practical application, and it is difficult to meet the precise control and safe flight requirements in complex dynamic environment.

[0003] From the perspective of environmental perception, the existing methods mostly rely on a single sensor to obtain environmental data, or the fusion processing of multi-source sensor data is not sufficient. Some schemes only use laser radar to obtain point cloud data, which easily ignores the visual details of the environment; or only rely on visual sensors, which are not robust in light changes and occlusion scenes; and when collecting multi-source data, there are often problems of time asynchronization and simple preprocessing methods, which make it difficult to accurately reflect the height characteristics of static obstacles and the motion trajectory probability of dynamic obstacles at the same time, and the precision and comprehensiveness of the environmental basic data provided for subsequent path planning are insufficient.

[0004] In the initial path generation link, traditional path planning algorithms such as conventional A algorithm mostly construct cost functions with static obstacle information, and pay little attention to the dynamic characteristics of dynamic obstacles, or do not dynamically adjust the obstacle weight according to the flight speed and safety distance of the unmanned aerial vehicle, resulting in low matching degree of path cost evaluation and actual environmental risk. At the same time, most algorithms do not fully consider the dynamic constraints of the unmanned aerial vehicle, and do not limit the node expansion range through dynamic window, so the generated path may exceed the maneuvering performance limit of the unmanned aerial vehicle, and there may be poor maneuverability problem, which is difficult to be directly applied to actual flight.

[0005] In terms of closed-loop control and obstacle avoidance, the state space design of the existing reinforcement learning control method often ignores the deviation information between the unmanned aerial vehicle and the planned route, or the action space definition does not match the actual control instruction requirements of the underlying flight controller, resulting in low route tracking accuracy. And some schemes do not introduce the maximum entropy reinforcement learning framework, and the strategy exploration is insufficient, when encountering unexpected dynamic obstacles, the control instruction cannot be quickly adjusted to achieve effective obstacle avoidance, and it is difficult to balance the route tracking stability and the timeliness of sudden obstacle avoidance.

[0006] The existing control strategies are mostly trained for specific scenes, and when the flight environment changes, large-scale data training needs to be performed again to adapt to the new scene, lacking online adaptive ability. And no meta-learning mechanism based on context information is constructed, which cannot extract task meta features from flight experience to dynamically adjust strategy update parameters, resulting in weak generalization ability of the strategy in unobserved flight environments, making it difficult to cope with complex and variable actual flight scenes, limiting the application range and autonomous control level of the unmanned aerial vehicle.

[0007] Therefore, an unmanned aerial vehicle route intelligent planning and obstacle avoidance method based on deep learning is proposed to solve the above problems. SUMMARY

[0008] The purpose of the present application is to provide an unmanned aerial vehicle route intelligent planning and obstacle avoidance method based on deep learning to solve the problems raised in the background art.

[0009] To achieve the above purpose, the present application provides the following technical solutions:

[0010] An unmanned aerial vehicle route intelligent planning and obstacle avoidance method based on deep learning, comprising the following steps:

[0011] Step S1, environment perception and multi-scale feature fusion: acquiring environment point cloud, image and pose data through the laser radar, visual sensor and IMU unit on board the unmanned aerial vehicle; inputting the acquired multi-source data into a convolutional neural network with a multi-scale encoder-decoder structure with skip connection for processing, the convolutional neural network extracting environment features of different receptive fields through a parallel hollow convolution module, and fusing to generate a dense grid map containing static obstacle elevation information and dynamic obstacle prediction trajectory probability;

[0012] Step S2, cooperative initial path generation: weighting and fusing the dynamic obstacle probability in the dense grid map with the static obstacle elevation gradient to construct a comprehensive path cost function; using an improved A algorithm with a fusion dynamic window to perform path search with the comprehensive path cost function as the core factor of the heuristic function, combined with the dynamic constraints of the unmanned aerial vehicle, to generate a feasible initial route that meets the maneuvering performance of the unmanned aerial vehicle;

[0013] Step S3, closed-loop deep reinforcement learning control: an Actor-Critic network based on a maximum entropy reinforcement learning framework is constructed; the state space of the Actor-Critic network is defined as a combined state vector composed of a dense grid map, a real-time state vector of the unmanned aerial vehicle, and lateral deviation, height deviation, and speed deviation from the flyable initial route; during the flight of the unmanned aerial vehicle, the Actor-Critic network outputs control instructions through the policy network of the Actor-Critic network according to the real-time state, to drive the unmanned aerial vehicle to form closed-loop tracking of the flyable initial route, and simultaneously avoid real-time dynamic obstacles;

[0014] Step S4, online meta-reinforcement learning: during the execution of the task by the unmanned aerial vehicle, a meta-learner based on context information is constructed; the meta-learner continuously extracts task meta-features from flight experience data, and dynamically adjusts the policy update gradient direction and step length of the Actor-Critic network, to realize rapid online self-adaptation and generalization of the control policy in different flight environments.

[0015] As a preferred scheme, the environment perception and multi-scale feature fusion of step S1 include:

[0016] Step S1.1: synchronously collecting environment point cloud data, image data, and pose data through the laser radar, visual sensor, and IMU unit on board the unmanned aerial vehicle;

[0017] Step S1.2: performing denoising and gridding processing on the environment point cloud data to generate a point cloud elevation map; performing distortion correction and color normalization processing on the image data to generate a standardized image; and performing filtering and interpolation processing on the IMU pose data to obtain a continuous pose sequence of the unmanned aerial vehicle;

[0018] Step S1.3: inputting the point cloud elevation map, the standardized image, and the continuous pose sequence into a convolutional neural network with a multi-scale encoder-decoder structure with skip connection, wherein the encoder part is composed of multiple convolutional layers and pooling layers, for layer-by-layer extraction of environment features, and the decoder part is composed of multiple deconvolutional layers and up-sampling layers, for reconstruction of feature maps;

[0019] Step S1.4: the convolutional neural network processes the features extracted by the encoder through a parallel atrous convolution module, wherein the parallel atrous convolution module contains multiple parallel branches, each branch uses an atrous convolution kernel with different dilation rates to extract environment features with different receptive fields;

[0020] Step S1.5: Element-wise addition fusion is performed between the multi-scale feature maps output by the parallel hole convolution module and the feature maps of the corresponding layers in the decoder through the jump connection, forming an enhanced multi-scale feature representation;

[0021] Step S1.6: The enhanced multi-scale feature is input into the output layer of the convolutional neural network, and a dense grid map containing static obstacle elevation information and dynamic obstacle predicted trajectory probability is generated through the Softmax activation function.

[0022] As a preferred solution, the step S2 of constructing the integrated path cost function includes the following steps:

[0023] Step S2.1: Extracting the dynamic obstacle predicted trajectory probability distribution map and the static obstacle elevation gradient map from the dense grid map;

[0024] Step S2.2: Probability normalization processing is performed on the dynamic obstacle predicted trajectory probability distribution map to obtain a normalized dynamic obstacle probability map;

[0025] Step S2.3: Gradient amplitude calculation and normalization processing are performed on the static obstacle elevation gradient map to obtain a normalized static obstacle elevation gradient map;

[0026] Step S2.4: According to the flight speed of the unmanned aerial vehicle and the safety distance requirement, the weight coefficients of the dynamic obstacle probability and the static obstacle elevation gradient are dynamically adjusted;

[0027] Step S2.5: The normalized dynamic obstacle probability map and the normalized static obstacle elevation gradient map are weighted and fused according to the weight coefficients to generate an integrated path cost map, which is used as the spatial representation of the integrated path cost function.

[0028] As a preferred solution, the step S2 of adopting the improved A algorithm with fusion dynamic window for path search includes the following steps:

[0029] Step S2.6: Based on the integrated path cost map, a search graph is constructed, wherein each node in the search graph corresponds to a grid position in the dense grid map, and each node is assigned an integrated path cost value;

[0030] Step S2.7: The open list and the closed list of the improved A algorithm are initialized, the starting position node of the unmanned aerial vehicle is added to the open list, and the actual cost value of the starting node is set to zero;

[0031] Step S2.8: Defining the heuristic function of the improved A algorithm, which is composed of the weighted sum of the Euclidean distance from the current node to the target node and the integrated path cost value of the current node, wherein the weight coefficient is dynamically adjusted according to the flight task requirement of the unmanned aerial vehicle;

[0032] Step S2.9: According to the real-time state and dynamic constraints of the UAV, a dynamic window is calculated, which defines the speed and position space range that the UAV can reach in a short time, and limits the node expansion range of the A algorithm based on the dynamic window;

[0033] Step S2.10: Select the node with the minimum heuristic function value from the open list as the current expansion node, check whether the current expansion node falls within the dynamic window, if it falls within the dynamic window, expand the node, otherwise skip the node;

[0034] Step S2.11: Traverse the neighbor nodes of the current expansion node, calculate the actual generation value and heuristic function value of each neighbor node, and update the open list and closed list;

[0035] Step S2.12: Repeat steps S2.10 and S2.11 until the open list is empty or the target node is successfully searched, thereby generating a feasible initial flight path from the starting position to the target position.

[0036] As a preferred solution, the step S3 of constructing an Actor-Critic network based on a maximum entropy reinforcement learning framework includes the following steps:

[0037] Step S3.1: State space vector construction: compress and vectorize the dense grid map to obtain an environmental feature vector; At the same time, normalize the real-time state vector of the UAV, the lateral deviation, the height deviation and the speed deviation of the feasible initial flight path to obtain a deviation feature vector; The environmental feature vector and the deviation feature vector are spliced and fused to form a combined state vector;

[0038] Step S3.2: Action space vector definition: according to the dynamic performance limit of the UAV, define the action space vector as three-axis acceleration instruction and yaw rate instruction, and standardize the value range of each action component;

[0039] Step S3.3: Actor network construction: construct a policy network based on a deep neural network, the input layer of the policy network receives the combined state vector, the feature is transformed through multiple fully connected hidden layers, and the output layer uses the Softmax activation function to generate the probability distribution of the action space vector;

[0040] Step S3.4: Critic network construction: construct a value network based on a deep neural network, the input layer of the value network receives the combined state vector, the value is estimated through multiple fully connected hidden layers, and the output layer generates a state value scalar;

[0041] Step S3.5: Maximum entropy objective function design: In the reinforcement learning framework, a maximum entropy objective function is designed, which contains a cumulative reward and a policy entropy term, where the policy entropy term is used to adjust the exploratory of the policy;

[0042] Step S3.6: Network parameter optimization: Using the state-action-reward sequence data collected in real time during the flight of the unmanned aerial vehicle, the parameters of the Actor network and the Critic network are updated by the stochastic gradient descent method to maximize the maximum entropy objective function.

[0043] As a preferred solution, during the flight of the unmanned aerial vehicle, the Actor-Critic network outputs control instructions through the policy network of the Actor-Critic network according to the real-time state, driving the unmanned aerial vehicle to form a closed-loop tracking of the flyable initial route, and at the same time, real-time avoidance of unanticipated dynamic obstacles, including the following steps:

[0044] Step S3.7: Real-time state acquisition and vector construction: During the flight of the unmanned aerial vehicle, the dense grid map, the real-time state vector of the unmanned aerial vehicle, and the lateral deviation, height deviation and speed deviation from the flyable initial route are acquired in real time; the dense grid map is compressed and vectorized to obtain the environmental feature vector, and the real-time state vector of the unmanned aerial vehicle and the deviation vector are normalized to obtain the deviation feature vector; the environmental feature vector and the deviation feature vector are spliced and fused to form a combined state vector;

[0045] Step S3.8: Forward propagation of policy network: The combined state vector is input into the policy network of the Actor-Critic network, and the feature is transformed through multiple fully connected hidden layers of the policy network, and finally the probability distribution of the action space vector is generated by the output layer using the Softmax activation function;

[0046] Step S3.9: Control action decision: According to the probability distribution of the action space vector, a random sampling strategy based on maximum entropy is used to select a specific control action to generate standardized three-axis acceleration instructions and yaw rate instructions;

[0047] Step S3.10: Control instruction transmission and execution: The generated three-axis acceleration instructions and yaw rate instructions are sent to the bottom flight controller of the unmanned aerial vehicle, and the bottom flight controller parses the instructions and drives the unmanned aerial vehicle to execute the corresponding flight action;

[0048] Step S3.11: Closed-loop tracking and dynamic obstacle avoidance: By continuously repeating steps S3.7 to S3.10, the unmanned aerial vehicle realizes closed-loop tracking of the flyable initial route, and at the same time, dynamically adjusts the control instructions according to the real-time environmental perception data to avoid unanticipated dynamic obstacles;

[0049] Step S3.12: Flight experience collection and storage: During the UAV flight, the state-action-reward sequence data is collected in real time, and these data are stored into the experience replay buffer, providing data basis for the online optimization of the subsequent Actor-Critic network.

[0050] As a preferred solution, the online meta-reinforcement learning of step S4 includes the following steps:

[0051] Step S4.1, flight experience data preprocessing: real-time acquisition of flight experience data from the experience replay buffer of the Actor-Critic network, the flight experience data including state-action-reward sequence data; time series alignment and normalization processing of the flight experience data, generating standardized flight experience data;

[0052] Step S4.2, context information extraction: inputting the standardized flight experience data into a context encoder based on a recurrent neural network, capturing the time series dependence in the flight task through the hidden state of the context encoder, and outputting a fixed-dimension context information vector;

[0053] Step S4.3, task meta-feature calculation: inputting the context information vector into a fully connected neural network, performing nonlinear transformation through multiple hidden layers of the fully connected neural network, and extracting a task meta-feature vector, which encodes the key statistical properties and dynamic patterns of the current flight environment;

[0054] Step S4.4, meta-learner parameter generation: inputting the task meta-feature vector into the meta-learner, which is composed of a multilayer perceptron, generating policy update parameters through the output layer of the multilayer perceptron, the policy update parameters including a gradient direction adjustment vector and a step scaling factor for the Actor-Critic network policy update;

[0055] Step S4.5, dynamic adjustment of policy update: in the parameter optimization process of the Actor-Critic network, the policy update parameters are used to modify the update rule of the stochastic gradient descent method, the dot product operation of the gradient direction adjustment vector and the original policy gradient is performed to obtain the adjusted policy gradient direction, and the step scaling factor is multiplied by the original learning rate to obtain an adaptive learning rate;

[0056] Step S4.6, online adaptive control policy: based on the adjusted policy gradient direction and the adaptive learning rate, the policy network and value network parameters of the Actor-Critic network are updated to realize the rapid online adaptation of the control policy in the current flight environment and improve the generalization ability of the policy in unobserved flight environments.

[0057] From the above technical solutions provided by the present application, the present application provides a kind of unmanned aerial vehicle route intelligent planning and obstacle avoidance method based on deep learning, and the beneficial effects are:

[0058] Through the synchronous acquisition of environment point cloud data, image data and pose data by the on-board laser radar, visual sensor and IMU unit of the unmanned aerial vehicle, after preprocessing such as denoising, gridding, distortion correction, filtering and interpolation, the data is input into the convolutional neural network with a multi-scale encoder-decoder structure with skip connection, and the environment features of different receptive fields are extracted by combining the parallel hollow convolution module, and finally a dense grid map containing static obstacle elevation information and dynamic obstacle predicted trajectory probability is generated;This process effectively integrates multi-source environment information, avoids the limitations of single sensor data, ensures that the environment perception results cover both the elevation details of static obstacles and the trajectory probability of dynamic obstacles, and provides accurate and comprehensive environment data support for subsequent initial path generation;

[0059] The present application first extracts the dynamic obstacle predicted trajectory probability distribution map and the static obstacle elevation gradient map from the dense grid map, and then dynamically adjusts the weight coefficient according to the flight speed and safety distance requirements of the unmanned aerial vehicle after normalization processing, and generates a comprehensive path cost map by weighted fusion to construct a comprehensive path cost function;Then, an improved A algorithm with a dynamic window is used, the comprehensive path cost function is used as the core factor of the heuristic function, the dynamic window is calculated based on the real-time state and dynamics constraints of the unmanned aerial vehicle to limit the node expansion range, and finally a flyable initial route is searched and generated;This process not only considers the environmental risks of static and dynamic obstacles, but also ensures that the route meets the maneuvering performance of the unmanned aerial vehicle through dynamic window and dynamics constraints, avoiding the problem of route exceeding the capability of the unmanned aerial vehicle in traditional path planning, and improving the actual flyability of the initial route;

[0060] The present application constructs an Actor-Critic network based on a maximum entropy reinforcement learning framework, the state space is composed of a dense grid map, a real-time state vector of the unmanned aerial vehicle and lateral deviation, height deviation and speed deviation from the initial route, and the action space is defined as the three-axis acceleration and yaw rate command of the bottom flight controller of the unmanned aerial vehicle;During flight, the network acquires and processes state data in real time, outputs control commands through the policy network and transmits them to the bottom controller, and continuously repeats the process to realize closed-loop tracking of the initial route;At the same time, it can dynamically adjust the control command according to the real-time environment perception data, effectively avoid unexpected dynamic obstacles, solve the problem that route tracking and sudden obstacle avoidance are difficult to balance in traditional control methods, and ensure the stability and safety of the flight process;

[0061] This invention constructs a context-based meta-learner during UAV mission execution. Flight experience data is acquired from the experience replay buffer of the Actor-Critic network, preprocessed, and then extracted into a context information vector via a recurrent neural network's context encoder. This vector is then passed through a fully connected neural network to obtain the task meta-feature vector. The meta-learner generates policy update parameters based on this vector, dynamically adjusting the policy update gradient direction and step size of the Actor-Critic network to update the network parameters. This design allows the control policy to quickly adapt online in different flight environments without large-scale retraining, while also improving generalization ability in unseen environments. This enhances the UAV's adaptability to complex and ever-changing flight scenarios and expands its application scope. Attached Figure Description

[0062] Figure 1 This is a schematic diagram illustrating the steps of a deep learning-based intelligent flight path planning and obstacle avoidance method for unmanned aerial vehicles (UAVs) according to the present invention. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0064] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific embodiments.

[0065] like Figure 1 As shown, this embodiment of the invention provides a method for intelligent flight path planning and obstacle avoidance for unmanned aerial vehicles (UAVs) based on deep learning, including the following steps:

[0066] Step S1, Environmental Perception and Multi-Scale Feature Fusion: Environmental point cloud, image and pose data are acquired through the airborne lidar, visual sensor and IMU unit of the UAV; the acquired multi-source data is input into a convolutional neural network with a multi-scale encoder-decoder structure with skip connections for processing. The convolutional neural network extracts environmental features of different receptive fields through parallel dilated convolution modules and fuses them to generate a dense grid map containing static obstacle elevation information and dynamic obstacle predicted trajectory probability.

[0067] Step S2, Collaborative Initial Path Generation: The dynamic obstacle probabilities and static obstacle elevation gradients in the dense grid map are weighted and fused to construct a comprehensive path cost function; an improved A-type dynamic window is adopted. The algorithm uses the comprehensive path cost function as the core factor of the heuristic function, combines the dynamic constraints of the UAV to perform path search, and generates an initial flight path that meets the maneuverability performance of the UAV.

[0068] Step S3, closed-loop deep reinforcement learning control: an Actor-Critic network based on a maximum entropy reinforcement learning framework is constructed; the state space of the Actor-Critic network is defined as a combined state vector composed of a dense grid map, a real-time state vector of the unmanned aerial vehicle, and lateral deviation, height deviation, and speed deviation from the flyable initial route; the action space of the Actor-Critic network is defined as the three-axis acceleration and yaw rate command of the bottom flight controller of the unmanned aerial vehicle; during the flight of the unmanned aerial vehicle, the Actor-Critic network outputs control instructions through the policy network of the Actor-Critic network according to the real-time state, to drive the unmanned aerial vehicle to form a closed-loop tracking of the flyable initial route, and at the same time to avoid real-time dynamic obstacles;

[0069] Step S4, online meta-reinforcement learning: during the execution of the task by the unmanned aerial vehicle, a meta-learner based on context information is constructed; the meta-learner continuously extracts task meta-features from flight experience data, and dynamically adjusts the policy update gradient direction and step size of the Actor-Critic network, to realize rapid online adaptation and generalization of the control policy in different flight environments.

[0070] In this embodiment, the function of step S1 is to obtain environmental data through the multi-source sensors on board the unmanned aerial vehicle, to input the preprocessed data into a convolutional neural network with a specific structure, to generate a dense grid map containing static obstacle elevation information and dynamic obstacle prediction trajectory probability through multi-scale feature extraction and fusion, to provide accurate and comprehensive environmental perception basis for subsequent initial route generation; the following are the detailed steps:

[0071] Step S1.1: Synchronous acquisition of multi-source environmental data:

[0072] Multi-sensor data acquisition: through the laser radar, visual sensor, and IMU unit on board the unmanned aerial vehicle, three types of core environmental and self-state data are synchronously acquired, wherein the laser radar acquires environmental point cloud data, the visual sensor acquires environmental image data, and the IMU unit acquires the pose data of the unmanned aerial vehicle itself;

[0073] Synchronization guarantee: ensure that the data acquisition processes of the three types of sensors are kept synchronous in the time dimension, to avoid mismatch of environmental information and pose information due to time difference in data acquisition, and to provide a time consistency basis for subsequent data fusion and processing;

[0074] Step S1.2: Preprocessing of multi-source acquisition data:

[0075] Point cloud data processing: denoising the environmental point cloud data collected by the laser radar, removing invalid point clouds caused by sensor noise and environmental interference; then rasterizing the denoised point cloud data, mapping the three-dimensional point cloud to a two-dimensional grid space, generating a point cloud elevation map, and intuitively reflecting the elevation information of different positions in the environment;

[0076] Image data processing: correcting the image distortion caused by the optical characteristics of the lens by correcting the image distortion caused by the optical characteristics of the lens; then performing color normalization processing to eliminate the influence of light changes on image color features, generating a standardized image to ensure the stability of image features;

[0077] IMU pose data processing: filtering the pose data collected by the IMU unit to reduce the interference of random noise on the pose data; then performing interpolation processing to fill in the gaps in the pose data collection interval, obtaining a continuous pose sequence of the UAV, accurately reflecting the position and attitude changes of the UAV during the collection process;

[0078] Step S1.3: input the preprocessed data into the multi-scale encoder-decoder convolutional neural network:

[0079] Network structure introduction: input the point cloud elevation map, standardized image and continuous pose sequence generated in step S1-2 into the convolutional neural network with a multi-scale encoder-decoder structure with skip connection;

[0080] Encoder function implementation: the encoder part of the convolutional neural network is composed of multiple convolutional layers and pooling layers, the convolutional layers extract local features in the input data through convolution kernels, and the pooling layers downsample the feature maps, retaining key features and reducing data dimensions, realizing the extraction of environmental features layer by layer from shallow basic features to deep abstract features;

[0081] Decoder function implementation: the decoder part is composed of multiple deconvolutional layers and up-sampling layers, the deconvolutional layers restore the dimensionality of the deep feature maps output by the encoder, and the up-sampling layers improve the resolution of the feature maps, realizing the reconstruction of the feature maps and providing adaptive feature dimensions for subsequent multi-scale feature fusion;

[0082] Step S1.4: parallel hole convolution module extracts multi-receptive field environmental features:

[0083] Parallel branch structure setting: the convolutional neural network processes the features extracted by the encoder through the built-in parallel hole convolution module, which contains multiple parallel branches, each branch is configured with a hole convolution kernel with different dilation rates;

[0084] Multi-receptive field feature extraction: different dilated convolution kernels with different expansion rates expand the receptive field of the convolution operation without increasing the amount of calculation. Among them, the low expansion rate convolution kernel extracts the local detail features in the environment, and the high expansion rate convolution kernel extracts the global macro features in the environment. Through multi-branch parallel operation, the comprehensive extraction of environmental features with different receptive fields is realized.

[0085] Step S1.5: multi-scale feature jump connection fusion:

[0086] Feature fusion method: the multi-scale feature maps output by the parallel dilated convolution module are element-wise added and fused with the feature maps of the corresponding level in the decoder through jump connection.

[0087] Fusion effect improvement: the jump connection directly transmits the shallow features extracted by the encoder to the corresponding layer of the decoder, and combines with the deep features reconstructed by the decoder, making up for the loss of details in the feature reconstruction process of the decoder, forming enhanced multi-scale feature representation, which contains both global environmental information and local detail information.

[0088] Step S1.6: dense grid map generation:

[0089] Feature input and activation processing: the enhanced multi-scale features formed in step S1.5 are input into the output layer of the convolutional neural network, and the output layer uses the Softmax activation function to process the feature data, mapping the feature value to a probability distribution.

[0090] Grid map output: after processing by the Softmax activation function, a dense grid map is generated, which contains two types of core information: static obstacle height information, reflecting the height distribution of static obstacles in the environment; and dynamic obstacle prediction trajectory probability, reflecting the probability of dynamic obstacles appearing at different grid positions, providing environmental risk assessment basis for subsequent path planning.

[0091] In this embodiment, the function of step S2 is to construct a comprehensive path cost function by weighting and fusing the obstacle information in the dense grid map, and then combining with the dynamics constraints of the UAV, using the improved A algorithm with fusion dynamic window to search for a path, finally generating a feasible initial flight path that meets the maneuvering performance of the UAV, providing a basic path reference for subsequent closed-loop control. The following are the detailed steps:

[0092] Step S2.1: extract obstacle-related feature maps:

[0093] From the dense grid map generated in step S1, two types of key information maps are extracted respectively; the first type is the dynamic obstacle predicted trajectory probability distribution map, which reflects the probability distribution of the dynamic obstacle appearing at different grid positions; the second type is the static obstacle elevation gradient map, which reflects the change rate of the elevation value of the static obstacle region, and the greater the elevation gradient, the more significant the terrain undulation or the more intense the obstacle height change in the static obstacle region;

[0094] Step S2.2: Dynamic obstacle probability normalization processing:

[0095] The extracted dynamic obstacle predicted trajectory probability distribution map is subjected to probability normalization processing; by mapping the probability value of each grid in the map to the interval of 0 to 1, the influence of the magnitude difference of different dynamic obstacle probability values is eliminated, so that all grids have a unified comparison basis for dynamic obstacle probability, and a normalized dynamic obstacle probability map is obtained after processing;

[0096] Step S2.3: Static obstacle elevation gradient normalization processing:

[0097] The extracted static obstacle elevation gradient map is subjected to two-step processing; the first step is to calculate the gradient amplitude of each grid in the map, which quantifies the change intensity of the elevation value at the grid position; the second step is to normalize the calculated gradient amplitude, which is also mapped to the interval of 0 to 1, to eliminate the influence of the difference in elevation gradient amplitude in different regions, and a normalized static obstacle elevation gradient map is obtained after processing;

[0098] Step S2.4: Dynamic adjustment of weight coefficients:

[0099] According to the current flight speed of the unmanned aerial vehicle and the preset safety distance requirement, two weight coefficients are dynamically determined, corresponding to the weight coefficient of the normalized dynamic obstacle probability map and the weight coefficient of the normalized static obstacle elevation gradient map respectively; when the flight speed of the unmanned aerial vehicle is faster, the dynamic obstacle poses a greater threat to flight safety, so the weight coefficient corresponding to the dynamic obstacle probability is increased; when the safety distance requirement is more stringent, the influence of the static obstacle is more prominent, so the weight coefficient corresponding to the static obstacle elevation gradient is increased, ensuring that the weight distribution matches the actual flight safety demand;

[0100] Step S2.5: Generating a comprehensive path cost map:

[0101] The normalized dynamic obstacle probability map and the normalized static obstacle elevation gradient map are fused by weighting operation according to the weight coefficient determined in step S2.4; the comprehensive cost of each grid is equal to the value of the grid in the normalized dynamic obstacle probability map multiplied by the corresponding weight coefficient, plus the value of the grid in the normalized static obstacle elevation gradient map multiplied by the corresponding weight coefficient; after fusion, a comprehensive path cost map is generated, which is the spatial representation of the comprehensive path cost function, and the higher the comprehensive cost of the grid, the greater the flight risk of the position;

[0102] Step S2.6: constructing a path search graph:

[0103] A search graph is constructed based on the comprehensive path cost map generated in step S2.5; each node in the search graph corresponds to one grid position in the dense grid map, and each node is assigned a comprehensive path cost value, which directly adopts the comprehensive cost of the corresponding grid in the comprehensive path cost map, so that the search graph can directly reflect the flight risk of different positions;

[0104] Step S2.7: initializing improved A Algorithm list:

[0105] Initialize improved A Open list and closed list required by the algorithm; the open list is used to store the nodes to be processed, and the closed list is used to store the nodes that have been processed; the node corresponding to the current starting position of the unmanned aerial vehicle is added to the open list, and the actual cost value of the starting node is set to zero, which represents the cumulative path cost from the starting node to the current node;

[0106] Step S2.8: defining improved A Algorithm heuristic function:

[0107] Define improved A The heuristic function of the algorithm is composed of two parts weighted summation; the first part is the Euclidean distance from the current node to the target node, which is used to estimate the straight line distance cost from the current node to the target node; the second part is the comprehensive path cost value of the current node, which is used to reflect the flight risk cost of the current position; the weight coefficients of the two parts are dynamically adjusted according to the flight task requirements of the unmanned aerial vehicle; if the task prioritizes path efficiency, the weight of the Euclidean distance can be increased; if the task prioritizes flight safety, the weight of the comprehensive path cost value can be increased;

[0108] Step S2.9: calculating dynamic window:

[0109] According to the real-time state of the UAV and dynamic constraints, a dynamic window is calculated; the real-time state of the UAV includes current speed, acceleration, etc., and the dynamic constraints include maximum speed, maximum acceleration, maximum angular velocity, etc.; the dynamic window defines the speed range and position range that the UAV can reach in a short time, and through the dynamic window, the node expansion range of the improved A* algorithm can be limited, and it is ensured that the position corresponding to the expanded node is a position that the UAV can actually reach under the dynamic constraints;

[0110] Step S2.10: screening and expanding nodes

[0111] The node with the minimum heuristic function value is selected from the open list as the current expansion node, and it is first checked whether the position corresponding to the current expansion node falls within the dynamic window calculated in step S2.9; if the node falls within the dynamic window, the node is expanded to generate neighbor nodes of the node; if the node does not fall within the dynamic window, the node is skipped and not expanded, and the next node with the minimum heuristic function value is selected from the open list;

[0112] Step S2.11: updating the algorithm list

[0113] All neighbor nodes of the current expansion node are traversed, and the actual generation value and the heuristic function value of each neighbor node are calculated; the actual generation value of the neighbor node is equal to the actual generation value of the current expansion node plus the path cost from the current expansion node to the neighbor node, and the heuristic function value of the neighbor node is calculated according to the definition of step S2.8; according to the calculation result, the open list and the closed list are updated: if the neighbor node is not in any list, it is added to the open list; if the neighbor node is in the open list and the newly calculated actual generation value is smaller, the actual generation value is updated; if the neighbor node is in the closed list and the newly calculated actual generation value is smaller, it is moved back from the closed list to the open list and the actual generation value is updated;

[0114] Step S2.12: generating a flyable initial flight path

[0115] Steps S2.10 and S2.11 are repeatedly executed to continuously screen, expand and update the list of nodes; when the open list is empty, it means that the path from the starting node to the target node cannot be searched; when the node corresponding to the target node is searched, the search is stopped, and a complete path is formed by backtracking from the target node to the starting node, which is a flyable initial flight path that meets the maneuvering performance of the UAV.

[0116] In this embodiment, the role of step S3 is to construct an Actor-Critic network based on a maximum entropy reinforcement learning framework, to output control instructions in the process of unmanned aerial vehicle flight by defining reasonable state space and action space, to realize closed-loop tracking of the flyable initial route, and to avoid dynamic obstacles in real time to ensure the safety and stability of the flight path. The following are the detailed steps:

[0117] Step S3.1: State space vector construction:

[0118] The dense grid map generated in step S1 is subjected to feature compression and vectorization processing, and the two-dimensional grid form of environmental information is converted into a one-dimensional vector to obtain an environmental feature vector. At the same time, the real-time state vector of the unmanned aerial vehicle, and the lateral deviation, height deviation and speed deviation between the unmanned aerial vehicle and the flyable initial route are obtained, and these real-time state vectors and deviation data are subjected to normalization processing to eliminate the influence of different physical quantity magnitude differences to obtain a deviation feature vector. The environmental feature vector and the deviation feature vector are spliced and fused to form a combined state vector containing environmental information, unmanned aerial vehicle state and path deviation, which is used as the input basis of the Actor-Critic network.

[0119] Step S3.2: Action space vector definition:

[0120] According to the dynamic performance limit of the unmanned aerial vehicle, the composition of the action space vector is determined as three-axis acceleration instruction and yaw angle rate instruction. The three-axis acceleration instruction corresponds to the acceleration control quantity of the unmanned aerial vehicle in the X-axis, Y-axis and Z-axis directions in the three-dimensional space, and the yaw angle rate instruction corresponds to the change rate control quantity of the heading angle of the unmanned aerial vehicle. The value range of each action component is standardized to map the component values to a unified interval, ensuring that the action instruction conforms to the hardware execution capability of the unmanned aerial vehicle and avoiding exceeding the dynamic limit.

[0121] Step S3.3: Actor network construction:

[0122] A policy network based on a deep neural network is constructed as the Actor network. The input layer of the network receives the combined state vector formed in step S3.1, and is connected to multiple fully connected hidden layers after the input layer. The activation function in the hidden layer is used to transform the input combined state vector to extract key features related to action decision. The output layer of the network uses a Softmax activation function to convert the features output by the hidden layer into a probability distribution of each action component in the action space vector. The probability distribution reflects the possibility of selecting different actions under the current state, providing a basis for action decision.

[0123] Step S3.4: Critic network construction:

[0124] A value network based on a deep neural network is constructed as a Critic network; the input layer of the network also receives the combined state vector formed in step S3.1, and is connected to multiple fully connected hidden layers after the input layer, and the value of the current state is estimated through the operation of the hidden layers, that is, the expected cumulative return obtained by executing the subsequent optimal action in the current state is evaluated; the output layer of the network does not use an activation function, and directly generates a state value scalar, which quantifies the value of the current state, providing value feedback for the policy optimization of the Actor network;

[0125] Step S3.5: Maximum entropy objective function design:

[0126] In the reinforcement learning framework, a maximum entropy objective function is designed, which includes a cumulative return and a policy entropy term; the cumulative return term is the cumulative income calculated by the preset reward function after the UAV executes a series of actions, and the reward function is usually set according to whether the UAV deviates from the flight path, whether it is close to an obstacle, whether it completes the flight task, etc., and the cumulative return term guides the network to learn the action strategy that can obtain high income; the policy entropy term is calculated from the action probability distribution output by the Actor network, and the greater the entropy value, the stronger the randomness of action selection, and the introduction of the policy entropy term is used to adjust the exploratory nature of the policy, avoid the network from falling into a local optimal policy, and ensure that better actions can still be explored in unknown environments;

[0127] Step S3.6: Network parameter optimization:

[0128] During the flight of the UAV, state-action-reward sequence data is collected in real time, which includes the combined state vector of the UAV at a certain time, the action vector selected based on the state, and the immediate reward obtained after the action is executed; the collected state-action-reward sequence data is stored as a training sample for network parameter optimization; the stochastic gradient descent method is used to maximize the maximum entropy objective function designed in step S3.5 as the optimization objective, and the weights, biases and other parameters of the Actor network and the Critic network are iteratively updated; through continuous optimization, the action strategy output by the Actor network is more likely to obtain high cumulative return, and the estimation of the Critic network on the state value is more accurate;

[0129] Step S3.7: Real-time state acquisition and vector construction:

[0130] During the flight of the UAV, three types of key data are acquired in real time: first, the update data of the dense grid map generated in step S1, reflecting the dynamic changes of obstacles in the current environment; second, the real-time state vector of the UAV, including data such as position, speed, acceleration, attitude angle, etc.; third, the lateral deviation, height deviation and speed deviation data between the UAV and the initial flight path of the flyability; the real-time environment feature vector is obtained by performing feature compression and vectorization processing on the real-time dense grid map; the real-time deviation feature vector is obtained by normalizing the real-time state vector and the deviation data; the real-time environment feature vector and the real-time deviation feature vector are spliced and fused to form a real-time combined state vector, which provides input for real-time action decision-making;

[0131] Step S3.8: Forward propagation of the policy network

[0132] The real-time combined state vector formed in step S3.7 is input into the policy network of the Actor-Critic network to start the forward propagation process of the policy network; the real-time combined state vector first enters the input layer, and then passes through each fully connected hidden layer in turn, and the feature transformation is gradually completed in the hidden layer through the operation of the weight matrix and the activation function, and the action decision-making features matching the current real-time state are extracted; finally, the features processed by all hidden layers are transmitted to the output layer, and the probability distribution of the action space vector under the current real-time state is generated through the Softmax activation function operation, which reflects the selection probability of each action under the current state;

[0133] Step S3.9: Control action decision

[0134] According to the action space vector probability distribution generated in step S3.8, a specific control action is selected by using a random sampling strategy based on maximum entropy; this sampling strategy, based on the probability distribution, combines the influence of the policy entropy term, tends to select actions with higher probability to ensure the stability of the decision, and retains a certain randomness to cope with the dynamic changes of the environment; according to the sampling result, the standardized three-axis acceleration command and yaw rate command are generated to ensure that the command conforms to the receiving format and execution requirements of the UAV bottom controller;

[0135] Step S3.10: Control command transmission and execution

[0136] The three-axis acceleration command and yaw rate command generated in step S3.9 are sent to the bottom flight controller of the UAV through the data transmission channel; after receiving the command, the bottom flight controller analyzes and processes the command, converts the command into hardware control signals such as motor speed and rudder angle, drives the power system and attitude adjustment system of the UAV to execute the corresponding flight action, and realizes acceleration adjustment or heading angle adjustment, so that the UAV changes the flight state according to the control command;

[0137] Step S3.11: Closed-loop tracking and dynamic obstacle avoidance:

[0138] Steps S3.7 to S3.10 are continuously repeated to form a closed-loop control process; in this process, the UAV continuously updates the combined state vector by obtaining real-time environmental and self-state data, generates new control instructions, adjusts the flight state, ensures stable tracking of the initial flight path, and reduces path deviation; at the same time, when real-time environmental data shows the presence of an unexpected dynamic obstacle, the Actor-Critic network adjusts the action probability distribution according to the position and motion trend of the obstacle to generate evasive control instructions, causing the UAV to change its flight path or speed and achieve real-time avoidance of dynamic obstacles;

[0139] Step S3.12: Flight experience collection and storage:

[0140] During the flight of the UAV, the state-action-reward sequence data in each closed-loop control process is collected in real time, which includes the real-time combined state vector in step S3.7, the control action instruction in step S3.9, and the immediate reward value calculated according to the reward function after the action is executed; these collected state-action-reward sequence data are stored in the experience replay buffer in chronological order, which provides continuous training data support for the network parameter optimization in step S3.6, and also provides a basis for flight experience data for subsequent online meta-learning in step S4.

[0141] In this embodiment, the role of step S4 is to extract task meta-features from flight experience data by constructing a meta-learner based on context information during the execution of the UAV task, dynamically adjusting the policy update gradient direction and step size of the Actor-Critic network, achieving rapid online adaptation and generalization of the control policy in different flight environments, and improving the adaptability of the UAV in complex and variable environments; the following are the detailed steps:

[0142] Step S4.1, Flight experience data preprocessing:

[0143] Flight experience data is obtained in real time from the experience replay buffer of the Actor-Critic network, which specifically includes state-action-reward sequence data generated during the flight of the UAV; two key processing are performed on the obtained flight experience data: one is time sequence alignment, which ensures that the state, action, and reward data are one-to-one corresponding in the time dimension, and eliminates the time sequence misalignment that may occur in the data collection or storage process; the other is normalization processing, which maps the state data, action data, and reward data to a unified data range, eliminating the interference of different types of data magnitude differences on subsequent feature extraction; after processing, standardized flight experience data is generated, providing a high-quality data basis for subsequent context information extraction;

[0144] Step S4.2, context information extraction:

[0145] The standardized flight experience data generated in step S4.1 is input into a context encoder based on a recurrent neural network; the recurrent neural network has the ability to capture the dependency relationship of time series data, and its hidden state is constantly updated during the process of processing sequence data, gradually accumulating the time series information in the standardized flight experience data, and finally outputting a fixed-dimensional context information vector; the vector integrates the time series correlation features in a period of flight task, and can reflect the dynamic correlation between the flight environment and the unmanned aerial vehicle action decision in this period;

[0146] Step S4.3, task meta-feature calculation:

[0147] The context information vector output in step S4.2 is input into a fully connected neural network; the fully connected neural network includes multiple hidden layers, which performs multi-layer nonlinear transformation on the context information vector through nonlinear activation functions in the hidden layers, gradually stripping redundant information and strengthening key features, and finally extracts a task meta-feature vector; The task meta-feature vector encodes the key statistical properties (such as obstacle distribution density, environmental complexity) and dynamic patterns (such as dynamic obstacle movement law, environmental change frequency) of the current flight environment, and is the core vector for describing the essential features of the current flight task environment;

[0148] Step S4.4, meta-learner parameter generation:

[0149] The task meta-feature vector extracted in step S4.3 is input into a meta-learner, which is composed of a multilayer perceptron; the multilayer perceptron receives the task meta-feature vector through the input layer, and generates strategy update parameters through the operation processing of multiple hidden layers after the output layer; The strategy update parameters contain two types of key information: one is the gradient direction adjustment vector of the Actor-Critic network strategy update, which is used to correct the direction of the original strategy gradient; The second is the step scaling factor of the Actor-Critic network strategy update, which is used to adjust the size of the original learning rate;

[0150] Step S4.5, dynamic adjustment of strategy update:

[0151] In the parameter optimization process of the Actor-Critic network, the update rule of the stochastic gradient descent method is updated by using the strategy generated in step S4.4; for the gradient direction, the gradient direction adjustment vector is dot multiplied with the original policy gradient of the Actor-Critic network, and the adjusted policy gradient direction is obtained through the operation, so that the gradient update direction is more suitable for the requirements of the current flight environment; for the update step, the step scaling factor is multiplied with the original learning rate of the Actor-Critic network to obtain the adaptive learning rate, so that the learning rate size can be dynamically adjusted according to the current environment, avoiding the problem of too fast or too slow update caused by the fixed learning rate;

[0152] Step S4.6, online adaptive control strategy:

[0153] Based on the adjusted policy gradient direction and adaptive learning rate obtained in step S4.5, the parameters of the policy network and the value network of the Actor-Critic network are updated; after the parameter update is completed, the control strategy output by the Actor-Critic network can better adapt to the current flight environment, realizing the rapid online adaptation of the control strategy in the current flight environment; at the same time, since the task meta feature vector encodes the general features of the environment, the updated strategy can also have stronger adaptability to the unobserved flight environment, thereby improving the generalization ability of the control strategy in the unobserved flight environment.

[0154] Although the embodiments of the present application have been shown and described, it can be understood by those skilled in the art that various changes, modifications, replacements and variations can be made to the embodiments without departing from the principles and spirits of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A method for intelligent flight path planning and obstacle avoidance for unmanned aerial vehicles (UAVs) based on deep learning, characterized in that: Includes the following steps: Step S1, Environmental Perception and Multi-Scale Feature Fusion: Environmental point cloud, image and pose data are acquired through the airborne lidar, visual sensor and IMU unit of the UAV; the acquired multi-source data is input into a convolutional neural network with a multi-scale encoder-decoder structure with skip connections for processing. The convolutional neural network extracts environmental features of different receptive fields through parallel dilated convolution modules and fuses them to generate a dense grid map containing static obstacle elevation information and dynamic obstacle predicted trajectory probability. Step S2, Collaborative Initial Path Generation: The dynamic obstacle probabilities and static obstacle elevation gradients in the dense grid map are weighted and fused to construct a comprehensive path cost function; an improved dynamic window fusion method is adopted. The algorithm uses the comprehensive path cost function as the core factor of the heuristic function, combines the dynamic constraints of the UAV to perform path search, and generates an initial flight path that meets the maneuverability performance of the UAV. Step S3, Closed-Loop Deep Reinforcement Learning Control: Construct an Actor-Critic network based on a maximum entropy reinforcement learning framework; the state space of the Actor-Critic network is defined as a combined state vector consisting of a dense grid map, the UAV's real-time state vector, and the lateral, altitude, and velocity deviations from the initial flyable flight path; the action space of the Actor-Critic network is defined as the three-axis acceleration and yaw rate commands of the UAV's underlying flight controller; during UAV flight, the Actor-Critic network outputs control commands through its policy network based on the real-time state, driving the UAV to form a closed-loop tracking of the initial flyable flight path and simultaneously avoid unforeseen dynamic obstacles in real time; Step S4, Online Meta-Reinforcement Learning: During the UAV's mission execution, a meta-learner based on contextual information is constructed. The meta-learner continuously extracts mission meta-features from flight experience data and dynamically adjusts the policy update gradient direction and step size of the Actor-Critic network to achieve rapid online adaptation and generalization of the control policy under different flight environments.

2. The method for intelligent flight path planning and obstacle avoidance of unmanned aerial vehicles based on deep learning according to claim 1, characterized in that: The environmental perception and multi-scale feature fusion in step S1 includes: Step S1.1: Simultaneously collect environmental point cloud data, image data, and pose data using the onboard lidar, visual sensor, and IMU unit of the UAV; Step S1.2: Denoise and rasterize the environmental point cloud data to generate a point cloud elevation map; perform distortion correction and color normalization on the image data to generate a standardized image; filter and interpolate the IMU pose data to obtain a continuous pose sequence of the UAV. Step S1.3: Input the point cloud elevation map, normalized image and continuous pose sequence into a convolutional neural network with a multi-scale encoder-decoder structure with skip connections. The encoder part consists of multiple convolutional layers and pooling layers to extract environmental features layer by layer, and the decoder part consists of multiple deconvolutional layers and upsampling layers to reconstruct the feature map. Step S1.4: The convolutional neural network processes the features extracted by the encoder through a parallel dilated convolution module, which contains multiple parallel branches. Each branch uses a dilated convolution kernel with a different dilation rate to extract environmental features with different receptive fields. Step S1.5: The multi-scale feature map output by the parallel dilated convolution module is fused with the feature map of the corresponding layer in the decoder through skip connections, element by element, to form an enhanced multi-scale feature representation; Step S1.6: Input the enhanced multi-scale features into the output layer of the convolutional neural network, and generate a dense grid map containing static obstacle elevation information and dynamic obstacle predicted trajectory probability through the Softmax activation function.

3. The method for intelligent flight path planning and obstacle avoidance of unmanned aerial vehicles based on deep learning according to claim 1, characterized in that: The construction of the comprehensive path cost function in step S2 includes the following steps: Step S2.1: Extract the probability distribution map of predicted trajectories of dynamic obstacles and the elevation gradient map of static obstacles from the dense grid map; Step S2.2: Perform probability normalization processing on the probability distribution map of predicted dynamic obstacle trajectories to obtain a normalized dynamic obstacle probability map; Step S2.3: Calculate and normalize the gradient magnitude of the static obstacle elevation gradient map to obtain a normalized static obstacle elevation gradient map; Step S2.4: Based on the UAV's flight speed and safe distance requirements, dynamically adjust the weighting coefficients of dynamic obstacle probability and static obstacle elevation gradient; Step S2.5: The normalized dynamic obstacle probability map and the normalized static obstacle elevation gradient map are weighted and fused according to the weight coefficients to generate a comprehensive path cost map, which serves as the spatial representation of the comprehensive path cost function.

4. The method for intelligent flight path planning and obstacle avoidance of unmanned aerial vehicles based on deep learning according to claim 1, characterized in that: The improvement in step S2 using a fused dynamic window The algorithm performs pathfinding through the following steps: Step S2.6: Construct a search graph based on the comprehensive path cost graph, where each node in the search graph corresponds to a grid location in the dense grid map, and a comprehensive path cost is assigned to each node; Step S2.7: Initialization Improvement The algorithm uses open and closed lists, adding the drone's starting position node to the open list and setting the actual cost value of the starting node to zero. Step S2.8: Define the improvement The algorithm's heuristic function is composed of a weighted sum of the Euclidean distance from the current node to the target node and the comprehensive path cost of the current node. The weight coefficients are dynamically adjusted according to the flight mission requirements of the UAV. Step S2.9: Based on the real-time state and dynamic constraints of the UAV, calculate the dynamic window, which defines the speed and position space range that the UAV can reach within a short time, and improve the dynamic window based on the constraints. The node expansion range of the algorithm; Step S2.10: Select the node with the smallest heuristic function value from the open list as the current expansion node. Check whether the current expansion node falls within the dynamic window. If it does, expand the node; otherwise, skip the node. Step S2.11: Traverse the neighboring nodes of the current expanded node, calculate the actual cost and heuristic function value of each neighboring node, and update the open list and closed list; Step S2.12: Repeat steps S2.10 and S2.11 until the open list is empty or the target node is successfully found, thereby generating an initial flyable route from the starting position to the target position.

5. The method for intelligent flight path planning and obstacle avoidance of unmanned aerial vehicles based on deep learning according to claim 1, characterized in that: The construction of an Actor-Critic network based on the maximum entropy reinforcement learning framework in step S3 includes the following steps: Step S3.1: State space vector construction: The dense grid map is compressed and vectorized to obtain the environmental feature vector; at the same time, the real-time state vector of the UAV and the lateral deviation, altitude deviation and speed deviation from the initial flight path are normalized to obtain the deviation feature vector; the environmental feature vector and the deviation feature vector are spliced ​​and fused to form a combined state vector. Step S3.2: Definition of motion space vector: Based on the dynamic performance limitations of the UAV, the motion space vector is defined as a three-axis acceleration command and a yaw rate command, and the value range of each motion component is standardized. Step S3.3: Actor Network Construction: Construct a policy network based on a deep neural network. The input layer of this policy network receives a combined state vector, performs feature transformation through multiple fully connected hidden layers, and the output layer uses the Softmax activation function to generate the probability distribution of the action space vector. Step S3.4: Critic Network Construction: Construct a value network based on a deep neural network. The input layer of this value network receives a combined state vector, performs value estimation through multiple fully connected hidden layers, and the output layer generates a state value scalar. Step S3.5: Design of the maximum entropy objective function: In the reinforcement learning framework, design a maximum entropy objective function that includes a cumulative reward and a policy entropy term, where the policy entropy term is used to adjust the exploratory nature of the policy; Step S3.6: Network parameter optimization: Using the state-action-reward sequence data collected in real time during the drone's flight, update the parameters of the Actor network and Critic network using stochastic gradient descent to maximize the maximum entropy objective function.

6. The method for intelligent flight path planning and obstacle avoidance of unmanned aerial vehicles based on deep learning according to claim 1, characterized in that: During the drone's flight, the Actor-Critic network outputs control commands through its policy network based on the real-time status, driving the drone to form a closed-loop tracking of the initial flyable route while simultaneously avoiding unforeseen dynamic obstacles in real time. This includes the following steps: Step S3.7: Real-time Status Acquisition and Vector Construction: During the UAV flight, acquire in real-time a dense grid map, the UAV's real-time status vector, and the lateral, altitude, and speed deviations from the initial flight path; compress and vectorize the dense grid map to obtain environmental feature vectors; normalize the UAV's real-time status vector and deviation vectors to obtain deviation feature vectors; and concatenate and fuse the environmental feature vectors and deviation feature vectors to form a combined status vector. Step S3.8: Forward propagation of the policy network: The combined state vector is input into the policy network of the Actor-Critic network, and the feature is transformed through multiple fully connected hidden layers of the policy network. Finally, the output layer uses the Softmax activation function to generate the probability distribution of the action space vector. Step S3.9: Control action decision: Based on the probability distribution of the action space vector, a random sampling strategy based on maximum entropy is used to select specific control actions, and standardized three-axis acceleration commands and yaw rate commands are generated. Step S3.10: Control command transmission and execution: The generated three-axis acceleration command and yaw rate command are sent to the UAV's underlying flight controller. The underlying flight controller parses the commands and drives the UAV to perform the corresponding flight actions. Step S3.11: Closed-loop tracking and dynamic obstacle avoidance: By continuously repeating steps S3.7 to S3.10, the UAV achieves closed-loop tracking of the initial flight path and dynamically adjusts control commands based on real-time environmental perception data to avoid unforeseen dynamic obstacles. Step S3.12: Flight experience collection and storage: During the flight of the UAV, state-action-reward sequence data are collected in real time and stored in the experience replay buffer to provide a data foundation for the subsequent online optimization of the Actor-Critic network.

7. The method for intelligent flight path planning and obstacle avoidance of unmanned aerial vehicles based on deep learning according to claim 1, characterized in that: The online meta-reinforcement learning in step S4 includes the following steps: Step S4.1, Flight experience data preprocessing: Real-time acquisition of flight experience data from the experience replay buffer of the Actor-Critic network, which includes state-action-reward sequence data; Time series alignment and normalization processing of the flight experience data to generate standardized flight experience data; Step S4.2, Context Information Extraction: Standardized flight experience data is input into a context encoder based on a recurrent neural network. The hidden state of the context encoder is used to capture the temporal dependencies in the flight mission and output a fixed-dimensional context information vector. Step S4.3, Task Meta Feature Calculation: Input the context information vector into a fully connected neural network, and perform nonlinear transformation through multiple hidden layers of the fully connected neural network to extract the task meta feature vector. This task meta feature vector encodes the key statistical characteristics and dynamic patterns of the current flight environment. Step S4.4, Meta-learner parameter generation: Input the task meta-feature vector into the meta-learner. The meta-learner consists of a multilayer perceptron. The policy update parameters are generated through the output layer of the multilayer perceptron. The policy update parameters include the gradient direction adjustment vector and step size scaling factor of the Actor-Critic network policy update. Step S4.5, Dynamic Adjustment of Policy Update: During the parameter optimization process of the Actor-Critic network, the update rule of the stochastic gradient descent method is modified by using the policy update parameters. The gradient direction adjustment vector is multiplied by the original policy gradient to obtain the adjusted policy gradient direction. At the same time, the step size scaling factor is multiplied by the original learning rate to obtain the adaptive learning rate. Step S4.6, Online Adaptation of Control Strategy: Based on the adjusted policy gradient direction and adaptive learning rate, update the policy network and value network parameters of the Actor-Critic network to achieve rapid online adaptation of the control strategy in the current flight environment and improve the generalization ability of the strategy in unseen flight environments.

Images