Unmanned aerial vehicle path planning method and system in unknown obstacle environment

Abstract

The invention discloses an unmanned aerial vehicle path planning method and system in an unknown obstacle environment, and particularly relates to the technical field of unmanned aerial vehicle autonomous navigation and path planning. Constructing an observation space containing a relative target position, a speed, an attitude angle, an angular speed and an obstacle; designing a continuous action space comprising total lift force, rolling moment, pitching moment and yawing moment; constructing a multi-dimensional award function fusing a task end award, a distance target award, a forward efficiency award and an obstacle avoidance safety award, and guiding an intelligent agent to optimize a path; on the basis of observation space and reward feedback, an intelligent agent control strategy is continuously optimized through exploration and development of a self-adaptive entropy regularization balance strategy in an SAC algorithm, and collision-free and short-path real-time path planning is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous navigation and path planning of unmanned aerial vehicles (UAVs), and in particular to a method and system for UAV path planning in an environment with unknown obstacles. Background Art

[0002] Due to their high maneuverability and flexible operation, drones (UAVs) are widely used in low-altitude operations such as logistics and transportation, environmental monitoring, and search and rescue. However, in complex low-altitude environments, UAV flight is easily disrupted by various obstacles. Existing path planning algorithms have many limitations. Common global planning methods (such as the A* algorithm, particle swarm optimization, RRT algorithm, and genetic algorithm) require prior knowledge of the global environment and are slow. Local planning methods (such as artificial potential fields and dynamic window methods) can only solve simple cases where the environment is partially unknown and are difficult to generalize to tasks in completely unknown environments. Their computational speed also fails to meet the real-time requirements of path planning. In the early days of reinforcement learning for path planning, it focused primarily on two-dimensional environments, making it difficult to achieve precise path planning in three-dimensional spaces. Therefore, developing an efficient and accurate path planning method to ensure safe and stable UAV flight in environments with unknown obstacles and improve mission execution efficiency has become an urgent issue. Summary of the Invention

[0003] To this end, the present invention provides a method and system for UAV path planning in an unknown obstacle environment to solve the problems raised in the background technology.

[0004] In order to achieve the above objectives, the present invention provides the following technical solution: a method for planning a path for a drone in an environment with unknown obstacles, comprising:

[0005] Based on the obstacle distance distribution information and drone status parameters within the detection range obtained by the drone's onboard sensors, an observation space containing the relative target position, speed, attitude angle, angular velocity and obstacles is constructed;

[0006] Design a continuous action space that includes total lift, rolling moment, pitching moment, and yaw moment;

[0007] Construct a multi-dimensional reward function that integrates task completion rewards, distance to target rewards, forward efficiency rewards, and obstacle avoidance safety rewards to guide the agent to optimize the path;

[0008] Based on the observation space and reward feedback, through the exploration and development of the adaptive entropy regularization balance strategy in the SAC algorithm, the intelligent agent control strategy is continuously optimized to achieve collision-free, short-path real-time path planning.

[0009] Preferably, the method for constructing the observation space includes:

[0010] Identify and extract the quantitative relationship between the obstacle distribution characteristics and threats within the detection range of the drone, including: the maximum angle δ of the drone's horizontal detection, the maximum angle χ of the drone's vertical detection, expand the area where the drone can detect obstacles and divide it into M nested circular detection layers, each layer is divided into N detection sectors, forming a total of M×N detection partitions, and the detection partition is recorded as C m,n , the detection area radius is r.

[0011] Preferably, the method for generating the continuous action space includes:

[0012] Obtain the deviation information between the current state of the drone and the target state as the input of the policy network actor;

[0013] The policy network Actor outputs continuous control quantities, including total lift, rolling moment, pitching moment, and yaw moment, forming a four-dimensional continuous action space;

[0014] Dual value network Critic evaluates state-action pairs (S t ,Α t ) is fed back to the Actor to adjust its parameters (via policy gradient update), guiding the strategy to optimize towards higher cumulative rewards.

[0015] Preferably, a multi-dimensional reward function is designed to balance multiple objective optimization indicators, including:

[0016] Identify and extract the relationship between multi-objective optimization indicators and weights for path planning, including task completion, target distance, forward efficiency, and obstacle avoidance safety;

[0017] Obtain real-time information about the drone's status and environment, including whether it has reached the target, the distance to obstacles, and the rate of change of its forward direction;

[0018] For task completion, set a success reward of +40 and a collision penalty of -20;

[0019] For target distance, set a linear penalty term;

[0020] For forward efficiency, the distance difference when approaching the target is set as positive incentive;

[0021] For obstacle avoidance safety, it is set to the weighted sum of the distance and velocity projection of close obstacles in each partition.

[0022] Preferably, the high-dimensional observation space including obstacle observation and self-state includes:

[0023] The obstacle parameters in each partition are expressed as:

[0024]

[0025] Where, To detect partition C m,n The distance between the nearest obstacle and the drone, r is the radius of the detection area, To detect partition C m,n Obstacle parameters in ;

[0026] The obstacle observation of the UAV at time t is expressed as:

[0027]

[0028] Where, Indicates that there are M×N detection partitions in total.

[0029] Obtain basic information of the drone's status parameters, including position coordinates, linear velocity, attitude angle, angular velocity, and relative target position, expressed as:

[0030]

[0031] Where, is the three-dimensional relative position parameter between the current position of the UAV and the target position, is the current three-dimensional velocity parameter, φ t ,θ t ,ψ t is the current three-dimensional rotation parameter, is the current three-dimensional angular velocity parameter.

[0032] For areas where obstacles are detected, their distance is quantified as the corresponding dimension of the observation space; for areas without obstacles, the corresponding dimension is set to the maximum detection distance of the sensor; combined with the drone's own state parameters, a high-dimensional observation space containing obstacle parameters and its own state is constructed, with a dimension of M×N+12;

[0033] The observation of the UAV at time t is expressed as:

[0034]

[0035] Where expand() means expanding the matrix into a one-dimensional matrix, and the subscript 1×(12+M×N) is the matrix dimension.

[0036] Preferably, a reward function is designed to balance multiple objective optimization indicators, including:

[0037] The task completion is expressed by setting the task completion reward:

[0038]

[0039] Where, P current is the current position parameter, P goal is the target position parameter, ò is the target radius, dcollision is the collision threshold, d i is the minimum distance between the drone and the obstacle, and otherwise represents other situations;

[0040] For the target distance, set a linear penalty term:

[0041] R dis =-||P current -P goal ||;

[0042] For forward efficiency, set the distance difference when approaching the target as positive excitation:

[0043] R close =σ1(||P t+1 -P goal ||-||P t -P goal ||);

[0044] Where σ1 is a positive number greater than 1, which means a positive reward when approaching the target and a negative reward when moving away from the target; t is the position parameter of the UAV at time t;

[0045] For obstacle avoidance safety, it is set to the weighted sum of the distance and velocity projection of close obstacles in each partition:

[0046]

[0047] Where, d mn To detect partition C m,n The distance from the nearest obstacle to the UAV, κ mn is the projection of the drone’s velocity toward the center of the partition, and σ2 is the hyperparameter that controls the amplitude of the reward function;

[0048] By integrating multiple objectives through weighted summation, the total reward at each time step is expressed as:

[0049] R step =ε1·R dis +ε2·R close +ε3·R obs ;

[0050] Where ε1, ε2, and ε3 are all proportional weight coefficients.

[0051] Preferably, the SAC algorithm for training the UAV agent includes:

[0052] By maximizing the expected value of the sum of reward and entropy, the optimal policy is expressed as:

[0053]

[0054] Where, is the expected value of the random variable under a given strategy π, R(S t ,Α t ) is the reward value under the current state and action, S t is the current state, Α t is the current action, H(π(·|S t )) is the entropy value of strategy π, α is the entropy regularization coefficient, which controls the importance of entropy value relative to cumulative return;

[0055] The Soft Q function in the SAC algorithm is defined as:

[0056]

[0057] Where γ is the discount factor. When updating the policy network, the loss function is:

[0058]

[0059] Where, is the log probability of the strategy used to maximize entropy, is the current Q network's valuation of the state-action pair;

[0060] When updating the parameters of the Q network, the loss function is:

[0061]

[0062] Where, is the current network parameter;

[0063] To prevent overestimation of the Q value, the target optimal Q value is defined as:

[0064]

[0065] A dual-Q network architecture is used to evaluate the state-action value and the network parameters are updated through a soft update mechanism. For the target network parameters, the following operations are performed:

[0066]

[0067] Where, is the target network parameter, ι is used to control the update amplitude and usually takes a very small value (greater than 0 and much smaller than 1);

[0068] By adaptively adjusting the entropy regularization coefficient α, dynamic balance strategy exploration and development are performed according to the formula:

[0069]

[0070] Where π t (At |π t ) is the action probability output by the policy network, is the target entropy value;

[0071] The experience replay pool is used to randomly sample data and the network parameters are updated through the Adam optimizer with a learning rate of η = 5×10 -4 , and iteratively optimize the control strategy until convergence.

[0072] The present invention also discloses a UAV path planning system in an unknown obstacle environment, which is used to implement the method claimed above, including:

[0073] The environmental perception and modeling module is used to collect real-time information about the drone's environment and state, such as the distance between the drone and obstacles and the drone's posture, through onboard sensors. It divides the detection range into multiple partitions, constructs a high-dimensional observation space containing obstacle parameters and the drone's state, and realizes dynamic modeling of unknown environments.

[0074] The strategy generation and control module uses the SAC algorithm, takes the drone's observation space as input to the policy network actor, and outputs continuous control actions including total lift, rolling moment, pitching moment, and yaw moment through the policy network actor to drive the drone to perform obstacle avoidance maneuvers. The dual value network Critic evaluates the value of the observation space and the continuous action space to guide the optimization of the policy network actor.

[0075] The reward evaluation and feedback module constructs a multi-dimensional reward function that integrates task completion rewards, distance target rewards, forward efficiency rewards, and obstacle avoidance safety rewards. It evaluates the reward value based on the real-time state of the drone and the interaction information with the environment, and feeds it back to the strategy generation and control module to optimize the path planning strategy.

[0076] The optimization iteration module explores and develops a balancing strategy by adaptively adjusting the entropy regularization coefficient, uses the experience replay pool to randomly sample data, updates the network parameters through the Adam optimizer, and iteratively optimizes the control strategy until convergence, achieving real-time path optimization.

[0077] The UAV path planning method and system provided by the present invention in an environment with unknown obstacles take into account both safety and operational efficiency by combining a multi-dimensional reward function with a reinforcement learning algorithm. The method and system have a low collision rate, a short path planning time, and an excellent path length. Compared with the existing technology, the method and system do not require pre-programmed maps and only rely on real-time data from onboard sensors to achieve autonomous obstacle avoidance and path planning in complex and unknown environments. The system also has strong generalization capabilities and can cope with obstacles of various shapes and dynamic obstacle scenarios. Its robustness is significantly better than traditional methods, which is of great significance to improving the application capabilities and mission execution reliability of UAVs in complex environments.

[0078] The present invention can effectively solve the path planning problem caused by unknown obstacles when UAVs operate in complex environments, taking into account both flight safety and operation efficiency, and giving full play to the flexible and maneuverable advantages of UAVs. It is of great significance to expand the application scenarios of UAVs and promote the widespread application of UAV technology in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 It is a flow chart of the method for planning the path of a UAV in an unknown obstacle environment according to the present invention;

[0080] Figure 2 This is a structural diagram of a UAV path planning system in an unknown obstacle environment provided by an embodiment of the present invention;

[0081] Figure 3 This is a schematic diagram of obstacle detection zones provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0082] The following describes the implementation of the present invention using specific embodiments. Those skilled in the art will readily understand the other advantages and benefits of the present invention from the disclosure herein. Obviously, the embodiments described are only a portion of the present invention, not all of it. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are intended to fall within the scope of protection of the present invention.

[0083] like Figure 1 As shown, Example 1: This embodiment provides a method for planning a drone path in an unknown obstacle environment, including:

[0084] Step A: Environmental modeling and observation construction, based on the real-time data of the drone's onboard sensors (lidar, IMU), build an unknown obstacle environment model and observation space. Specifically including:

[0085] Step A1: Detection partitioning: Expand the area where the drone can detect obstacles and divide it into M nested circular detection layers. Each layer is divided into N detection sectors, forming a total of M×N detection partitions.

[0086] Step A2: Data collection and processing: The LiDAR scans each partition at a frequency of 10 Hz to obtain the distance to the nearest obstacle. When there are no obstacles, the maximum detection distance is set.

[0087] Step A3: Fusion the obstacle partition and state parameters into an observation space with a dimension of M×N+12.

[0088] The obstacle parameters in each partition are expressed as:

[0089]

[0090] Where, To detect partition C m,n The distance between the nearest obstacle and the UAV, r is the radius of the detection area.

[0091] The obstacle observation of the UAV at time t can be expressed as:

[0092]

[0093] Obtain basic information of drone status parameters, including position coordinates, linear velocity, attitude angle, angular velocity, relative target position, etc., which can be expressed as:

[0094]

[0095] Where, is the three-dimensional relative position parameter between the current position of the UAV and the target position, is the current three-dimensional velocity parameter, φ t ,θ t ,ψ t are the current three rotation parameters, are the current three angular velocity parameters.

[0096] The observation of the UAV at time t can be expressed as:

[0097]

[0098] Step B: Design a continuous action space that includes total lift, rolling moment, pitching moment, and yaw moment. Construct a four-dimensional continuous control action that includes total lift, rolling moment, pitching moment, and yaw moment, and integrate a multi-dimensional function that integrates mission completion rewards, distance target rewards, forward efficiency incentives, and obstacle avoidance safety rewards. Specifically, it includes:

[0099] Step B1: Obtain the real-time status of the UAV and its interaction with the environment, including: whether it has reached the target, the distance to the obstacle, the rate of change of the forward direction, etc.

[0100] Step B2: For task completion, set the success reward to +20 and the collision penalty to -40. The task completion reward is expressed as:

[0101]

[0102] Where, P current is the current position parameter, P goal is the target position parameter, ò is the target radius, d collision is the collision threshold.

[0103] Step B3: For each time step, introduce heuristic rewards, including:

[0104] (1) Set a linear penalty term for the distance target reward:

[0105] R dis =-||P current -P goal ||

[0106] (2) The forward efficiency reward is a positive incentive based on the distance difference when approaching the target:

[0107] R close =σ1(||P t+1 -P goal ||-||P t -P goal ||)

[0108] Where σ1 is a positive number greater than 1, which means a positive reward when approaching the target and a negative reward when moving away from the target.

[0109] (3) Obstacle avoidance safety reward: the weighted sum of the distance and velocity projection of each partition's close-range obstacles:

[0110]

[0111] Where, d mn To detect partition C m,n The distance from the nearest obstacle to the UAV, κ mn is the projection of the drone’s velocity toward the center of the partition, and σ2 is a hyperparameter that controls the amplitude of the reward function.

[0112] The total reward at each time step can be expressed as:

[0113] R step =ε1·R dis +ε2·R close +ε3·R obs

[0114] Where ε1, ε2, and ε3 are proportional weight coefficients.

[0115] Step C: Based on the observation space and reward feedback, the agent control strategy is iteratively optimized through exploration and development of an adaptive entropy regularization balance strategy.

[0116] By maximizing the expected value of the sum of reward and entropy, the optimal policy can be expressed as:

[0117]

[0118] In the formula, R(S t ,Α t ) is the reward value under the current state and action, H(π(·|S t)) is the entropy value of strategy π, and α is the entropy regularization coefficient, which controls the importance of entropy value relative to cumulative return.

[0119] The Soft Q function in the SAC algorithm is defined as:

[0120]

[0121] Where γ is the discount factor. When updating the policy network, the loss function is:

[0122]

[0123] When updating the parameters of the Q network, the loss function is:

[0124]

[0125] To prevent overestimation of the Q value, the target optimal Q value is defined as:

[0126]

[0127] A dual-Q network architecture is used to evaluate the state-action value and the network parameters are updated through a soft update mechanism. For the target network parameters, the following operations are performed:

[0128]

[0129] Wherein, the value of ι is greater than 0 but much less than 1.

[0130] By adaptively adjusting the entropy regularization coefficient α, dynamic balance strategy exploration and development are performed according to the formula:

[0131]

[0132] Where π t (A t |π t ) is the action probability output by the policy network, is the target entropy value;

[0133] The experience replay pool is used to randomly sample data and the network parameters are updated through the Adam optimizer with a learning rate of η = 5×10 -4 , and iteratively optimize the control strategy until convergence.

[0134] like Figure 2 As shown, Example 2: Based on Example 1, this embodiment provides a drone path planning system in an unknown obstacle environment, including:

[0135] like Figure 3As shown, the environmental perception and modeling module 01 randomly generates the drone's initial and target positions within a 400m × 500m × 400m cube and deploys four cylindrical obstacles of varying radius and height. The obstacle detection area is expanded and divided into M nested circular detection layers, each layer divided into N detection sectors, for a total of M × N detection partitions. The obstacle distance to each partition is acquired in real time, and the IMU collects drone state parameters such as speed and attitude angle to construct an observation space. The obstacle parameters in each partition are expressed as:

[0136]

[0137] Where, To detect partition C m,n The distance between the nearest obstacle and the UAV, r is the radius of the detection area.

[0138] The obstacle observation of the UAV at time t can be expressed as:

[0139]

[0140] Obtain basic information of drone status parameters, including position coordinates, linear velocity, attitude angle, angular velocity, relative target position, etc., which can be expressed as:

[0141]

[0142] Where, is the three-dimensional relative position parameter between the current position of the UAV and the target position, is the current three-dimensional velocity parameter, φ t ,θ t ,ψ t are the current three rotation parameters, are the current three angular velocity parameters.

[0143] The observation of the UAV at time t can be expressed as:

[0144]

[0145] In Strategy Generation and Control Module 02, the policy network outputs continuous control actions based on the observation space. For example, when an obstacle is detected in the forward sector, it generates a rolling torque to steer the drone to the left while simultaneously adjusting lift to maintain altitude. The flight controller converts these control commands into motor speeds, driving the drone to execute the obstacle avoidance maneuver.

[0146] Reward evaluation and feedback module 03 calculates the total reward based on the flight results. For mission completion, a success reward of +20 and a collision penalty of -40 are set. The mission completion reward is expressed as:

[0147]

[0148] Where, P current is the current position parameter, P goal is the target position parameter, ò is the target radius, d collision is the collision threshold.

[0149] For each time step, a heuristic reward is introduced, including:

[0150] (1) Set a linear penalty term for the distance target reward:

[0151] R dis =-||P current -P goal ||

[0152] (2) The forward efficiency reward is a positive incentive based on the distance difference when approaching the target:

[0153] R close =σ1(||P t+1 -P goal ||-||P t -P goal ||)

[0154] Where σ1 is a positive number greater than 1, which means a positive reward when approaching the target and a negative reward when moving away from the target.

[0155] (3) Obstacle avoidance safety reward: the weighted sum of the distance and velocity projection of each partition's close-range obstacles:

[0156]

[0157] Where, d mn To detect partition C m,n The distance from the nearest obstacle to the UAV, κ mn is the projection of the drone’s velocity toward the center of the partition, and σ2 is a hyperparameter that controls the amplitude of the reward function.

[0158] The total reward at each time step can be expressed as:

[0159] R step =ε1·R dis +ε2·R close +ε3·R obs

[0160] Where ε1, ε2, and ε3 are proportional weight coefficients.

[0161] Optimizing iterative module 04, by maximizing the expected value of the sum of reward and entropy, the optimal strategy can be expressed as:

[0162]

[0163] In the formula, R(S t ,Α t ) is the reward value under the current state and action, H(π(·|S t )) is the entropy value of strategy π, and α is the entropy regularization coefficient, which controls the importance of entropy value relative to cumulative return.

[0164] The Soft Q function in the SAC algorithm is defined as:

[0165]

[0166] Where γ is the discount factor. When updating the policy network, the loss function is:

[0167]

[0168] When updating the parameters of the Q network, the loss function is:

[0169]

[0170] To prevent overestimation of the Q value, the target optimal Q value is defined as:

[0171]

[0172] A dual-Q network architecture is used to evaluate the state-action value and the network parameters are updated through a soft update mechanism. For the target network parameters, the following operations are performed:

[0173]

[0174] Wherein, the value of ι is greater than 0 but much less than 1.

[0175] By adaptively adjusting the entropy regularization coefficient α, dynamic balance strategy exploration and development are performed according to the formula:

[0176]

[0177] Where π t (A t |π t ) is the action probability output by the policy network, is the target entropy value;

[0178] The experience replay pool is used to randomly sample data and the network parameters are updated through the Adam optimizer with a learning rate of η = 5×10 -4 , and iteratively optimize the control strategy until convergence.

[0179] Although the present invention has been described in detail above using general descriptions and specific embodiments, it will be apparent to those skilled in the art that modifications and improvements may be made thereto. Therefore, such modifications and improvements, without departing from the spirit of the present invention, are intended to be within the scope of protection claimed herein.

Claims

1. A method for UAV path planning in an unknown obstacle environment, characterized by: include: Based on the obstacle distance distribution information and drone status parameters within the detection range obtained by the drone's onboard sensors, an observation space containing the relative target position, speed, attitude angle, angular velocity and obstacles is constructed; Design a continuous action space that includes total lift, rolling moment, pitching moment, and yaw moment; Construct a multi-dimensional reward function that integrates task completion rewards, distance to target rewards, forward efficiency rewards, and obstacle avoidance safety rewards to guide the agent to optimize the path; Based on the observation space and reward feedback, through the exploration and development of the adaptive entropy regularization balance strategy in the SAC algorithm, the intelligent agent control strategy is continuously optimized to achieve collision-free, short-path real-time path planning.

2. The method for UAV path planning in an unknown obstacle environment according to claim 1, characterized in that: The methods for constructing the observation space include: Identify and extract the quantitative relationship between the obstacle distribution characteristics and threats within the detection range of the drone, including: the maximum angle δ of the drone's horizontal detection, the maximum angle χ of the drone's vertical detection, expand the area where the drone can detect obstacles and divide it into M nested circular detection layers, each layer is divided into N detection sectors, forming a total of M×N detection partitions, and the detection partition is recorded as C m,n , the detection area radius is r.

3. The method for UAV path planning in an unknown obstacle environment according to claim 1, characterized in that: Methods for generating continuous action spaces include: Obtain the deviation information between the current state of the drone and the target state as the input of the policy network actor; The policy network Actor outputs continuous control quantities, including total lift, rolling moment, pitching moment, and yaw moment, forming a four-dimensional continuous action space; Dual value network Critic evaluates state-action pairs (S t ,Α t ) is fed back to the Actor to adjust its parameters and guide the strategy to optimize towards higher cumulative rewards.

4. The method for UAV path planning in an unknown obstacle environment according to claim 1, characterized in that: Design a multi-dimensional reward function that balances multiple objective optimization metrics, including: Identify and extract the relationship between multi-objective optimization indicators and weights for path planning, including task completion, target distance, forward efficiency, and obstacle avoidance safety; Relying on the deviation between the current position of the UAV and the target position and the distance to obstacles in the observation space, the real-time state of the UAV and the interaction information with the environment are obtained, including whether the UAV has reached the target, the distance to obstacles, and the rate of change of the forward direction; For task completion, set a success reward of +40 and a collision penalty of -20; For target distance, set a linear penalty term; For forward efficiency, the distance difference when approaching the target is set as positive incentive; For obstacle avoidance safety, it is set to the weighted sum of the distance and velocity projection of close obstacles in each partition.

5. The method for UAV path planning in an unknown obstacle environment according to claim 2, characterized in that: The high-dimensional observation space containing obstacle observations and its own state includes: The obstacle parameters in each partition are expressed as: Where, To detect partition C m,n The distance between the nearest obstacle and the drone, r is the radius of the detection area, To detect partition C m,n Obstacle parameters in ; The obstacle observation of the UAV at time t is expressed as: Where, Indicates that there are M×N detection partitions in total. Obtain basic information of the drone's status parameters, including position coordinates, linear velocity, attitude angle, angular velocity, and relative target position, expressed as: Where, is the three-dimensional relative position parameter between the current position of the UAV and the target position, is the current three-dimensional velocity parameter, φ t ,θ t ,ψ t is the current three-dimensional rotation parameter, is the current three-dimensional angular velocity parameter. For areas where obstacles are detected, their distance is quantified as the corresponding dimension of the observation space; for areas without obstacles, the corresponding dimension is set to the maximum detection distance of the sensor; combined with the drone's own state parameters, a high-dimensional observation space containing obstacle parameters and its own state is constructed, with a dimension of M×N+12; The observation of the UAV at time t is expressed as: Where expand() means expanding the matrix into a one-dimensional matrix, and the subscript 1×(12+M×N) is the matrix dimension.

6. The method for UAV path planning in an unknown obstacle environment according to claim 4, characterized in that: Design a reward function that balances multiple optimization objectives, including: The task completion is expressed by setting the task completion reward: Where, P current is the current position parameter, P goal is the target position parameter, is the target radius, d collision is the collision threshold, d i is the minimum distance between the drone and the obstacle, and otherwise represents other situations; For the target distance, set a linear penalty term: R dis =-||P current -P goal ||; For forward efficiency, set the distance difference when approaching the target as positive excitation: R close =σ1(||P t+1 -P goal ||-||P t -P goal ||); Where σ1 is a positive number greater than 1, which means a positive reward when approaching the target and a negative reward when moving away from the target; t is the position parameter of the UAV at time t; For obstacle avoidance safety, it is set to the weighted sum of the distance and velocity projection of close obstacles in each partition: Where, d mn To detect partition C m,n The distance from the nearest obstacle to the UAV, κ mn is the projection of the drone’s velocity toward the center of the partition, and σ2 is the hyperparameter that controls the amplitude of the reward function; By integrating multiple objectives through weighted summation, the total reward at each time step is expressed as: R step =ε1·R dis +ε2·R close +ε3·R obs ; Where ε1, ε2, and ε3 are all proportional weight coefficients.

7. The method for UAV path planning in an unknown obstacle environment according to claim 1, characterized in that: The SAC algorithm used to train the drone agent includes: By maximizing the expected value of the sum of reward and entropy, the optimal policy is expressed as: Where, is the expected value of the random variable under a given strategy π, R(S t ,Α t ) is the reward value under the current state and action, S t is the current state, Α t is the current action, H(π(·|S t )) is the entropy value of strategy π, α is the entropy regularization coefficient, which controls the importance of entropy value relative to cumulative return; The Soft Q function in the SAC algorithm is defined as: Where γ is the discount factor. When updating the policy network, the loss function is: Where, is the logarithmic probability of the strategy used to maximize entropy, Q θ (S t ,A t ) is the current Q network's estimate of the state-action pair; When updating the parameters of the Q network, the loss function is: Where θ i is the current network parameter; To prevent overestimation of the Q value, the target optimal Q value is defined as: A dual-Q network architecture is used to evaluate the state-action value and the network parameters are updated through a soft update mechanism. For the target network parameters, the following operations are performed: Where, is the target network parameter, ι is used to control the update amplitude and usually takes a very small value (greater than 0 and much smaller than 1); By adaptively adjusting the entropy regularization coefficient α, dynamic balance strategy exploration and development are performed according to the formula: Where π t (A t |π t ) is the action probability output by the policy network, is the target entropy value; The experience replay pool is used to randomly sample data and the network parameters are updated through the Adam optimizer with a learning rate of η = 5×10 -4 , and iteratively optimize the control strategy until convergence.

8. UAV path planning system in unknown obstacle environment, characterized by: The system is used to implement the method according to any one of claims 1 to 7, comprising: The environmental perception and modeling module is used to collect real-time information about the drone's environment and state, such as the distance between the drone and obstacles and the drone's posture, through onboard sensors. It divides the detection range into multiple partitions, constructs a high-dimensional observation space containing obstacle parameters and the drone's state, and realizes dynamic modeling of unknown environments. The strategy generation and control module uses the SAC algorithm, takes the drone's observation space as input to the policy network actor, and outputs continuous control actions including total lift, rolling moment, pitching moment, and yaw moment through the policy network actor to drive the drone to perform obstacle avoidance maneuvers. The dual value network Critic evaluates the value of the observation space and the continuous action space to guide the optimization of the policy network actor. The reward evaluation and feedback module constructs a multi-dimensional reward function that integrates task completion rewards, distance target rewards, forward efficiency rewards, and obstacle avoidance safety rewards. It evaluates the reward value based on the real-time state of the drone and the interaction information with the environment, and feeds it back to the strategy generation and control module to optimize the path planning strategy. The optimization iteration module explores and develops a balancing strategy by adaptively adjusting the entropy regularization coefficient, uses the experience replay pool to randomly sample data, updates the network parameters through the Adam optimizer, and iteratively optimizes the control strategy until convergence, achieving real-time path optimization.

Images