Trajectory planning method for unmanned aerial vehicle to quickly pass through window

Abstract

The invention discloses a trajectory planning method for an unmanned aerial vehicle to quickly pass through a window, and the method combines ESDFfree gradient optimization of EGOPlanner with adaptive parameter adjustment of reinforcement learning, and carries out the steps of constructing a state space, designing an action space, optimizing a target function, setting a window specific constraint, designing a reward function, training a reinforcement learning agent, and the like. The problems that in the prior art, calculation efficiency is low, parameter adjustment is complex, and safety and speed are difficult to balance are solved. Time-consuming distance field calculation is avoided, and the planning speed is increased by one order of magnitude; track parameters are automatically optimized through reinforcement learning to adapt to windows of different sizes and angles; the safety margin and the passing speed are comprehensively considered, so that the unmanned aerial vehicle passes through the window at the maximum possible speed on the premise of ensuring the safety; through trajectory time redistribution and anisotropic curve fitting, it is automatically ensured that the generated trajectory meets dynamic constraints.

Description

Technical Field

[0001] This invention relates to the field of autonomous navigation and trajectory planning technology for unmanned aerial vehicles (UAVs), specifically to a trajectory planning method for UAVs to quickly pass through windows based on a combination of gradient optimization and reinforcement learning. Background Technology

[0002] With the rapid development of drone technology, drones are being used more and more widely in scenarios such as search and rescue, building inspection, and urban delivery. In these applications, drones often need to navigate through narrow passages such as windows and door frames, which places high demands on trajectory planning algorithms. Drones need to navigate these narrow passages quickly and safely while maintaining sufficient safety margins and meeting the drone's dynamic constraints.

[0003] Currently, the main methods for planning the trajectory of drones passing through windows can be categorized as follows:

[0004] 1. Sampling-based methods: such as RRT (Rapidlyexploring Random Tree) and its variants. These methods can search for feasible paths in complex environments, but it is difficult to guarantee the smoothness and optimality of the path, and they usually require post-processing to satisfy dynamic constraints.

[0005] 2. Optimization-based methods: such as gradient optimization methods like CHOMP and STOMP. These methods optimize the trajectory by defining an objective function, generating smooth and dynamically feasible trajectories. However, most optimization-based methods rely on the Euclidean Signature Distance Field (ESDF) to calculate the gradient. Constructing the ESDF is very time-consuming, consuming more than 70% of the computational resources in the planning process, which is unacceptable for window crossing scenarios with high real-time requirements.

[0006] 3. Learning-based methods: such as using deep reinforcement learning to directly learn control policies. These methods can generate control commands end-to-end, but their performance is heavily dependent on the quality and generalization ability of the training data, and it is difficult to guarantee security and reliability.

[0007] The recently proposed EGOPlanner is a gradient optimization method that does not require ESDF, significantly reducing computational complexity compared to traditional methods. However, it still has the following problems: 1) It lacks targeted optimization for specific scenarios such as window passage. 2) Parameter tuning relies on expert experience and is difficult to automatically adapt to windows of different sizes and angles. 3) When facing high-speed window passage, it is difficult to balance safety and passage speed. Summary of the Invention

[0008] The purpose of this invention is to solve the problems of low computational efficiency, complex parameter adjustment, and difficulty in balancing safety and speed in existing drone window-crossing trajectory planning methods, and to provide an efficient and adaptive trajectory planning method that enables drones to quickly and safely cross windows of various sizes and angles.

[0009] To achieve the above objectives, this invention proposes a UAV window crossing trajectory planning method based on a combination of EGOPlanner and reinforcement learning. EGOPlanner serves as the basic trajectory generation module, and reinforcement learning is used for adaptive optimization of parameter configuration. The specific technical solution is as follows:

[0010] Step S1: Construct system input and state representation. Acquire and organize system inputs, including the current state of the UAV (3D position coordinates, linear velocity vector, acceleration vector), window information (center point coordinates, length and width dimensions, 3D spatial orientation angle), and environmental obstacle information; establish a state space representation. The drone's status information is acquired in real time through onboard sensors, and window information is extracted through visual sensors or lidar perception systems.

[0011] Step S2: Construct the reinforcement learning action space and trajectory parameterization. Design the reinforcement learning action space. Trajectory parameterization adopts uniform spline curve Number of control points The number of nodes is adaptively determined based on trajectory complexity and window distance, typically ranging from 715. Using a uniform distribution, where The initial control points are generated by fitting a relaxed B-spline that directly connects the start point, the window center, and the end point. The positions of the control points are then finely adjusted by a reinforcement learning agent.

[0012] Step S3: Design the trajectory optimization objective function. Construct a comprehensive objective function based on EGOPlanner: Each weight coefficient is adaptively adjusted by the reinforcement learning agent; smoothness penalty term This ensures the smoothness and energy efficiency of the trajectory; collision penalty term Calculated using the ESDFfree method: For the window frame, an analytical distance function was specifically designed, eliminating the need for pre-calculation of the distance field and significantly improving computational efficiency; a dynamic feasibility penalty term was also included. Designed as follows:

[0013] Step S4: Construct the window's geometric model. The window is represented as a combination of a rectangular plane and a bounding box, located through its center point. ,size and normal vector The window crossing point is uniquely determined; the constraints are: a) the crossing point must be located inside the window and maintain a safe distance from the window frame edge; b) the angle between the velocity direction and the normal direction of the window plane during crossing is less than a preset threshold; c) the crossing speed is greater than a minimum threshold. To ensure a quick passage

[0014] Step S5: Design the reinforcement learning reward function. Construct the comprehensive reward function: .

[0015] Step S6, train the reinforcement learning agent. The reinforcement learning agent is trained using the Proximal Policy Optimization (PPO) algorithm to maximize cumulative rewards; the agent network architecture is designed, including:

[0016] a) Feature extraction layer: Uses a 3D convolutional neural network to process environmental obstacle information, with the input being... The occupied grid is processed through 3 convolutional layers; a fully connected network is used to process window geometry features and drone state information;

[0017] b) Feature fusion layer: This layer stitches together environmental features, window features, and UAV status features and fuses them through a two-layer fully connected network (256 and 128 hidden units);

[0018] c) Dual-head output layer: The policy head outputs the mean and variance, which are used to sample actions; the value head estimates the state value, which is used to optimize training;

[0019] The training process includes:

[0020] a) Construct diverse training environments, including windows of different sizes, locations, and orientations;

[0021] b) Introduce a learning strategy that gradually transitions from simple scenarios (front view, wide window) to complex scenarios (tilted view, narrow window);

[0022] c) Employ an experience replay buffer to store and reuse successful traversal trajectories, accelerating the learning process;

[0023] d) Implement multi-core parallel training to improve sample efficiency and training speed.

[0024] Step S7, trajectory generation and execution. The system operation flow is as follows:

[0025] a) The environmental perception module acquires window information and obstacle distribution to construct a state representation;

[0026] b) The reinforcement learning agent outputs action parameters based on the current state, including control point adjustment, time coefficient, and optimization weights;

[0027] c) Perform EGOPlanner-based trajectory optimization using the adjusted parameters to generate a preliminary trajectory;

[0028] d) Check if the trajectory meets the window crossing constraint. If not, return to step b and readjust the parameters.

[0029] e) Conduct a trajectory dynamics feasibility test and calculate the exceedance rate: ;

[0030] f) If Then time reallocation will be performed: The trajectory is optimized using anisotropic curve fitting methods to maintain the trajectory shape while satisfying dynamic constraints.

[0031] g) Trajectory refinement and smoothing to ensure that the generated trajectory meets the requirements of the UAV control system;

[0032] h) The final trajectory is converted into a sequence of control commands that can be executed by the UAV, and then executed by a high-frequency closed-loop controller.

[0033] This invention also discloses a trajectory planning system for a drone to quickly pass through a window, including an environmental perception module, a reinforcement learning agent, and an EGOPlanner optimization module, characterized by:

[0034] Environmental perception module: responsible for acquiring window boundary information and surrounding obstacle distribution data in the current scene; this module relies on a binocular camera for visual perception, and uses real-time captured binocular images to achieve depth estimation, thereby obtaining a dense depth map and a three-dimensional spatial point cloud;

[0035] In terms of window recognition, a YOLOv5-based convolutional neural network is used to detect window targets in the left eye image. Combined with binocular matching, the corresponding spatial position is calculated. The geometric boundary and normal direction of the window are further estimated through image plane fitting and edge extraction algorithms, so as to achieve accurate modeling of the window center point, width and height dimensions and attitude angle.

[0036] In terms of obstacle mapping, the system constructs the binocular depth map as a local sparse point cloud set, or projects it as a two-dimensional grid map to represent the distribution of obstacles in the flight space, supporting subsequent trajectory feasibility assessment and obstacle avoidance gradient construction.

[0037] Reinforcement learning agent module: Constructs the current state based on the perception results. The system takes into account factors such as the relative position, speed, orientation, and obstacle density distribution between the drone and the window as inputs to the reinforcement learning agent. It employs the Proximal Policy Optimization (PPO) algorithm for policy learning. During training, a reward function is constructed that integrates crossing success rate, trajectory smoothness, and collision penalty to drive the policy network to learn parameter adjustment rules. The network structure uses a three-layer fully connected network, with multiple neurons in each layer. The ReLU activation function is used, and parallel sampling and experience replay mechanisms are employed to accelerate convergence during training. During execution, the reinforcement learning agent outputs an action vector based on its current state. This is used to adjust the control point spacing, time allocation, and weights of each penalty term in the cost function during trajectory optimization. express Adjustment amount of spline control points, This represents the trajectory time adjustment factor. , , These represent the weights of smoothness, collision avoidance, and dynamic feasibility in EGOPlanner, respectively.

[0038] The EGO-Planner optimization module: After receiving the parameter configuration, it generates a preliminary trajectory. Based on the B-spline trajectory modeling method, it constructs the following optimization problem with trajectory smoothness, collision distance, and dynamic feasibility as cost functions: ,

[0039] in For the set of trajectory control points, , , These represent the smoothness, collision penalty, and dynamic feasibility cost terms, respectively. , , These represent the weights of smoothness, collision avoidance, and dynamic feasibility in EGOPlanner, respectively. During trajectory optimization, if a conflict with dynamic constraints is detected, the system will automatically trigger the trajectory time redistribution and anisotropic curve fitting module, and finally output a trajectory that satisfies the crossing constraints for control execution.

[0040] Beneficial effects

[0041] 1) Improved computational efficiency: Employing EGOPlanner's ESDF-free gradient calculation method avoids time-consuming distance field calculations, increasing planning speed by an order of magnitude and enabling the UAV to adjust its trajectory in real time during flight. 2) Enhanced adaptability: By automatically optimizing trajectory parameters through reinforcement learning, the system can adapt to windows of different sizes, shapes, and angles without manual parameter tuning, significantly improving the system's versatility and robustness.

[0042] 2) Safety and speed balance optimization: The reinforcement learning reward function takes into account both safety margin and speed of passage, enabling the drone to pass through the window at the maximum possible speed while ensuring safety.

[0043] 3) Automatic satisfaction of dynamic constraints: Through trajectory time redistribution and anisotropic curve fitting, the generated trajectory is automatically ensured to meet the dynamic constraints of the UAV without the need for additional post-processing.

[0044] 4) Improved planning success rate: Compared with traditional methods, the success rate of planning in complex environments is significantly improved, especially for narrow or tilted windows, the success rate is improved by about 30%. Attached Figure Description

[0045] Figure 1 This is a system architecture diagram of a trajectory planning method for a drone to quickly pass through a window, according to the present invention.

[0046] Figure 2 This is a schematic diagram illustrating the window's trajectory and specific constraints. Detailed Implementation

[0047] Example 1

[0048] This invention proposes a window-crossing trajectory planning method for UAVs based on a combination of EGOPlanner and reinforcement learning. EGOPlanner serves as the basic trajectory generation module, while reinforcement learning is used for adaptive optimization of parameter configuration. The specific technical solution is as follows:

[0049] Step S1: Construct the state space S for system input and state representation. Acquire and organize the system input, including the current state of the UAV (3D position coordinates, linear velocity vector, acceleration vector), window information (center point coordinates, length and width dimensions, 3D spatial orientation angle), and environmental obstacle information; establish the state space representation. ,in , , These represent the position, velocity, and acceleration vectors of the UAV, respectively. , , These represent the window's position (pos), size (size), and orientation (orient), respectively. This indicates information about environmental obstacles. The drone's status information is acquired in real time through onboard sensors, while window information is extracted through visual sensors or a lidar perception system.

[0050] Step S2: Construct the reinforcement learning action space and trajectory parameterization. Design the reinforcement learning action space. ( Indicates the spacing between trajectory control points. This indicates the execution time of the trajectory and controls the speed timing characteristics during flight. The coefficient of the trajectory smoothness penalty term. As the weight of the collision risk penalty item, Then it is the coefficient of the dynamic feasibility penalty term), and the trajectory parameterization adopts uniformity. spline curve Number of control points The number of nodes is adaptively determined based on trajectory complexity and window distance, typically ranging from 715. Using a uniform distribution, where ( (where the order is 1). The initial control points are generated by fitting a relaxed B-spline that directly connects the start point, the window center, and the end point. The position of the control points is then finely adjusted by a reinforcement learning agent.

[0051] Step S3: Design the trajectory optimization objective function. Construct a comprehensive objective function based on EGOPlanner: Each weight coefficient is adaptively adjusted by the reinforcement learning agent;

[0052] The smoothness penalty is calculated using the following formula:

[0053]

[0054] in, Let i be the acceleration vector of the i-th segment of the trajectory. This corresponds to the acceleration control point. This ensures the smoothness of the trajectory and energy efficiency;

[0055] Collision Penalty Calculated using the ESDFfree method:

[0056]

[0057] in, Indicates the first One trajectory control point, The total number of control points. To and The number of relevant obstacle point pairs, For the The control point and the first The local collision cost term between pairs of obstacle points. Let the safety threshold be... Then define intermediate variables. ,in Points on the surface of the obstacle. From point to The unit vector, further defined Therefore, the local cost function Expressed as:

[0058]

[0059] For window frames, an analytical distance function was specially designed, which eliminates the need to pre-calculate the distance field and significantly improves computational efficiency;

[0060] Dynamic feasibility penalty item Designed as follows:

[0061]

[0062]

[0063] Where the subscript i indicates the number of trajectory segments, and v, a, and j represent velocity, acceleration, and jerk, respectively. , , For each weight coefficient, It is a piecewise quadratic penalty function. The maximum allowable value for each physical quantity. This indicates the number of control points.

[0064] Step S4: Set window-specific trajectory constraints and construct the window's geometric model to ensure the drone safely passes through the window. The window is represented as a combination of a rectangular plane and a bounding box, with its center point located... ,size and normal vector The window crossing point is uniquely determined; the constraints are: a) the crossing point must be located inside the window and maintain a safe distance from the window frame edge; b) the angle between the velocity direction and the normal direction of the window plane during crossing is less than a preset threshold; c) the crossing speed is greater than a minimum threshold. To ensure a quick passage.

[0065] Step S5: Design the reinforcement learning reward function and train the reinforcement learning agent using the proximal policy optimization algorithm. Construct the comprehensive reward function: .

[0066] in, , , These are weighting coefficients. It's a reward for passing through the window. It is an efficiency reward. In the proposed reward function R, three types of trajectory costs are first used. (smoothness), (Collision distance) and (Dynamic feasibility) constitutes a penalty term, which is then multiplied by the normalized weights. , ,and Each weight takes a value between 0 and 1 and satisfies + + =1, which keeps the policy constant in the early stages of training and allows for adaptive fine-tuning during the learning process; then two positive incentives are added: one is the window crossing reward. Secondly, efficiency rewards This comprehensive approach aims to achieve faster and more energy-efficient window passage while ensuring safety and dynamic constraints.

[0067] The window crossing reward Defined as:

[0068]

[0069] in, It's the progress of the journey. , , .

[0070] The efficiency reward Defined as:

[0071]

[0072] Where T is the time taken to traverse, >0 is the exponential adjustment factor. >0 represents the efficiency weighting coefficient.

[0073] Training a reinforcement learning agent. The reinforcement learning agent is trained using the Proximal Policy Optimization (PPO) algorithm to maximize cumulative rewards; the agent network architecture is designed, including:

[0074] a) Feature extraction layer: Uses a 3D convolutional neural network to process environmental obstacle information, with the input being... The occupied grid is processed through 3 convolutional layers; a fully connected network is used to process window geometry features and drone state information;

[0075] b) Feature fusion layer: This layer stitches together environmental features, window features, and UAV status features and fuses them through a two-layer fully connected network (256 and 128 hidden units);

[0076] c) Dual-head output layer: The policy head outputs the mean and variance, which are used to sample actions; the value head estimates the state value, which is used to optimize training;

[0077] The training process includes:

[0078] a) Construct diverse training environments, including windows of different sizes, locations, and orientations;

[0079] b) Introduce a learning strategy that gradually transitions from simple scenarios (front view, wide window) to complex scenarios (tilted view, narrow window);

[0080] c) Employ an experience replay buffer to store and reuse successful traversal trajectories, accelerating the learning process;

[0081] d) Implement multi-core parallel training to improve sample efficiency and training speed.

[0082] Step S6: Generate and optimize the trajectory based on the parameters output by the reinforcement learning agent, and output the final trajectory for the UAV to execute. The system operation flow is as follows:

[0083] a) The environmental perception module acquires window information and obstacle distribution to construct a state representation;

[0084] b) The reinforcement learning agent outputs action parameters based on the current state, including control point adjustment, time coefficient, and optimization weights;

[0085] c) Perform EGOPlanner-based trajectory optimization using the adjusted parameters to generate a preliminary trajectory;

[0086] d) Check if the trajectory meets the window crossing constraint. If not, return to step b and readjust the parameters.

[0087] e) Conduct a trajectory dynamics feasibility test and calculate the exceedance rate: ;

[0088] f) If Then time reallocation will be performed: The trajectory was optimized using anisotropic curve fitting methods. While maintaining the trajectory shape, dynamic constraints are satisfied;

[0089] g) Trajectory refinement and smoothing to ensure that the generated trajectory meets the requirements of the UAV control system;

[0090] h) The final trajectory is converted into a sequence of control commands that can be executed by the UAV, and then executed by a high-frequency closed-loop controller.

[0091] Example 2

[0092] Figure 1This is an architecture diagram of a trajectory planning method for UAVs to quickly pass through windows, based on the present invention. The diagram illustrates a flowchart of a trajectory planning system for UAVs to quickly pass through windows. This system integrates gradient optimization and reinforcement learning methods to construct a closed-loop planning framework from sensor input to trajectory execution. The system first collects environmental data through a sensor system. The environmental perception module analyzes the window position, attitude, and obstacle distribution to generate state information and construct the current system state S. This state is then input into a reinforcement learning agent to generate a parameter adjustment strategy to guide subsequent trajectory generation and optimization. The strategy output affects the trajectory parameter configuration, including the control point spacing ΔCP, trajectory duration τ, smoothness penalty coefficient λs, collision penalty coefficient λc, and dynamic feasibility penalty coefficient λd. These parameters are fed into the EGO-Planner module to generate a preliminary trajectory. The trajectory then enters a detection and evaluation phase to determine whether it meets the window structure constraints and dynamic feasibility. If the constraints are not met, a trajectory time reallocation or strategy update mechanism is triggered. If the requirements are met, anisotropic curve fitting and trajectory refinement continue to generate a final smooth and executable trajectory. The final trajectory is sent from the trajectory execution module to the UAV control system to realize the window-passing maneuver. The entire system introduces a reward function calculation and training feedback mechanism to send the trajectory evaluation results and actual execution effects back to the reinforcement learning module, enabling adaptive policy updates. This forms a closed-loop trajectory planning architecture of perception-decision-execution-feedback, effectively improving the path generation quality, mission adaptability, and flight safety of UAVs in structured narrow-channel environments.

[0093] See Figure 1 As shown, the trajectory planning system architecture proposed in this invention includes three main parts: an environment perception module, a reinforcement learning agent, and an EGOPlanner optimization module.

[0094] The environmental perception module acquires window information and obstacle distribution: This module is responsible for acquiring window boundary information and surrounding obstacle distribution data in the current scene, providing fundamental perception support for state construction and trajectory optimization. In a typical implementation, this module primarily relies on a binocular camera for visual perception, using real-time captured binocular images to achieve depth estimation, thereby obtaining a dense depth map and a 3D spatial point cloud. For window recognition, the system utilizes a YOLOv5-based convolutional neural network to detect window targets in the left-eye image, combines binocular matching to calculate its corresponding spatial location, and further estimates the window's geometric boundaries and normal direction through image plane fitting and edge extraction algorithms, achieving accurate modeling of the window's center point, dimensions, and attitude angle. For obstacle mapping, the system constructs the binocular depth map as a local sparse point cloud set, or projects it as a 2D grid map, to represent the obstacle distribution within the flight space, supporting subsequent trajectory feasibility assessment and obstacle avoidance gradient construction.

[0095] The reinforcement learning agent adjusts its output parameters based on the current state: the state representation module constructs the current state based on the perception results. The data includes factors such as the relative position, speed, orientation, and obstacle density distribution between the drone and the window, which serve as inputs to the reinforcement learning agent. The reinforcement learning agent uses the Proximal Policy Optimization (PPO) algorithm for policy learning. During the training phase, a reward function containing comprehensive metrics such as crossing success rate, trajectory smoothness, and collision penalty is constructed to drive the policy network to learn parameter adjustment rules. The network structure can adopt a multilayer perceptron architecture, typically a three-layer fully connected network with 64 neurons per layer. The activation function is ReLU, and the learning rate is set to a specific value. The discount factor is 0.99, and training employs parallel sampling and experience replay mechanisms to accelerate convergence. During the execution phase, the reinforcement learning agent outputs an action vector based on its current state. It is used to adjust the control point spacing, time allocation, and weights of each penalty term in the cost function during the trajectory optimization process.

[0096] After receiving the parameter configuration, EGO-Planner generates a preliminary trajectory. This process is based on the B-spline trajectory modeling method, using trajectory smoothness, collision distance, and dynamic feasibility as cost functions to construct the following optimization problem:

[0097] in For the set of trajectory control points, , , These represent the smoothness, collision penalty, and dynamic feasibility cost terms, respectively. , , These are the coefficients for smoothness, collision penalty, and dynamic feasibility penalty terms. During trajectory optimization, if a conflict with dynamic constraints is detected, the system will automatically trigger the trajectory time reallocation and anisotropic curve fitting module to further optimize the spatial feasibility and dynamic safety of the trajectory, and finally output a trajectory that satisfies the crossing constraints for control execution.

[0098] Figure 2 This demonstrates the key trajectory structure design and constraint region division for a drone navigating through a window. The path, from the starting point to the end point, sequentially passes through control point 1, control point 2, and the window center to achieve precise constraints and safe control of the dynamic process. In the initial segment, velocity and acceleration constraints are applied to the entry region, limiting them to their maximum speeds. ( (for maximum speed limit) and maximum acceleration ( (To limit maximum acceleration), ensuring the drone remains controllable and safe before entering the window area; the trajectory then approaches the center of the window, with a safe distance constraint between the window center and the trajectory. ( This represents the minimum safe distance constraint for the drone during window passage, and geometrically adjusts the path direction based on the window normal vector to enhance the geometric alignment between the trajectory and the window. Within the window area, the path must satisfy passage constraints, including perpendicularly crossing the window normal direction and maintaining a distance not less than [a certain value]. The system considers spatial redundancy distance and speed limitations; the exit area after crossing requires the UAV to quickly restore a stable attitude and controllable speed. The overall trajectory uses multiple control points to segmentally constrain speed, acceleration, and spatial distance, thereby constructing a crossing trajectory that meets physical feasibility, safety, and dynamic continuity, providing an achievable reference path structure for autonomous flight of UAVs in complex scenarios.

[0099] like Figure 2 As shown, the trajectory for crossing the window must meet specific constraints: the crossing point is located inside the window and maintains a safe distance from the edge; the angle between the velocity direction and the normal direction of the window plane is less than a preset threshold; and the crossing speed is greater than a minimum threshold.

[0100] This invention automatically adjusts the parameter configuration of EGOPlanner through reinforcement learning, which not only improves the computational efficiency of trajectory planning, but also enhances the system's adaptability to different windows, while achieving a better balance between safety and speed. It is particularly suitable for scenarios where drones pass through windows at high speed.

[0101] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. A trajectory planning method for a drone to quickly pass through a window, characterized in that, Includes the following steps: Step S1: Construct the state space S of the system input and state representation; Step S2: Construct the reinforcement learning action space With trajectory parameterization; Step S3: Design the trajectory optimization objective function; Step S4: Set window-specific trajectory constraints to ensure the drone safely passes through the window; Step S5: Design a reinforcement learning reward function and train a reinforcement learning agent using a proximal policy optimization algorithm; Step S6: Generate and optimize the trajectory based on the parameters output by the reinforcement learning agent, and output the final trajectory for the UAV to execute.

2. The trajectory planning method for a drone rapidly passing through a window according to claim 1, characterized in that: In step S1: the state space S is designed as follows: in, , , These represent the position, velocity, and acceleration vectors of the UAV, respectively. , , These indicate the window's location, size, and orientation, respectively. Indicates information about environmental obstacles; In step S2, the action space Designed as follows: ,in, express Adjustment amount of spline control points, This represents the trajectory time adjustment coefficient, where s, c, and d represent smoothness, collision avoidance, and dynamics, respectively. , , These represent the weights of smoothness, collision avoidance, and dynamic feasibility in EGOPlanner, respectively; trajectory parameterization uses uniformity. spline curve By order Control points and node vectors The only certainty is that among them , This indicates the number of control points.

3. The trajectory planning method for a drone rapidly passing through a window according to claim 2, characterized in that: The trajectory optimization objective function in step S3 is defined as: ,in, For smoothness penalty term, For collision penalties, For dynamic feasibility penalty term; for smoothness penalty term Defined as: ； in, Let i be the acceleration vector of the i-th segment of the trajectory. This corresponds to the acceleration control point.

4. The trajectory planning method for a drone rapidly passing through a window according to claim 3, characterized in that: In step S3, the collision penalty term The method employs a method that does not require pre-construction of the Euclidean signed distance field. It constructs a local collision cost term based on the geometric relationship between the control point and the obstacle surface, and accumulates it to form a global collision cost function, thereby realizing gradient modeling and optimization guidance of collision risk.

5. The trajectory planning method for a drone rapidly passing through a window according to claim 4, characterized in that: In step S3, the dynamic feasibility penalty term Based on the limitations of velocity, acceleration, and jerk, the design is as follows: ； ； Where the subscript i indicates the number of trajectory segments, and v, a, and j represent velocity, acceleration, and jerk, respectively. , , For each weight coefficient, It is a piecewise quadratic penalty function. The maximum allowable value for each physical quantity. This indicates the number of control points.

6. The trajectory planning method for a drone rapidly passing through a window according to claim 1, characterized in that: In step S4, the window-specific trajectory constraints include: a) The crossing point must be located inside the window and maintain a safe distance from the window frame edge; b) The angle between the velocity direction and the normal direction of the window plane during crossing must be less than a preset threshold; c) The crossing speed must be greater than a minimum threshold. To ensure a quick passage.

7. The trajectory planning method for a drone rapidly passing through a window according to claim 3, characterized in that: In step S5, the reward function Designed as follows: ; in, , , These are weighting coefficients. It's a reward for passing through the window. It is an efficiency reward; in the proposed reward function R, three types of trajectory costs are first used. , and The penalty term is formed and multiplied by the normalized weight. , ,and Each weight takes a value between 0 and 1 and satisfies + + =1, which is kept constant during the initial training phase to allow the policy to adaptively fine-tune during the learning process; then two positive incentives are added: one is the window-crossing reward. Secondly, efficiency rewards This comprehensive approach aims to achieve faster and more energy-efficient window crossings while ensuring safety and dynamic constraints.

8. The trajectory planning method for a drone rapidly passing through a window according to claim 7, characterized in that: The window crossing reward Defined as: ； in, It's the progress of the journey. , , .

9. The trajectory planning method for a drone rapidly passing through a window according to claim 7, characterized in that: The efficiency reward Defined as: ； Where T is the time taken to traverse, >0 is the exponential adjustment factor. >0 represents the efficiency weighting coefficient.

10. The trajectory planning method for a drone rapidly passing through a window according to claim 1, characterized in that: In step S6, when the initial trajectory does not meet the dynamic constraints, trajectory time reallocation is performed, and the excess ratio is calculated: ； in, It is the first The velocity of the segment trajectory, This is the maximum speed allowed; It is the first The acceleration of the segment trajectory, It is the maximum allowable acceleration; It is the first The acceleration of the segment trajectory, It is the maximum permissible jerk; The new time interval is set as follows: Then, the trajectory is optimized using anisotropic curve fitting methods: ； in It is a normalized time variable. and These are axial and radial displacements, respectively. and These are the semi-major and semi-minor axes of the ellipse.

11. A trajectory planning system for a drone to quickly pass through a window, comprising an environmental perception module, a reinforcement learning agent, and an EGOPlanner optimization module, characterized by: Environmental perception module: responsible for acquiring window boundary information and surrounding obstacle distribution data in the current scene; this module relies on a binocular camera for visual perception, and uses real-time captured binocular images to achieve depth estimation, thereby obtaining a dense depth map and a three-dimensional spatial point cloud; In terms of window recognition, a YOLOv5-based convolutional neural network is used to detect window targets in the left eye image. Combined with binocular matching, the corresponding spatial position is calculated. The geometric boundary and normal direction of the window are further estimated through image plane fitting and edge extraction algorithms, so as to achieve accurate modeling of the window center point, width and height dimensions and attitude angle. In terms of obstacle mapping, the system constructs the binocular depth map as a local sparse point cloud set, or projects it as a two-dimensional grid map to represent the distribution of obstacles in the flight space, supporting subsequent trajectory feasibility assessment and obstacle avoidance gradient construction. Reinforcement learning agent module: Constructs the current state based on the perception results. The data includes the relative position, speed, orientation, and obstacle density distribution between the drone and the window, which serve as inputs to the reinforcement learning agent. The Proximal Policy Optimization (PPO) algorithm is used for policy learning. During the training phase, a reward function containing a comprehensive index of crossing success rate, trajectory smoothness, and collision penalty is constructed to drive the policy network to learn parameter adjustment rules. The network structure adopts a three-layer fully connected network, with multiple neurons in each layer. The ReLU activation function is used, and parallel sampling and experience replay mechanisms are employed to accelerate convergence during training. During the execution phase, the reinforcement learning agent outputs an action vector based on its current state. This is used to adjust the control point spacing, time allocation, and weights of each penalty term in the cost function during trajectory optimization. express Adjustment amount of spline control points, This represents the trajectory time adjustment factor. , , These represent the weights of smoothness, collision avoidance, and dynamic feasibility in EGOPlanner, respectively. The EGO-Planner optimization module: After receiving the parameter configuration, it generates a preliminary trajectory. Based on the B-spline trajectory modeling method, it constructs the following optimization problem with trajectory smoothness, collision distance, and dynamic feasibility as cost functions: , in For the set of trajectory control points, , , These represent the smoothness, collision penalty, and dynamic feasibility cost terms, respectively. , , These represent the weights of smoothness, collision avoidance, and dynamic feasibility in EGOPlanner, respectively. During trajectory optimization, if a conflict with dynamic constraints is detected, the system will automatically trigger the trajectory time redistribution and anisotropic curve fitting module, and finally output a trajectory that satisfies the crossing constraints for control execution.

12. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored program, wherein the program, when executed, controls the device on which the non-volatile storage medium resides to perform the method described in any one of claims 1 to 10.

13. A terminal device, characterized in that, The terminal device includes: a processor, a memory, a communication interface, and a bus; the processor, the memory, and the communication interface are connected through the bus and communicate with each other; the memory stores executable program code; the processor reads the executable program code stored in the memory to run a program corresponding to the executable program code, so as to execute the method as described in any one of claims 1-10 above.

Images