A trajectory planning method for rapid crossing of a window by a drone

Abstract

The application discloses a trajectory planning method for rapid crossing of a window by a UAV, which combines the ESDFfree gradient optimization of EGOPlanner with adaptive parameter adjustment of reinforcement learning, solves the problems of low calculation efficiency, complex parameter adjustment, difficult balance between safety and speed and the like in the prior art through steps of constructing a state space, designing an action space, optimizing an objective function, setting window-specific constraints, designing a reward function and training a reinforcement learning agent, avoids time-consuming distance field calculation, improves planning speed by one order of magnitude, automatically optimizes trajectory parameters through reinforcement learning to adapt to windows of different sizes and angles, comprehensively considers safety margin and passing speed, enables the UAV to cross the window at the maximum possible speed under the premise of ensuring safety, and automatically ensures that the generated trajectory meets dynamic constraints through trajectory time redistribution and anisotropic curve fitting.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle autonomous navigation and trajectory planning, and particularly relates to a trajectory planning method for unmanned aerial vehicle rapid window crossing based on gradient optimization and reinforcement learning combination. BACKGROUND

[0002] With the rapid development of unmanned aerial vehicle technology, unmanned aerial vehicles are increasingly widely used in search and rescue, building detection, urban express delivery and other scenarios. In these application scenarios, unmanned aerial vehicles often need to cross windows, door frames and other narrow passages, which puts high requirements on trajectory planning algorithms. Unmanned aerial vehicles need to quickly and safely cross these narrow passages while maintaining sufficient safety margins and meeting the dynamics constraints of unmanned aerial vehicles.

[0003] Currently, there are mainly the following categories of trajectory planning methods for unmanned aerial vehicle window crossing:

[0004] 1. Sampling-based methods: such as RRT (Rapidly Exploring Random Tree) and its variants. This kind of method can search for feasible paths in complex environments, but it is difficult to guarantee the smoothness and optimality of the path, and usually needs post-processing to meet the dynamics constraints.

[0005] 2. Optimization-based methods: such as CHOMP, STOMP and other gradient optimization methods. This kind of method can generate smooth and dynamically feasible trajectories by defining an objective function to optimize the trajectory. However, most optimization-based methods rely on the Euclidean signed distance field (ESDF) to calculate the gradient, and the construction of ESDF is very time-consuming, occupying more than 70% of the computing resources of the planning process, which is unacceptable for window crossing scenarios with high real-time requirements.

[0006] 3. Learning-based methods: such as using deep reinforcement learning to directly learn control strategies. This kind of method can generate control instructions end-to-end, but its performance is heavily dependent on the quality of training data and generalization ability, and it is difficult to guarantee safety and reliability.

[0007] The recently proposed EGOPlanner is a gradient optimization method without ESDF, which greatly reduces the computational complexity compared to traditional methods. However, it still has the following problems: 1) Lack of targeted optimization for specific scenarios such as window crossing. 2) Parameter adjustment relies on expert experience and is difficult to automatically adapt to windows of different sizes and angles. 3) It is difficult to balance safety and crossing speed when facing high-speed window crossing. SUMMARY

[0008] The purpose of the present application is to solve the problems of low calculation efficiency, complex parameter adjustment, and difficulty in balancing safety and speed in the existing trajectory planning method for unmanned aerial vehicles (UAVs) crossing windows. The present application provides an efficient and adaptive trajectory planning method that enables UAVs to quickly and safely cross windows of various sizes and angles.

[0009] To achieve the above-mentioned purpose, the present application proposes a UAV window crossing trajectory planning method based on the combination of EGOPlanner and reinforcement learning. EGOPlanner is used as the basic trajectory generation module, and reinforcement learning is used for adaptive optimization of parameter configuration. The specific technical solutions are as follows:

[0010] Step S1, construct the system input and state representation. Obtain and organize the system input, including the current state of the UAV (three-dimensional position coordinates, linear velocity vector, acceleration vector), window information (center point coordinates, length and width dimensions, three-dimensional space orientation angle), and environmental obstacle information; establish the state space representation The state information of the UAV is obtained in real time by the onboard sensor, and the window information is obtained by the vision sensor or laser radar perception system.

[0011] Step S2, construct the reinforcement learning action space and trajectory parameterization. Design the reinforcement learning action space , and the trajectory parameterization adopts uniform spline curve ; the number of control points is adaptively determined according to the trajectory complexity and window distance, and the general value range is 715, and the node vector is uniformly distributed, wherein ; the initial control points are generated by fitting a relaxed B-spline curve directly connecting the starting point, window center, and ending point, and the subsequent control point positions are fine-tuned by the reinforcement learning agent.

[0012] Step S3, design the trajectory optimization objective function. Construct the comprehensive objective function based on EGOPlanner: , and the weight coefficients are adaptively adjusted by the reinforcement learning agent; the smoothness penalty term ensures the smoothness and energy efficiency of the trajectory; the collision penalty term is calculated using the ESDFfree method: ; for the window frame, a special analytic distance function is designed, which does not require pre-computation of the distance field, significantly improving the calculation efficiency; the dynamics feasibility penalty term is designed as:

[0013] Step S4, construct the window geometry model. The window is represented as a combination of a rectangular plane and a bounding box, and the center point position , dimensions , and normal vector Unique determination; where the window crossing point constraints are: a) the crossing point must be located inside the window, maintaining a safe distance from the window frame edges b) the angle between the velocity direction during crossing and the normal direction of the window plane is less than a pre-set threshold c) the crossing velocity is greater than a minimum threshold , ensuring fast passage

[0014] Step S5, design reinforcement learning reward function. Construct a comprehensive reward function: .

[0015] Step S6, train reinforcement learning agent. Use the Proximal Policy Optimization algorithm (PPO) to train the reinforcement learning agent to maximize cumulative rewards; design the agent network architecture, including:

[0016] a) Feature extraction layer: use a 3D convolutional neural network to process environmental obstacle information, input as an occupancy grid, processed through 3 convolutional layers; use a fully connected network to process window geometric features and UAV state information;

[0017] b) Feature fusion layer: concatenate environmental features, window features, and UAV state features and fuse them through two fully connected networks (256 and 128 hidden units);

[0018] c) Dual-head output layer: the policy head outputs mean and variance for action sampling; the value head estimates state value for optimization training;

[0019] While implementing the training process, including:

[0020] a) Construct a diverse training environment, including windows of different sizes, positions, and orientations;

[0021] b) Introduce a curriculum learning strategy, gradually transitioning from simple scenarios (front, spacious windows) to complex scenarios (inclined, narrow windows);

[0022] c) Use an experience replay buffer to store and reuse successful crossing trajectories, accelerating the learning process;

[0023] d) Implement multi-core parallel training to improve sample efficiency and training speed.

[0024] Step S7, trajectory generation and execution. The system runs as follows:

[0025] a) The environment perception module obtains window information and obstacle distribution, constructing a state representation;

[0026] b) The reinforcement learning agent outputs action parameters based on the current state, including control point adjustment, time coefficient, and optimization weight;​

[0027] c) using the adjusted parameters to perform EGOPlanner-based trajectory optimization, generating a preliminary trajectory;

[0028] d) checking whether the trajectory meets the window crossing constraint, and if not, returning to step b to adjust the parameters again;

[0029] e) performing trajectory dynamics feasibility check, calculating the over-limit ratio: ;

[0030] f) if , performing time redistribution: , and using anisotropic curve fitting method to optimize the trajectory, maintaining the shape of the trajectory while meeting the dynamics constraints;

[0031] g) trajectory refinement and smoothing processing to ensure that the generated trajectory meets the requirements of the UAV control system;

[0032] h) converting the final trajectory into a sequence of control instructions executable by the UAV, and executing it through a high-frequency closed-loop controller.

[0033] The application also discloses a trajectory planning system for rapid window crossing of a UAV, comprising an environment perception module, a reinforcement learning agent, and an EGOPlanner optimization module, characterized in that:

[0034] The environment perception module is responsible for obtaining window boundary information and surrounding obstacle distribution data in the current scene; this module relies on binocular cameras for visual perception, and through real-time capture of binocular image pairs, depth estimation is achieved to obtain dense depth maps and three-dimensional spatial point clouds;

[0035] In terms of window recognition, a convolutional neural network based on YOLOv5 is used to detect window targets in the left eye image, and the corresponding spatial position is calculated by binocular matching, and the geometric boundary and normal direction of the window are further estimated through image plane fitting and edge extraction algorithm to realize accurate modeling of the window center point, width and height, and attitude angle;

[0036] In terms of obstacle mapping, the system constructs a local sparse point cloud set from the binocular depth map, or projects it into a two-dimensional grid map, which is used to represent the obstacle distribution in the flight space, supporting subsequent trajectory feasibility evaluation and obstacle avoidance gradient construction;

[0037] The reinforcement learning agent module: according to the perception results, the current state , including the relative position, velocity, orientation, obstacle density distribution elements between the UAV and the window, as the input of the reinforcement learning agent; the proximal policy optimization algorithm PPO is used for policy learning, and in the training stage, a reward function containing the comprehensive indexes of the penetration success rate, trajectory smoothness and collision penalty is constructed to drive the policy network to learn the parameter adjustment rule; the network structure adopts a three-layer fully connected network, each layer contains multiple neurons, the activation function uses ReLU, and the training uses parallel sampling and experience replay mechanism to accelerate convergence; in the execution stage, the reinforcement learning agent outputs an action vector , for adjusting the control point spacing, time allocation and the weights of each penalty term in the cost function in the trajectory optimization process, wherein, represents the adjustment amount of the spline control point, , represents the trajectory time adjustment coefficient, , , respectively represent the weights of smoothness, collision avoidance and dynamics feasibility in EGOPlanner;

[0038] EGO-Planner optimization module: after receiving the parameter configuration, a preliminary trajectory is generated, based on the B-spline trajectory modeling method, taking the trajectory smoothness, collision distance and dynamics feasibility as the cost function, the following optimization problem is constructed: ,

[0039] wherein is the set of trajectory control points, , , respectively represent the smoothness, collision penalty and dynamics feasibility cost items, , , respectively represent the weights of smoothness, collision avoidance and dynamics feasibility in EGOPlanner; in the trajectory optimization process, if a conflict with the dynamics constraint is detected, the system will automatically trigger the trajectory time redistribution and anisotropic curve fitting module, and finally output a trajectory that meets the penetration constraint for control execution.

[0040] Advantages

[0041] 1) Improved computational efficiency: the ESDFfree gradient calculation method of EGOPlanner is adopted, avoiding time-consuming distance field calculation, and the planning speed is improved by an order of magnitude, enabling the UAV to adjust the trajectory in real time during flight. Enhanced adaptability: through automatic optimization of trajectory parameters by reinforcement learning, the system can adapt to windows of different sizes, shapes and angles without manual parameter tuning, greatly improving the versatility and robustness of the system.

[0042] 2) Safety and speed balance optimization: The reinforcement learning reward function takes into account both safety margin and speed of passage, enabling the drone to pass through the window at the maximum possible speed while ensuring safety.

[0043] 3) Automatic satisfaction of dynamic constraints: Through trajectory time redistribution and anisotropic curve fitting, the generated trajectory is automatically ensured to meet the dynamic constraints of the UAV without the need for additional post-processing.

[0044] 4) Improved planning success rate: Compared with traditional methods, the success rate of planning in complex environments is significantly improved, especially for narrow or tilted windows, the success rate is improved by about 30%. Attached Figure Description

[0045] Figure 1 This is a system architecture diagram of a trajectory planning method for a drone to quickly pass through a window, according to the present invention.

[0046] Figure 2 This is a schematic diagram illustrating the window's trajectory and specific constraints. Detailed Implementation

[0047] Example 1

[0048] This invention proposes a window-crossing trajectory planning method for UAVs based on a combination of EGOPlanner and reinforcement learning. EGOPlanner serves as the basic trajectory generation module, while reinforcement learning is used for adaptive optimization of parameter configuration. The specific technical solution is as follows:

[0049] Step S1: Construct the state space S for system input and state representation. Acquire and organize the system input, including the current state of the UAV (3D position coordinates, linear velocity vector, acceleration vector), window information (center point coordinates, length and width dimensions, 3D spatial orientation angle), and environmental obstacle information; establish the state space representation. ,in , , These represent the position, velocity, and acceleration vectors of the UAV, respectively. , , These represent the window's position (pos), size (size), and orientation (orient), respectively. This indicates information about environmental obstacles. The drone's status information is acquired in real time through onboard sensors, while window information is extracted through visual sensors or a lidar perception system.

[0050] Step S2: Construct the reinforcement learning action space and trajectory parameterization. Design the reinforcement learning action space. ( Indicates the spacing between trajectory control points. denotes the execution time length of the trajectory, controls the speed timing characteristics in the flight process, is the coefficient of the trajectory smoothness penalty term, is the weight of the collision risk penalty term, is the coefficient of the dynamics feasibility penalty term), the trajectory parameterization adopts a uniform spline curve ; the number of control points is adaptively determined according to the trajectory complexity and the window distance, generally taking a value range of 715, the node vector adopts a uniform distribution, wherein ( is the order); the initial control points are generated by fitting a relaxed B-spline directly connecting the start point, the window center and the end point, and the subsequent fine adjustment of the control point positions is performed by the reinforcement learning agent.

[0051] Step S3, design a trajectory optimization objective function. An integrated objective function based on EGOPlanner is constructed: , and each weight coefficient is adaptively adjusted by the reinforcement learning agent;

[0052] The smoothness penalty term is calculated by the following formula:

[0053]

[0054] wherein, is the acceleration vector of the i-th segment of the trajectory, is the corresponding acceleration control point. This term ensures the smoothness and energy efficiency of the trajectory;

[0055] collision penalty term is calculated by the ESDFfree method:

[0056]

[0057] wherein, denotes the i-th trajectory control point, is the total number of control points, is the number of obstacle point pairs related to , and is the local collision cost term between the i-th control point and the j-th obstacle point pair. Let the safety threshold be , then define the intermediate variable , wherein is the obstacle surface point, is the direction from to , and is the direction from to unit vector, further defined as . The local cost function is expressed as:

[0058]

[0059] For window frame, a special designed parsing distance function is used, which does not need to pre-compute distance field, significantly improves the computational efficiency;

[0060] The dynamic feasibility penalty term is designed as:

[0061]

[0062]

[0063] where subscript i denotes the number of trajectory segments, v, a, j denote velocity, acceleration, jerk, respectively, , , are weight coefficients, is a piecewise quadratic penalty function, is the maximum value allowed for each physical quantity, denotes the number of control points.

[0064] Step S4, set the window-specific trajectory constraints, build the window geometry model, and ensure the safety of the UAV crossing the window. The window is represented as a combination of a rectangular plane and a bounding box, which is uniquely determined by the center point position , size , and normal vector ; where the window crossing point constraints are: a) the crossing point must be located inside the window, and maintain a safe distance from the window frame edge; b) the angle between the velocity direction and the normal direction of the window plane during the crossing process is less than a preset threshold; c) the crossing velocity is greater than a minimum threshold to ensure fast passing.

[0065] Step S5, design a reinforcement learning reward function, and train the reinforcement learning agent through a proximal policy optimization algorithm. The comprehensive reward function is constructed as: .

[0066] where , , are weight coefficients, is the window crossing reward, is the efficiency reward. In the proposed reward function R, three types of trajectory costs (smoothness), (collision distance), and (Dynamics feasibility) constitute the penalty term, multiplied by the normalized weight , , and , where each weight value is between 0-1 and satisfies + + =1, which can be kept constant at the beginning of training, also allows the policy to adaptively fine-tune during the learning process; then add two positive incentives: one is the window crossing reward , the other is the efficiency reward . Thus comprehensively promote the policy to ensure safety and dynamic constraints while pursuing faster and more energy-efficient window crossing.

[0067] The window crossing reward is defined as:

[0068]

[0069] where, is the crossing progress, , , .

[0070] The efficiency reward is defined as:

[0071]

[0072] where T is the crossing time, >0 is the exponential adjustment factor, >0 is the efficiency weight coefficient.

[0073] Training reinforcement learning agent. The reinforcement learning agent is trained using the Proximal Policy Optimization (PPO) algorithm to maximize the cumulative reward; the agent network architecture is designed, including:

[0074] a) Feature extraction layer: use a 3D convolutional neural network to process environmental obstacle information, input is occupancy grid, processed by 3 convolutional layers; use a fully connected network to process window geometric features and UAV state information;

[0075] b) Feature fusion layer: concatenate environmental features, window features and UAV state features and fuse them through two fully connected networks (256 and 128 hidden units);

[0076] c) Double-headed output layer: the policy head outputs mean and variance for sampling actions; the value head estimates state value for optimization training;

[0077] The training process includes:

[0078] a) Construct a diverse training environment with windows of different sizes, positions, and orientations;

[0079] b) Introduce a curriculum learning strategy, gradually transitioning from simple scenarios (front, wide windows) to complex scenarios (inclined, narrow windows);

[0080] c) Employ an experience replay buffer to store and reuse successful traversal trajectories, accelerating the learning process;

[0081] d) Implement multi-core parallel training to improve sample efficiency and training speed.

[0082] Step S6, trajectory generation and optimization according to the parameters output by the reinforcement learning agent, output the final trajectory for the UAV to execute. The system running process is as follows:

[0083] a) The environment perception module obtains window information and obstacle distribution, and constructs the state representation;

[0084] b) The reinforcement learning agent outputs action parameters according to the current state, including control point adjustment amount, time coefficient and optimization weight;

[0085] c) Use the adjusted parameters to perform EGOPlanner-based trajectory optimization to generate a preliminary trajectory;

[0086] d) Check if the trajectory meets the window traversal constraints, if not, return to step b to adjust the parameters;

[0087] e) Perform trajectory dynamics feasibility check, calculate the over-limit ratio: ;

[0088] f) If , perform time reallocation: , and use anisotropic curve fitting method to optimize the trajectory: , maintain the shape of the trajectory while meeting the dynamics constraints;

[0089] g) Trajectory refinement and smoothing processing to ensure that the generated trajectory meets the requirements of the UAV control system;

[0090] h) Convert the final trajectory into a sequence of control instructions executable by the UAV, and execute it through a high-frequency closed-loop controller.

[0091] Example 2

[0092] Figure 1is a schematic diagram of a trajectory planning method for a UAV to quickly cross a window. The diagram shows a trajectory planning system flowchart for a UAV to quickly cross a window task. The system combines gradient optimization and reinforcement learning methods to construct a closed-loop planning framework from perception input to trajectory execution. The system first collects environmental data through a sensor system, analyzes the window position, attitude, and obstacle distribution through an environmental perception module, generates state information, and constructs the current system state S. The state is then input into the reinforcement learning agent to generate a parameter adjustment strategy, which is used to guide the subsequent trajectory generation and optimization process. The strategy output affects the trajectory parameter configuration, including the control point spacing ΔCP, the trajectory duration τ, the smoothness penalty coefficient λs, the collision penalty coefficient λc, and the dynamics feasibility penalty coefficient λd. These parameters are passed into the EGO-Planner module to generate a preliminary trajectory. The trajectory then enters the detection and evaluation stage to determine whether it meets the window structure constraints and dynamics feasibility; if the constraints are not met, the trajectory time redistribution or strategy update mechanism is triggered; if the requirements are met, anisotropic curve fitting and trajectory refinement are performed to generate a final smooth and executable trajectory. The final trajectory is issued to the UAV control system by the trajectory execution module to achieve the window crossing action. The entire system introduces a reward function calculation and training feedback mechanism to return the trajectory evaluation results and actual execution effects to the reinforcement learning module, enabling adaptive updates of the strategy, thereby forming a perception-decision-execution-feedback closed-loop trajectory planning architecture, effectively improving the UAV's path generation quality, task adaptability, and flight safety in structured narrow channel environments.

[0093] Referring to Figure 1 As shown, the trajectory planning system architecture proposed by the present application includes an environmental perception module, a reinforcement learning agent, and an EGOPlanner optimization module.

[0094] The environmental perception module obtains window information and obstacle distribution: The environmental perception module is responsible for obtaining window boundary information and surrounding obstacle distribution data in the current scene, providing basic perception support for state construction and trajectory optimization. In a typical implementation, this module mainly relies on binocular cameras for visual perception, and performs depth estimation through real-time captured binocular image pairs to obtain dense depth maps and three-dimensional spatial point clouds. In window recognition, the system uses a convolutional neural network based on YOLOv5 to detect window targets in the left eye image, calculates the corresponding spatial position, and further estimates the geometric boundary and normal direction of the window through image plane fitting and edge extraction algorithms to accurately model the window center point, width and height, and attitude angle. In obstacle mapping, the system constructs a local sparse point cloud set from the binocular depth map, or projects it into a two-dimensional grid map to represent the obstacle distribution in the flight space, supporting subsequent trajectory feasibility evaluation and obstacle avoidance gradient construction.

[0095] The reinforcement learning agent outputs parameter adjustment according to the current state: the state representation module constructs the current state according to the perception results , including the relative position, velocity, orientation, obstacle density distribution, etc. between the UAV and the window, as the input of the reinforcement learning agent. The reinforcement learning agent adopts the proximal policy optimization algorithm PPO for policy learning, and in the training stage, a reward function containing comprehensive indicators such as penetration success rate, trajectory smoothness and collision penalty is constructed to drive the policy network to learn the parameter adjustment rule; the network structure can adopt a multi-layer perceptron architecture, and a typical setting is a three-layer fully connected network, each layer containing 64 neurons, the activation function using ReLU, the learning rate being , the discount factor being 0.99, and the training using parallel sampling and experience replay mechanism to accelerate convergence. In the execution stage, the reinforcement learning agent outputs an action vector according to the current state, which is used to adjust the control point spacing, time allocation and the weight of each penalty term in the cost function in the trajectory optimization process.

[0096] The EGO-Planner generates a preliminary trajectory after receiving the parameter configuration: this process is based on the B-spline trajectory modeling method, taking trajectory smoothness, collision distance and dynamic feasibility as the cost function, and constructing the following optimization problem:

[0097] where is the set of trajectory control points, , , respectively represent the smoothness, collision penalty and dynamic feasibility cost terms, , , are the smoothness, collision penalty and dynamic feasibility penalty coefficients. In the trajectory optimization process, if a conflict with the dynamic constraint is detected, the system will automatically trigger the trajectory time redistribution and anisotropic curve fitting module to further optimize the spatial feasibility and dynamic safety of the trajectory, and finally output a trajectory that meets the penetration constraints for control execution.

[0098] Figure 2 The key trajectory structure design and constraint region division in the process of the UAV penetrating the window are shown, in which the path passes through control point 1, control point 2 and the window center in turn from the starting point to the ending point to realize accurate constraint and safety control on the penetration dynamic process. In the starting section, the entrance region imposes velocity and acceleration constraints, respectively limited to the maximum velocity (Vmax is the maximum velocity limit) and the maximum acceleration (a to ensure controllability and safety of the UAV before entering the window region; then the trajectory approaches the window center with a safety distance constraint between the window center and the trajectory represents the minimum safety distance constraint of the UAV during the window crossing), and in the geometric sense, the window normal vector is used as the path direction adjustment basis, which enhances the alignment degree of the trajectory and the window geometry. In the window region, the path needs to meet the crossing constraint conditions, including crossing the window normal direction vertically, maintaining a spatial redundancy distance not less than , and a speed limit condition; while the exit region after crossing requires the UAV to quickly recover to a state of smooth attitude and controllable speed. The overall trajectory is segmented by multiple control points to segmentally constrain the velocity, acceleration and spatial distance, thereby constructing a crossing trajectory that meets the physical feasibility, safety and dynamic continuity, providing an implementable reference path structure for UAV autonomous flight in complex scenarios.

[0099] As shown in Figure 2 , the window crossing specific constraint requires that the trajectory must meet: the crossing point is located inside the window and maintains a safety distance from the edge; the angle between the velocity direction and the window plane normal direction is less than a preset threshold; the crossing velocity is greater than a minimum threshold.

[0100] The present application automatically adjusts the parameter configuration of EGOPlanner through reinforcement learning, not only improves the calculation efficiency of trajectory planning, but also enhances the adaptive ability of the system to different windows, while achieving a better balance between safety and speed, especially suitable for the scenario of high-speed window crossing of UAV.

[0101] The basic principles, main features and advantages of the present application are shown and described above. Those skilled in the art should understand that the present application is not limited by the above examples, and the above examples and descriptions in the specification are only the principles of the present application. Without departing from the spirit and scope of the present application, various changes and improvements can be made to the present application, and these changes and improvements all fall within the scope of the claimed present application. The scope of protection claimed by the present application is defined by the appended claims and their equivalents.​

Claims

1. A trajectory planning method for rapid crossing of a window by a UAV, characterized in that, The method comprises the following steps: Step S1, constructing a state space S of system input and state representation; Step S2, constructing reinforcement learning action space Parametrization with trajectories; Step S3, designing a trajectory optimization objective function; Step S4, setting window-specific trajectory constraints to ensure the safety of the UAV crossing the window; Step S5, designing a reinforcement learning reward function to train a reinforcement learning agent through a proximal policy optimization algorithm; Step S6, generating and optimizing a trajectory according to the parameters output by the reinforcement learning agent, and outputting a final trajectory for the UAV to execute; wherein in step S1: the state space S is designed as: wherein, , , respectively represent the position, velocity and acceleration vector of the UAV, , , respectively represent the position, size and orientation of the window, represent the environmental obstacle information; In step S2, the action space is designed as: wherein, represents the adjustment amount of the spline control point, represents the trajectory time adjustment coefficient, s, c, and d respectively represent smoothness, collision avoidance, and dynamics feasibility, , , respectively represent the weights of smoothness, collision avoidance, and dynamics feasibility in the EGOPlanner; the trajectory parameterization adopts a uniform spline curve which is uniquely determined by the order , the control point , and the node vector , wherein , represents the number of control points; The trajectory optimization objective function in step S3 is defined as: wherein, is a smoothness penalty term, is a collision penalty term, is a dynamics feasibility penalty term; the smoothness penalty term is defined as: ； wherein, is the i-th segment trajectory acceleration vector, is the corresponding acceleration control point. 2.The method of claim 1, wherein: In the step S3, the collision penalty term The local collision cost item is constructed by controlling the geometric relationship between the control point and the surface of the obstacle, and the global collision cost function is accumulated accordingly to realize the gradient modeling and optimization guidance of the collision risk. 3.The method of claim 2, wherein: In the step S3, the kinetic feasibility penalty term According to the speed, acceleration, jerk limit design: ； ； wherein subscript i denotes the number of trajectory segments, v, a, j denote velocity, acceleration, jerk, respectively, , , are each a weight coefficient, is a piecewise quadratic penalty function, is the maximum value allowed for each physical quantity, denotes the number of control points. 4.The method of claim 1, wherein: In the step S4, the window-specific trajectory constraints include: a) the crossing point must be inside the window, at a safe distance from the window frame edges; b) the angle between the velocity direction during crossing and the normal to the window plane is less than a preset threshold; c) the crossing velocity is greater than a minimum threshold , ensuring a fast passage. 5.The method of claim 4, wherein: In the step S5, the reward function is designed as: ； wherein, , , is a weight coefficient, is a window crossing reward, is an efficiency reward; in the proposed reward function R, first, three types of trajectory costs , and are used to form the penalty term, multiplied by the normalized weights , and , respectively, where each weight takes a value between 0-1 and satisfies + + =1, which is kept constant at the beginning of training, allowing the policy to adaptively fine-tune during the learning process; then, two positive incentives are added: one is the window crossing reward , and the other is the efficiency reward ; thus, the policy is comprehensively prompted to pursue faster and more energy-efficient window crossing while ensuring safety and dynamic constraints. 6.The method of claim 5, wherein: The window traversing reward is defined as: ； wherein is a progress of crossing, , , . 7.The method of claim 6, wherein: Efficiency reward Defined as: ； where T is the traversal time, >0 is an exponential adjustment factor, >0 is an efficiency weight coefficient. 8.The method of claim 1, wherein: In the step S6, when the initial trajectory does not satisfy the dynamic constraints, trajectory time redistribution is performed, and the over-limit ratio is calculated: ； wherein, is the maximum allowed speed for the segment trajectory, is the speed of the segment trajectory, is the maximum allowed speed; is the maximum allowed acceleration for the segment trajectory, is the acceleration of the segment trajectory, is the maximum allowed acceleration; is the maximum allowed jerk for the segment trajectory, is the jerk of the segment trajectory, is the maximum allowed jerk. The new time interval is set to: The trajectory is then optimized using an anisotropic curve fitting method: ； wherein is a normalized time variable, and are axial and radial displacements, respectively, and are the semi-major and semi-minor axes of the ellipse.

9. A trajectory planning system for a UAV to quickly cross a window, comprising an environment perception module, a reinforcement learning agent, and an EGOPlanner optimization module, characterized in that: The environment perception module is responsible for obtaining window boundary information and surrounding obstacle distribution data in the current scene; the module relies on a binocular camera for visual perception, and performs depth estimation through a real-time captured binocular image pair, thereby obtaining a dense depth map and a three-dimensional space point cloud; In terms of window recognition, a convolutional neural network based on YOLOv5 is used to detect window targets in the left eye image, the corresponding spatial position is calculated through binocular matching, and the geometric boundary and normal direction of the window are further estimated through image plane fitting and edge extraction algorithm, thereby realizing accurate modeling of the window center point, width and height size, and attitude angle; In terms of obstacle mapping, the system constructs a local sparse point cloud set from the binocular depth map, or projects it into a two-dimensional grid map, which is used to represent the obstacle distribution in the flight space, and supports subsequent trajectory feasibility evaluation and obstacle avoidance gradient construction; The reinforcement learning agent module: according to the perception result, the current state is constructed , including the relative position, speed, orientation, obstacle density distribution elements between the unmanned aerial vehicle and the window, as the input of the reinforcement learning agent; the proximal policy optimization algorithm PPO is used for policy learning, and in the training stage, a reward function containing the comprehensive indexes of the penetration success rate, the trajectory smoothness and the collision penalty is constructed, so that the parameter adjustment law of the policy network is learned; the network structure adopts a three-layer fully connected network, each layer contains a plurality of neurons, the activation function adopts ReLU, and the training adopts the parallel sampling and the experience replay mechanism to accelerate the convergence; In the execution phase, the reinforcement learning agent outputs an action vector based on the current state , for adjusting the spacing between control points, time allocation and the weights of each penalty term in the cost function in the trajectory optimization process, wherein, represents the adjustment amount of the spline control point, represents the trajectory time adjustment coefficient, , , , respectively represent the weights of smoothness, collision avoidance and dynamics feasibility in EGOPlanner. EGO-Planner optimization module: After receiving the parameter configuration, a preliminary trajectory is generated. Based on the B-spline trajectory modeling method, the trajectory smoothness, collision distance, and dynamics feasibility are taken as the cost function to construct the following optimization problem: , wherein is a set of trajectory control points, 、 、 respectively denote the smoothness, collision penalty and dynamics feasibility cost terms, 、 、 respectively denote the weights of smoothness, collision avoidance and dynamics feasibility in EGOPlanner. In the trajectory optimization process, if a conflict with the dynamic constraints is detected, the system will automatically trigger the trajectory time redistribution and anisotropic curve fitting module, and finally output a trajectory that satisfies the crossing constraints for control execution.

10. A non-volatile storage medium, comprising: The non-volatile storage medium comprises a stored program, wherein the program controls the device in which the non-volatile storage medium is located to execute the method of any one of claims 1 to 8 when the program is run.

11. A terminal device, comprising: The terminal device comprises a processor, a memory, a communication interface, and a bus; the processor, the memory, and the communication interface are connected through the bus and complete communication with each other; the memory stores executable program code; the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, to execute the method of any one of claims 1-8.

Images