Unmanned aerial vehicle autonomous obstacle avoidance and path planning method and system based on deep learning

Abstract

The application provides a kind of unmanned aerial vehicle autonomous obstacle avoidance and path planning method and system based on deep learning, it is related to unmanned aerial vehicle control field, including obtaining position information and environmental perception data, constructs space-time feature matrix, extracts target motion and background feature vector, and maps into target-environment fusion feature field;Calculate the reachability matrix and cost matrix to build the track search space, generate candidate track set and determine the optimal planning track;Segment optimization is carried out to the planning track to obtain continuous attitude sequence, and an adaptive control strategy is generated.The application realizes intelligent obstacle avoidance and efficient path planning of unmanned aerial vehicle in complex environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle control, in particular to an unmanned aerial vehicle autonomous obstacle avoidance and path planning method and system based on deep learning. BACKGROUND

[0002] With the rapid development of unmanned aerial vehicle technology, its application in complex environments such as inspection, logistics, and rescue is becoming increasingly widespread. Traditional path planning methods rely heavily on static mapping and heuristic search algorithms, making it difficult to deal with dynamic obstacles and complex terrain conditions. In particular, in urban low-altitude flight or forest crossing scenarios, there are problems of obstacle avoidance lag and discontinuous path, which restrict the safety and stability of unmanned aerial vehicle autonomous flight. In recent years, the advantages of deep learning in image recognition and spatio-temporal modeling are significant, providing a new approach to unmanned aerial vehicle environmental perception and path planning. However, existing methods based on deep learning focus on a single task of image recognition or trajectory prediction, lack joint modeling of dynamic targets and background environment, and have not yet formed an integrated solution framework for autonomous obstacle avoidance and global path planning, resulting in generated flight paths lacking real-time and continuity.

[0003] In addition, existing methods generally ignore the cooperative matching of planned flight paths and unmanned aerial vehicle motion control systems, and problems such as sharp changes in trajectory curvature and slow response of thrust allocation seriously affect trajectory tracking accuracy. Therefore, there is a need for an unmanned aerial vehicle autonomous obstacle avoidance and path planning method that can integrate target and environmental features, have global path planning capabilities, and support dynamic attitude control. SUMMARY

[0004] The embodiments of the present application provide an unmanned aerial vehicle autonomous obstacle avoidance and path planning method and system based on deep learning, which can solve the problems in the prior art.

[0005] In a first aspect, the embodiments of the present application provide an unmanned aerial vehicle autonomous obstacle avoidance and path planning method based on deep learning, comprising:

[0006] Obtaining position information and environmental perception data of the unmanned aerial vehicle;

[0007] Constructing a spatio-temporal feature matrix from the environmental perception data, extracting a target motion feature vector and a background feature vector in the spatio-temporal feature matrix, calculating a motion trajectory projection based on the target motion feature vector, constructing an environmental constraint boundary based on the background feature vector, and mapping the motion trajectory projection and the environmental constraint boundary to a three-dimensional grid space to obtain a target-environment fusion feature field;

[0008] The reachability matrix of grid points is calculated based on the target-environment fusion feature field. The state transition cost between adjacent grid points is calculated to generate the cost matrix. The reachability matrix and the cost matrix are combined to construct the trajectory search space. A set of candidate trajectories that meet the motion constraints is generated in the trajectory search space. The optimal planned trajectory is determined by calculating the cumulative cost of each trajectory in the candidate trajectory set.

[0009] The flight path is divided into multiple segments based on the curvature change of the optimal planned trajectory. Online trajectory optimization is performed on the flight segments to obtain a continuous attitude sequence. Based on the continuous attitude sequence and environmental disturbance compensation, an adaptive thrust allocation strategy and a real-time attitude control strategy are generated and combined into segmented execution commands. The UAV is controlled to complete the trajectory tracking of the flight segments according to the segmented execution commands.

[0010] In one alternative embodiment,

[0011] A spatiotemporal feature matrix is ​​constructed from the environmental perception data. Target motion feature vectors and background feature vectors are extracted from this matrix, including:

[0012] Multiple sets of environmental data are sampled at preset time intervals for environmental perception data. The environmental data are arranged in the time dimension to form a data stream. The data stream is subjected to temporal filtering and amplitude normalization to obtain a time-series data sequence. Multi-scale convolution kernels are used to extract features from the time-series data sequence to obtain a multi-scale feature map. The multi-scale feature maps are fused to obtain a spatiotemporal feature matrix.

[0013] The data differences between adjacent time points are calculated in the spatiotemporal feature matrix and threshold segmentation is performed to obtain the moving target region and the background region;

[0014] The cross-correlation coefficient is calculated for data from adjacent time points in the moving target region to obtain a correlation coefficient matrix. The position offset corresponding to the maximum correlation is extracted from the correlation coefficient matrix to obtain the motion direction vector. Morphological processing is performed on the moving target region to extract boundary coordinates to obtain the target contour. The motion feature vector is obtained based on the centroid displacement of the target contour along the motion direction and the motion direction vector.

[0015] Feature points are extracted from the background region to calculate disparity and obtain a depth map. Bilateral filtering is applied to the depth map to obtain depth information. Edge detection is performed on the background region to obtain an edge point set. Edge information is obtained by connecting the edge points using a double threshold method. The depth information and edge information are spatially aligned and combined to obtain a background feature vector.

[0016] In one alternative embodiment,

[0017] The motion trajectory projection is calculated based on the target motion feature vector, and the environmental constraint boundary is constructed based on the background feature vector. The motion trajectory projection and the environmental constraint boundary are mapped onto the three-dimensional mesh space to obtain the target-environment fusion feature field, which includes:

[0018] Based on the target motion feature vector, a state equation is established to calculate the target position and velocity changes. Based on the target position and velocity changes, a recursive iterative method is used to predict the target's future position sequence to obtain the target motion trajectory. The trajectory projection is obtained by projecting it onto the observation space through coordinate transformation.

[0019] Point cloud data is constructed using depth information in background feature vectors, and the point cloud data is segmented and clustered to obtain the three-dimensional spatial structure of the environment. Obstacle contours are extracted from the three-dimensional spatial structure of the environment to obtain obstacle boundaries, and passable areas are calculated to obtain passable boundaries. The obstacle boundaries and passable boundaries are combined to obtain environmental constraint boundaries.

[0020] The motion trajectory projection is divided into grid cells according to a preset resolution and mapped to a three-dimensional grid space to obtain trajectory grid features. At the same time, the environmental constraint boundary is divided into grid cells according to a preset resolution and mapped to a three-dimensional grid space to obtain boundary grid features.

[0021] The motion continuity score is obtained by calculating the state consistency between adjacent grid cells of the trajectory grid features. The structural integrity score is obtained by calculating the spatial coverage and boundary integrity of the boundary grid features. The motion continuity score and structural integrity score are normalized to obtain the feature fusion weight. The trajectory grid features and boundary grid features are adaptively weighted and superimposed according to the feature fusion weight to obtain the target-environment fusion feature field.

[0022] In one alternative embodiment,

[0023] The reachability matrix of grid points is calculated based on the target-environment fusion feature field. The state transition cost between adjacent grid points is calculated to generate the cost matrix. The reachability matrix and the cost matrix are combined to construct the track search space, including:

[0024] Calculate the eigenvalue gradient of each grid point in the target-environment fusion feature field, decompose the eigenvalue gradient in three-dimensional space to obtain gradient components, determine the gradient principal direction based on the gradient components, calculate the gradient projection value of the grid point according to the gradient principal direction, divide adjacent grid points into multi-level gradient descent domains based on the gradient projection value, and select adjacent points with eigenvalues ​​less than the current grid point as reachable candidate points in the multi-level gradient descent domains.

[0025] The detection step size is determined based on the rate of change of the feature values ​​of the region where the line connecting the current grid point and the reachable candidate point is located. The detection step size is used to detect the line segment by segment to determine whether it intersects with the obstacle boundary. The continuity of the feature values ​​on the line is checked. Reachable candidate points that do not intersect with obstacles and whose feature values ​​change continuously are determined as reachable points. A reachability matrix is ​​established based on the reachability distribution of each grid point.

[0026] The reachability is obtained by calculating the proportion of reachable points of each grid point in the reachability matrix to the total number of adjacent points. The reachability is then graded and weighted according to the level of the gradient descent domain in which the grid point is located to obtain the weight coefficient. The weight coefficient is combined with the distance between grid points, the change in heading, and the change rate of the feature field to obtain the state transition cost, and a state transition cost matrix is ​​constructed.

[0027] The reachability matrix and the state transition cost matrix are combined to construct the track search space.

[0028] In one alternative embodiment,

[0029] Within the trajectory search space, a set of candidate trajectories satisfying motion constraints is generated. The optimal planned trajectory is determined by calculating the cumulative cost of each trajectory in the candidate trajectory set, including:

[0030] Within the trajectory search space, the weight coefficients of the grid points are normalized to obtain the basic transition probability. Based on the basic transition probability and the gradient principal direction, the state transition probability distribution is calculated, and an initial set of sampling nodes is generated in each gradient descent domain.

[0031] The node search radius is determined based on the level of the gradient descent domain where the sampling node is located. A nearest neighbor search is performed within the search radius to obtain a set of connectable nodes. Based on the reachability matrix, it is determined whether the connectable nodes meet the motion constraints. Nodes that meet the motion constraints are connected and given a state transition cost to construct a candidate trajectory search tree.

[0032] Motion constraint verification is performed on the paths in the candidate track search tree, and path nodes that satisfy the steering angle constraint and steering angle acceleration constraint are extracted as control point sequences. The control point sequences are then smoothed to generate a candidate track set.

[0033] Based on the state transition cost and gradient descent domain level, hierarchical cost weights are set, the cumulative cost of each track in the candidate track set is calculated, the track with the minimum cumulative cost is selected as the current optimal track, local sampling is performed with the control point of the optimal track as the center to obtain a new control point sequence, the new control point sequence is smoothed to generate a new candidate track, the cumulative cost of the new candidate track is calculated and the optimal track is updated, and local optimization is iteratively performed until the cumulative cost converges to obtain the optimal planned track.

[0034] In one alternative embodiment,

[0035] The flight path is divided into multiple segments based on the curvature change of the optimal planned trajectory. Online trajectory optimization is then performed on these segments to obtain a continuous attitude sequence, including:

[0036] Obtain the curvature change value of adjacent trajectory points on the optimal planned trajectory, determine the trajectory points with curvature greater than the preset curvature threshold as curvature change points, and divide the optimal planned trajectory into multiple flight segments with the curvature change points as the boundary;

[0037] A fixed-length sliding time window is established for each flight segment. The initial position of the sliding time window is set at the beginning of the flight segment. Within the sliding time window, the expected overload command range and expected roll rate range of the control point are determined based on the curvature gradient sequence of the current position, and multiple sets of candidate control input sequences are generated.

[0038] From multiple candidate control input sequences, the control input sequence that satisfies flight dynamics constraints and has the minimum curvature change is selected as the optimal control input, and a continuous attitude sequence for the current time window is generated based on the optimal control input.

[0039] The final state of the continuous attitude sequence is used as the initial state of the next sliding time window. The sliding time window is slid forward and the optimization process is repeated until the trajectory optimization of the current flight segment to be optimized is completed. After the trajectory optimization of all flight segments is completed, the optimized continuous attitude sequences of each flight segment are combined to obtain the final continuous attitude sequence.

[0040] In one alternative embodiment,

[0041] An adaptive thrust allocation strategy and a real-time attitude control strategy are generated based on continuous attitude sequences and environmental disturbance compensation, and combined into segmented execution commands. The UAV is then controlled to complete trajectory tracking for each flight segment according to these segmented execution commands, including:

[0042] The desired attitude command and desired trajectory information are calculated based on a continuous attitude sequence. The current flight state and environmental disturbance information of the UAV are collected, and the attitude deviation value is obtained by comparing the current flight state with the desired attitude command.

[0043] The thrust compensation is calculated based on the attitude deviation value, and the thrust compensation is adaptively adjusted in combination with environmental disturbance information to generate an adaptive thrust allocation strategy. A real-time attitude control strategy is generated based on the desired attitude command and attitude deviation value.

[0044] Based on the feature points of the desired trajectory information, the flight segment is divided into multiple continuous control intervals, and the adaptive thrust allocation strategy and real-time attitude control strategy are combined to form segmented execution commands.

[0045] Configure corresponding control parameters in each control interval, and use a smooth transition function at the boundary point of adjacent control intervals to achieve continuous switching of the segmented execution commands. Control the UAV to complete the trajectory tracking of the flight segment according to the switched segmented execution commands.

[0046] A second aspect of this invention provides a deep learning-based autonomous obstacle avoidance and path planning system for unmanned aerial vehicles (UAVs), comprising:

[0047] The first unit is used to acquire the drone's location information and environmental perception data;

[0048] The second unit is used to construct a spatiotemporal feature matrix from environmental perception data, extract target motion feature vectors and background feature vectors from the spatiotemporal feature matrix, calculate motion trajectory projection based on target motion feature vectors, construct environmental constraint boundaries based on background feature vectors, and map motion trajectory projections and environmental constraint boundaries onto a three-dimensional mesh space to obtain a target-environment fusion feature field.

[0049] The third unit is used to calculate the reachability matrix of grid points based on the target-environment fusion feature field, calculate the state transition cost between adjacent grid points to generate the cost matrix, combine the reachability matrix and the cost matrix to construct the trajectory search space, generate a set of candidate trajectories that meet the motion constraints in the trajectory search space, and determine the optimal planned trajectory by calculating the cumulative cost of each trajectory in the candidate trajectory set.

[0050] The fourth unit is used to divide the flight path into multiple flight segments based on the curvature change of the optimal planned trajectory, perform online trajectory optimization on the flight segments to obtain a continuous attitude sequence, generate an adaptive thrust allocation strategy and a real-time attitude control strategy based on the continuous attitude sequence and environmental disturbance compensation, and combine them into segmented execution commands. The UAV is then controlled to complete the trajectory tracking of the flight segments according to the segmented execution commands.

[0051] A third aspect of the present invention provides an electronic device, comprising:

[0052] processor;

[0053] Memory used to store processor-executable instructions;

[0054] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0055] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0056] In this embodiment, target motion features and background features are extracted by constructing a spatiotemporal feature matrix. The motion trajectory projection and environmental constraint boundary are mapped onto a three-dimensional mesh space to form a fused feature field. This enables the UAV to accurately perceive and predict the trajectory of moving obstacles in dynamic environments, improving the accuracy and real-time performance of obstacle avoidance decisions. Based on the fused feature field, the reachability of mesh points and state transition costs are calculated. A trajectory search space is constructed, and a set of candidate trajectories that satisfy motion constraints is generated. The optimal planned trajectory is determined through cumulative cost evaluation, enabling the UAV to adaptively plan a safe and efficient flight path in complex environments, avoiding the local optima and path non-smoothness problems in traditional methods. The optimal planned trajectory is divided into multiple flight segments and online trajectory optimization is performed. Combined with environmental disturbance compensation, adaptive thrust allocation and real-time attitude control strategies are generated, achieving accurate trajectory tracking of the UAV in uncertain environments. This improves the stability and robustness during flight, while reducing energy consumption and extending endurance. Attached Figure Description

[0057] Figure 1 This is a flowchart illustrating the deep learning-based autonomous obstacle avoidance and path planning method for unmanned aerial vehicles (UAVs) according to an embodiment of the present invention.

[0058] Figure 2 This is a flowchart of the drone trajectory planning process according to an embodiment of the present invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0061] Figure 1 This is a flowchart illustrating the deep learning-based autonomous obstacle avoidance and path planning method for unmanned aerial vehicles (UAVs) according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0062] Acquire the drone's location information and environmental perception data;

[0063] A spatiotemporal feature matrix is ​​constructed from environmental perception data. Target motion feature vectors and background feature vectors are extracted from the spatiotemporal feature matrix. The motion trajectory projection is calculated based on the target motion feature vectors. An environmental constraint boundary is constructed based on the background feature vectors. The motion trajectory projection and the environmental constraint boundary are mapped onto a three-dimensional mesh space to obtain a target-environment fusion feature field.

[0064] The reachability matrix of grid points is calculated based on the target-environment fusion feature field. The state transition cost between adjacent grid points is calculated to generate the cost matrix. The reachability matrix and the cost matrix are combined to construct the trajectory search space. A set of candidate trajectories that meet the motion constraints is generated in the trajectory search space. The optimal planned trajectory is determined by calculating the cumulative cost of each trajectory in the candidate trajectory set.

[0065] The flight path is divided into multiple segments based on the curvature change of the optimal planned trajectory. Online trajectory optimization is performed on the flight segments to obtain a continuous attitude sequence. Based on the continuous attitude sequence and environmental disturbance compensation, an adaptive thrust allocation strategy and a real-time attitude control strategy are generated and combined into segmented execution commands. The UAV is controlled to complete the trajectory tracking of the flight segments according to the segmented execution commands.

[0066] In one optional implementation, a spatiotemporal feature matrix is ​​constructed from the environmental perception data, and the target motion feature vector and background feature vector are extracted from the spatiotemporal feature matrix, including:

[0067] Multiple sets of environmental data are sampled at preset time intervals for environmental perception data. The environmental data are arranged in the time dimension to form a data stream. The data stream is subjected to temporal filtering and amplitude normalization to obtain a time-series data sequence. Multi-scale convolution kernels are used to extract features from the time-series data sequence to obtain a multi-scale feature map. The multi-scale feature maps are fused to obtain a spatiotemporal feature matrix.

[0068] The data differences between adjacent time points are calculated in the spatiotemporal feature matrix and threshold segmentation is performed to obtain the moving target region and the background region;

[0069] The cross-correlation coefficient is calculated for data from adjacent time points in the moving target region to obtain a correlation coefficient matrix. The position offset corresponding to the maximum correlation is extracted from the correlation coefficient matrix to obtain the motion direction vector. Morphological processing is performed on the moving target region to extract boundary coordinates to obtain the target contour. The motion feature vector is obtained based on the centroid displacement of the target contour along the motion direction and the motion direction vector.

[0070] Feature points are extracted from the background region to calculate disparity and obtain a depth map. Bilateral filtering is applied to the depth map to obtain depth information. Edge detection is performed on the background region to obtain an edge point set. Edge information is obtained by connecting the edge points using a double threshold method. The depth information and edge information are spatially aligned and combined to obtain a background feature vector.

[0071] In a specific implementation, the environmental perception data can come from raw data collected by various sensing devices such as radar, lidar, and visual sensors. First, the environmental perception data is sampled at preset time intervals of 10 milliseconds to obtain multiple sets of environmental data. For example, in one implementation, environmental data is collected every 10 milliseconds for 5 seconds, resulting in 500 sets of environmental data samples. These environmental data are arranged in the time dimension to form a continuous data stream. A bandpass filter is applied to this data stream for time-domain filtering, with the low cutoff frequency set to 0.5Hz and the high cutoff frequency set to 20Hz to filter out environmental noise and irrelevant signal components. Next, the filtered data undergoes amplitude normalization processing, scaling the data range to the [-1, 1] interval to obtain a time-series data sequence.

[0072] For normalized time-series data sequences, multi-scale convolutional kernels are used for feature extraction. Specifically, three different kernel sizes are designed: 3×3, 5×5, and 7×7. Smaller kernels capture fine local features, while larger kernels capture broader contextual features. Each kernel has 16 channels, a stride of 1, and zero-padding at the edges to maintain consistent feature map size. Convolution operations are performed on the time-series data sequences using these three different kernel sizes, resulting in three feature maps. These three feature maps are then fused using channel concatenation and dimensionality reduction via 1×1 convolutions, ultimately yielding a spatiotemporal feature matrix of dimension M×N×K, where M and N represent the spatial dimensions of the feature maps, and K represents the feature dimension, which can be set to 128×128×64 in practical applications.

[0073] In the constructed spatiotemporal feature matrix, the data differences between adjacent time points are calculated to identify moving target regions and background regions. First, the Euclidean distance between two adjacent frames in the spatiotemporal feature matrix is ​​calculated, yielding the difference matrix D. In the difference matrix, larger values ​​indicate more significant changes at that location. An adaptive thresholding method is applied to the difference matrix, with the threshold T set to the mean of the difference matrix plus twice the standard deviation. Regions with difference values ​​greater than the threshold T are marked as moving target regions, and regions with difference values ​​less than or equal to the threshold T are marked as background regions. In one example, the mean of the difference matrix is ​​0.15, and the standard deviation is 0.08; therefore, the threshold T is calculated to be 0.31.

[0074] For the identified moving target region, cross-correlation coefficients are calculated for data from adjacent time points to construct a correlation coefficient matrix R. Within a 64×64 pixel moving target region, a sliding window method is used to calculate the cross-correlation coefficients between the target region at time t and time t+1. The window size is set to 16×16 pixels with a step size of 4 pixels. From the calculated correlation coefficient matrix, the positional offset (Δx, Δy) corresponding to the maximum correlation is identified; this offset represents the target's direction of motion vector. For example, if the maximum correlation coefficient occurs at offset (8, 12), it indicates that the target has moved 8 units in the x-direction and 12 units in the y-direction.

[0075] Morphological processing is applied to the moving target region to extract the target contour. First, the target region is binarized, then an opening operation (erosion followed by dilation) is applied to remove noise, using a 3×3 rectangular kernel as the structuring element. Next, a closing operation (dilation followed by erosion) is applied to fill the internal voids of the target, using a 5×5 rectangular kernel as the structuring element. After morphological processing, a contour tracking algorithm is used to extract the target boundary coordinate set, obtaining the target contour. The centroid coordinates (cx, cy) of the target contour are calculated, and combined with the previously obtained motion direction vector (Δx, Δy), a motion feature vector is constructed, represented as (cx, cy, Δx, Δy, v), where v is the displacement magnitude, calculated as (Δx...). 2 +Δy 2 ) 1 / 2 .

[0076] For the background region, feature points are extracted and disparity is calculated to obtain a depth map. A corner detection algorithm is used to identify feature points in the background region, with a detection threshold set to 0.01 and a maximum number of feature points limited to 1000. For stereo vision data, disparity is calculated by matching feature points in the left and right images, with the disparity search range set to 0 to 64 pixels. Based on the camera calibration parameters and the calculated disparity values, depth values ​​are converted to generate the original depth map. To improve the quality of the depth map, a bilateral filter is applied, with a spatial kernel size of 5×5, a spatial standard deviation of 3.0, and an intensity standard deviation of 0.1. Bilateral filtering can smooth the depth map while preserving edge information and reducing the impact of noise.

[0077] Edge detection is performed on the background region to supplement environmental structure information. First, the gradient magnitude and direction of the background region are calculated, with a gradient threshold set to 1.5 times the average gradient of the background region. Edge points are then connected using a dual-threshold method: a high threshold of 0.2 and a low threshold of 0.1. Points above the high threshold are directly marked as edge points, while points below the low threshold are discarded. Points in between are marked as edge points only if they are adjacent to already identified edge points. This method yields a continuous set of edge points, constituting the edge information.

[0078] Finally, the depth and edge information are combined using spatial alignment to obtain the background feature vector. Spatial alignment is achieved through pixel-level mapping, ensuring a one-to-one correspondence between the depth map and the edge map in spatial location. The combined background feature vector is represented as (x, y, d, e), where (x, y) are pixel coordinates, d is the depth value, and e is the edge intensity. For a 128×128 pixel background region in practical applications, a background feature description containing 16,384 four-dimensional feature vectors was constructed.

[0079] In this embodiment, the spatiotemporal feature matrix extracted through multi-scale convolution effectively captures temporal changes and local features in environmental perception data, improving the accuracy of moving target recognition. The motion feature vector constructed through cross-correlation and morphological processing accurately depicts the target's motion direction and contour shape, enhancing the robustness of path prediction. Depth information of the background region is extracted through disparity calculation and bilateral filtering, combined with edge detection to extract structural information, ensuring that the background feature vector possesses complete geometric constraints and boundary characteristics. This helps construct an environmental model in a real-world scene, providing accurate input for subsequent path planning and obstacle avoidance strategies.

[0080] In one optional implementation, the motion trajectory projection is calculated based on the target motion feature vector, an environmental constraint boundary is constructed based on the background feature vector, and the motion trajectory projection and the environmental constraint boundary are mapped onto a three-dimensional mesh space to obtain a target-environment fusion feature field, including:

[0081] Based on the target motion feature vector, a state equation is established to calculate the target position and velocity changes. Based on the target position and velocity changes, a recursive iterative method is used to predict the target's future position sequence to obtain the target motion trajectory. The trajectory projection is obtained by projecting it onto the observation space through coordinate transformation.

[0082] Point cloud data is constructed using depth information in background feature vectors, and the point cloud data is segmented and clustered to obtain the three-dimensional spatial structure of the environment. Obstacle contours are extracted from the three-dimensional spatial structure of the environment to obtain obstacle boundaries, and passable areas are calculated to obtain passable boundaries. The obstacle boundaries and passable boundaries are combined to obtain environmental constraint boundaries.

[0083] The motion trajectory projection is divided into grid cells according to a preset resolution and mapped to a three-dimensional grid space to obtain trajectory grid features. At the same time, the environmental constraint boundary is divided into grid cells according to a preset resolution and mapped to a three-dimensional grid space to obtain boundary grid features.

[0084] The motion continuity score is obtained by calculating the state consistency between adjacent grid cells of the trajectory grid features. The structural integrity score is obtained by calculating the spatial coverage and boundary integrity of the boundary grid features. The motion continuity score and structural integrity score are normalized to obtain the feature fusion weight. The trajectory grid features and boundary grid features are adaptively weighted and superimposed according to the feature fusion weight to obtain the target-environment fusion feature field.

[0085] For example, after the system acquires the target's motion feature vector, it establishes a state equation describing the target's motion state. This state equation includes the target's position, velocity, and their changing relationships. Assume the target's current position is (10, 15, 0) m, its velocity is (2, 1, 0) m / s, and its acceleration is (0.1, 0.2, 0) m / s². 2 A linear recursive approach is used, with a time step of 0.1 seconds, to calculate the target position sequence within the next 5 seconds. Specifically, for each time point t+1, the position update formula is: current position plus velocity multiplied by the time step, plus acceleration multiplied by half the square of the time step; the velocity update formula is: current velocity plus acceleration multiplied by the time step. After 50 iterations, the future trajectory of the target containing 51 points is obtained, for example, the position of the 10th point is (12.5, 16.7, 0) meters. Subsequently, this trajectory is transformed from the target coordinate system to the observation coordinate system, assuming the transformation matrix parameters are a rotation angle of 30 degrees and a translation vector of (5, 8, 0) meters, and the trajectory projection in the observation space is obtained through coordinate transformation.

[0086] The background feature vector contains depth information with a resolution of 640×480 pixels. Point cloud data is generated using these depth values, with each pixel corresponding to a 3D spatial point, totaling 307,200 points. This point cloud data is segmented using a region growing algorithm based on Euclidean distance, with a clustering distance threshold set to 0.05 meters and a minimum cluster size of 100 points. After segmentation and clustering, the main structures in the environment, such as the ground, walls, and obstacles, are obtained. The ground point cloud contains approximately 150,000 points, located in a planar region with a z-coordinate close to 0; the wall point cloud contains approximately 80,000 points, mainly distributed in the xz and yz planes; the obstacle point cloud contains several independent clusters, such as tables (approximately 5,000 points) and chairs (approximately 3,000 points). Obstacle boundaries are extracted from the point cloud clusters by calculating the convex hull or bounding box of each cluster. For example, the bounding box of the table is a cuboid ranging from (2, 3, 0) to (3, 4, 0.8) meters. The passable area, or passable boundary, is calculated by complementing the obstacle boundaries. The obstacle boundaries are marked as prohibited areas, and the passable boundaries are marked as permitted areas, forming the environmental constraint boundaries.

[0087] The motion trajectory projection is divided into grid cells at a preset resolution of 0.1 meters and mapped onto a 100×100×10 three-dimensional grid space. For each point on the trajectory, its corresponding grid index is calculated; for example, the position (12.5, 16.7, 0) corresponds to the grid index (125, 167, 0). To represent trajectory continuity, the grid cells traversed by the trajectory are assigned a value of 1, and the rest are assigned a value of 0, forming the trajectory grid feature. Similarly, the environmental constraint boundaries are divided into grid cells at the same resolution and mapped onto a 3D grid space of the same size. Obstacle boundary grid cells are assigned a value of -1, and passage boundary grid cells are assigned a value of 0.5, forming the boundary grid feature.

[0088] The state consistency of adjacent grid cells in the trajectory grid features is calculated to evaluate motion continuity. For each trajectory grid cell, its 26 neighboring grids are checked; if an adjacent grid is also part of the trajectory, the continuity count is increased. The final total continuity count for the trajectory is 47 (for 50 trajectory segments), and the continuity ratio is 47 / 50 = 0.94, which is the motion continuity score. Simultaneously, the spatial coverage of the boundary grid features is calculated, which is the ratio of the number of non-zero grid cells to the total number of grid cells, for example, 35000 / 100000 = 0.35; boundary integrity is calculated, i.e., the degree of boundary closure, assumed to be 0.88. The average of the spatial coverage and boundary integrity, 0.615, is used as the structural integrity score.

[0089] The motion continuity score and structural integrity score were normalized using Min-Max, resulting in a normalized motion continuity score of 0.9 and a structural integrity score of 0.8. Based on these scores, feature fusion weights were calculated: the trajectory grid feature weight was 0.9 / (0.9+0.8)=0.529, and the boundary grid feature weight was 0.8 / (0.9+0.8)=0.471. Finally, the trajectory grid features and boundary grid features were weighted and superimposed according to these weights. For example, for the grid index (125, 167, 0), the fused feature value was 0.529×1+0.471×0.5=0.765. Similar calculations were performed on all grid cells, ultimately yielding a fused feature field representing the target's motion trajectory and environmental constraints. This feature field reflects both the target's possible future motion path and information about obstacles and traversable areas in the environment.

[0090] In this embodiment, the target trajectory constructed through state equations and recursive prediction can dynamically reflect the future movement trend of the target and be projected onto the observation space to enhance the spatial correlation of the trajectory. Simultaneously, the environmental constraint boundary constructed using deep point clouds can accurately characterize obstacle morphology and passable areas, providing realistic and effective spatial constraints for path planning. Mapping the motion trajectory and environmental boundary to a 3D mesh and calculating feature fusion weights achieves a balanced assessment of motion continuity and environmental structural integrity. Constructing a fused feature field through adaptive weighting effectively enhances the system's collaborative understanding of dynamic target behavior and static environmental constraints, providing accurate and structured prior support for the subsequent generation of feasible paths.

[0091] In one optional implementation, the reachability matrix of grid points is calculated based on the target-environment fusion feature field, the state transition cost between adjacent grid points is calculated to generate a cost matrix, and the reachability matrix and cost matrix are combined to construct the track search space, including:

[0092] Calculate the eigenvalue gradient of each grid point in the target-environment fusion feature field, decompose the eigenvalue gradient in three-dimensional space to obtain gradient components, determine the gradient principal direction based on the gradient components, calculate the gradient projection value of the grid point according to the gradient principal direction, divide adjacent grid points into multi-level gradient descent domains based on the gradient projection value, and select adjacent points with eigenvalues ​​less than the current grid point as reachable candidate points in the multi-level gradient descent domains.

[0093] The detection step size is determined based on the rate of change of the feature values ​​of the region where the line connecting the current grid point and the reachable candidate point is located. The detection step size is used to detect the line segment by segment to determine whether it intersects with the obstacle boundary. The continuity of the feature values ​​on the line is checked. Reachable candidate points that do not intersect with obstacles and whose feature values ​​change continuously are determined as reachable points. A reachability matrix is ​​established based on the reachability distribution of each grid point.

[0094] The reachability is obtained by calculating the proportion of reachable points of each grid point in the reachability matrix to the total number of adjacent points. The reachability is then graded and weighted according to the level of the gradient descent domain in which the grid point is located to obtain the weight coefficient. The weight coefficient is combined with the distance between grid points, the change in heading, and the change rate of the feature field to obtain the state transition cost, and a state transition cost matrix is ​​constructed.

[0095] The reachability matrix and the state transition cost matrix are combined to construct the track search space.

[0096] In this embodiment, the method for constructing the trajectory search space first calculates the eigenvalue gradient of each grid point in the target-environment fusion feature field. The feature field is a three-dimensional spatial representation composed of obstacles and targets, where each grid point contains eigenvalue information. By calculating the difference between the eigenvalues ​​of the current grid point and its neighboring points, the rate of change of the eigenvalues ​​in the x, y, and z directions is determined, forming the eigenvalue gradient vector. For example, for the grid point with coordinates (10, 15, 20), its eigenvalue is 0.75. By comparing it with the eigenvalues ​​of its neighboring points, the gradient vector is calculated to be (0.05, 0.12, -0.03).

[0097] The eigenvalue gradient is decomposed in three-dimensional space to obtain gradient components. For the gradient vector above, the x-direction component is 0.05, the y-direction component is 0.12, and the z-direction component is -0.03. Based on these components, the principal gradient direction is determined, that is, the direction with the largest absolute value of the gradient component. In this example, the y-direction is the principal gradient direction, and its component value is 0.12.

[0098] Calculate the gradient projection value of the grid points based on the principal gradient direction. Project the position vectors of each adjacent point relative to the current point onto the gradient direction to obtain the projection value. For the adjacent point located at (11, 15, 20), its position vector is (1, 0, 0), and the value projected onto the gradient direction is 0.05×1+0.12×0+(-0.03)×0=0.05.

[0099] Based on the gradient projection values, adjacent grid points are divided into multi-level gradient descent regions. According to the magnitude of the projection values, adjacent points are classified into strong gradient descent regions, medium gradient descent regions, and weak gradient descent regions. Points with projection values ​​less than -0.1 are assigned to the strong gradient descent region, points with projection values ​​between -0.1 and -0.05 are assigned to the medium gradient descent region, and points with projection values ​​between -0.05 and 0 are assigned to the weak gradient descent region. Within each multi-level gradient descent region, adjacent points with eigenvalues ​​less than the current grid point are selected as reachable candidate points. If the current point has an eigenvalue of 0.75, and an adjacent point has an eigenvalue of 0.70 and is located within the gradient descent region, then that point is selected as a reachable candidate point.

[0100] Next, the detection step size is determined based on the rate of change of feature values ​​in the region connecting the current grid point and reachable candidate points. If the rate of change of feature values ​​in the connecting region is 0.02 / unit distance, the detection step size is set to the reciprocal of the rate of change multiplied by a safety factor, i.e., 0.4, meaning a detection is performed every 0.4 units of distance. The detection step size is used to perform segment-by-segment detection on the connecting line to determine whether it intersects with the obstacle boundary. Starting from the current point, the feature value at that location is checked every 0.4 units of distance. If the feature value suddenly increases beyond the threshold of 0.3, it indicates that the obstacle boundary may have been crossed.

[0101] Check the continuity of feature values ​​along the connection line. If the feature value of a point on the connection line is 1.5, while the feature values ​​of its immediate and adjacent detection points are 0.6 and 0.7 respectively, the feature value jump exceeds the threshold of 0.3, and it is determined that the feature value is discontinuous. Reachable candidate points that do not intersect with obstacles and whose feature values ​​change continuously are identified as reachable points. For example, the feature values ​​along the connection line from point (10, 15, 20) to point (12, 16, 20) are 0.75, 0.72, 0.68, and 0.65 respectively, showing continuous change and not exceeding the threshold, so (12, 16, 20) is identified as a reachable point. Establish a reachability matrix based on the reachability distribution of each grid point. In the matrix, an element value of 1 indicates that the corresponding grid point is reachable, and 0 indicates that it is not reachable.

[0102] The reachability is calculated by proposing the proportion of reachable points to the total number of neighboring points for each grid point in the reachability matrix. If a grid point has 26 neighboring points, and 18 of them are reachable, then the reachability of that point is 18 / 26 = 0.69. The reachability is then weighted according to the level of the gradient descent region in which the grid point is located. For points in a strong gradient descent region, the weighting factor is 0.6; for points in a medium gradient descent region, it is 0.8; for points in a weak gradient descent region, it is 1.0; and for points outside the gradient descent region, it is 1.5.

[0103] The state transition cost is obtained by combining the weighting coefficients with the distance between grid points, the change in heading, and the rate of change of the feature field. For example, if the distance from the current point (10, 15, 20) to the reachable point (12, 16, 20) is 2.24, the change in heading is 0.1 radians, the rate of change of the feature field is 0.05, and this point is located in the middle gradient descent region with a weighting coefficient of 0.8, then the state transition cost is 0.8 × (2.24 + 0.1 × 10 + 0.05 × 20) = 3.79. A state transition cost matrix is ​​constructed, where each element represents the state transition cost between corresponding grid points, and the cost between unreachable points is set to infinity.

[0104] Finally, the reachability matrix and state transition cost matrix are combined to construct the path search space. The reachability matrix determines whether grid points can be directly connected, while the state transition cost matrix provides information on the cost of such connections. The search algorithm can find the optimal path from the starting point to the ending point in this space, avoiding obstacles and satisfying navigation constraints. For example, in the path search from the starting point (5, 5, 10) to the ending point (25, 25, 10), feasible connections can be filtered based on the reachability matrix, and the path with the minimum cost can be selected using the state transition cost matrix. The final path point sequence is (5, 5, 10), (8, 7, 10), (12, 11, 11), (16, 15, 11), (20, 20, 10), (25, 25, 10), with a total cost of 42.5.

[0105] In this embodiment, by spatially decomposing and directionally projecting the eigenvalue gradients of grid points in the target-environment fusion feature field, the terrain change trend and passable directions can be accurately identified. Constructing a multi-level gradient descent domain helps to dynamically adapt to environmental structures of varying complexity. By setting the detection step size based on the eigenvalue change rate and performing segment-by-segment connection detection, it is ensured that reachable paths do not intersect obstacle boundaries in space and have good feature continuity, thus improving the accuracy of reachability determination. The reachability matrix constructed based on the distribution of reachable points reflects the connectivity characteristics of the path network. The state transition cost matrix, constructed by integrating factors such as gradient level, heading change, and terrain complexity, enables a quantitative assessment of path traversal difficulty, thereby forming a trajectory search space with environmental adaptability and path optimization capabilities, providing a solid foundation for generating high-quality autonomous trajectories.

[0106] like Figure 2 The diagram illustrates the drone trajectory planning process in this embodiment.

[0107] In one optional implementation, generating a set of candidate tracks that satisfy motion constraints within the track search space, and determining the optimal planned track by calculating the cumulative cost of each track in the candidate track set includes:

[0108] Within the trajectory search space, the weight coefficients of the grid points are normalized to obtain the basic transition probability. Based on the basic transition probability and the gradient principal direction, the state transition probability distribution is calculated, and an initial set of sampling nodes is generated in each gradient descent domain.

[0109] The node search radius is determined based on the level of the gradient descent domain where the sampling node is located. A nearest neighbor search is performed within the search radius to obtain a set of connectable nodes. Based on the reachability matrix, it is determined whether the connectable nodes meet the motion constraints. Nodes that meet the motion constraints are connected and given a state transition cost to construct a candidate trajectory search tree.

[0110] Motion constraint verification is performed on the paths in the candidate track search tree, and path nodes that satisfy the steering angle constraint and steering angle acceleration constraint are extracted as control point sequences. The control point sequences are then smoothed to generate a candidate track set.

[0111] Based on the state transition cost and gradient descent domain level, hierarchical cost weights are set, the cumulative cost of each track in the candidate track set is calculated, the track with the minimum cumulative cost is selected as the current optimal track, local sampling is performed with the control point of the optimal track as the center to obtain a new control point sequence, the new control point sequence is smoothed to generate a new candidate track, the cumulative cost of the new candidate track is calculated and the optimal track is updated, and local optimization is iteratively performed until the cumulative cost converges to obtain the optimal planned track.

[0112] For example, within the track search space, the weight coefficients of grid points are normalized to obtain the basic transition probability. Specifically, for each grid point (x, y) in the track search space, a weight coefficient w(x, y) is calculated based on factors such as obstacle distance, terrain complexity, and tactical threat. By normalizing all weight coefficients in the entire search space, the basic transition probability p(x, y) is obtained. For instance, when the weight coefficient w(x, y) = 0.8 and the maximum weight coefficient in the search space is 1.0, the basic transition probability p(x, y) of that point = 0.8 / 1.0 = 0.8.

[0113] The state transition probability distribution is calculated based on the basic transition probabilities and the principal gradient direction. The principal gradient direction is determined by calculating the derivatives of the basic transition probabilities of grid points in each direction. For example, if the derivatives of the basic transition probabilities for the east, south, west, and north directions of a grid point (5, 3) are 0.2, -0.1, -0.3, and 0.1 respectively, then the principal gradient direction is east. The state transition probability distribution is weighted according to the principal gradient direction; the transition probability increases in the principal direction, and decreases in directions perpendicular to the principal direction. If the principal gradient direction is east, the transition probability in the east direction can increase by 20%, decrease by 10% in the north-south direction, and decrease by 30% in the west direction.

[0114] Initial sampling node sets are generated within each level of the gradient descent domain. The gradient descent domain is divided into multiple levels based on the magnitude of the base transition probability. For example, the region with a base transition probability greater than 0.8 is the first-level gradient descent domain, 0.6-0.8 is the second-level, 0.4-0.6 is the third-level, and so on. Different sampling densities are used within different levels of the gradient descent domain. The first-level domain has the highest sampling density, which can be set to one sampling point every 10 meters; the second-level domain can be set to one sampling point every 20 meters; the third-level domain can be set to one sampling point every 30 meters, and so on. In this way, an initial sampling node set is generated throughout the entire trajectory search space.

[0115] The search radius for a node is determined based on the level of the gradient descent region in which the sampled node is located. For example, the search radius for nodes in a first-level region is set to 50 meters, for a second-level region it is 40 meters, for a third-level region it is 30 meters, and so on. A nearest neighbor search is performed within the determined search radius to obtain the set of connectable nodes. For example, if node A is located in a first-level gradient descent region with a search radius of 50 meters, then all nodes within the search radius constitute the set of connectable nodes for A.

[0116] The reachability matrix is ​​used to determine whether connectable nodes satisfy motion constraints. The reachability matrix includes constraint parameters such as the mobile platform's minimum turning radius, maximum climb / descent angle, and maximum speed. For example, if the mobile platform's minimum turning radius is 100 meters, then if the angle between the line connecting two nodes and the node's current heading exceeds 30 degrees, it is considered unreachable. For nodes that satisfy the motion constraints, a connection is established and a state transition cost is assigned, constructing a candidate path search tree. The state transition cost considers a weighted sum of factors such as distance, change in turning angle, and change in altitude. For example, if the distance between two points is 200 meters, the change in turning angle is 15 degrees, and the change in altitude is 10 meters, then the state transition cost can be calculated as 200 + 15 × 5 + 10 × 10 = 375.

[0117] Motion constraint verification is performed on the paths in the candidate path search tree. Verification includes steering angle constraints and steering angle acceleration constraints. The steering angle constraint requires that the steering angle of adjacent path segments does not exceed the maximum allowable value, such as 30 degrees; the steering angle acceleration constraint requires that the rate of change of the steering angle per unit time does not exceed the maximum allowable value, such as 5 degrees / second. Path nodes that satisfy these constraints are extracted as a control point sequence. For example, a path from the starting point to the ending point contains 10 nodes. After motion constraint verification, only nodes 1, 3, 5, 7, and 10 may satisfy the constraints; these 5 nodes constitute the control point sequence.

[0118] A candidate track set is generated by smoothing the control point sequence. Cubic spline interpolation is used to smooth the control point sequence, inserting multiple transition points between every two adjacent control points. This ensures the continuity of the first and second derivatives of the track at the control points, thus guaranteeing track smoothness. For example, five transition points are inserted between the control point sequences (0, 0), (100, 100), (200, 150), (300, 200), and (400, 100) to form a smooth track containing 25 points.

[0119] Hierarchical cost weights are set based on state transition cost and gradient descent domain level. The state transition cost weight is lowest in the first-level gradient descent domain, set to 0.8; 1.0 for the second-level domain; 1.2 for the third-level domain, and so on. The cumulative cost of each track in the candidate track set is calculated. For example, if the total cost of a track through the first-level domain is 300, the total cost through the second-level domain is 200, and the total cost through the third-level domain is 100, then the cumulative cost of this track is 300 × 0.8 + 200 × 1.0 + 100 × 1.2 = 560. The track with the minimum cumulative cost is selected as the current optimal track.

[0120] A new control point sequence is obtained by local sampling centered on the control point of the optimal trajectory. Five new points are randomly sampled within a 10-meter radius of each control point, replacing the original control points to generate a new control point sequence. The new control point sequence is then smoothed to generate new candidate trajectories. The cumulative cost of the new candidate trajectories is calculated, and the optimal trajectory is updated. For example, if the cumulative cost of the original optimal trajectory is 560, and the cumulative cost of the new trajectory obtained after local optimization is 530, then the optimal trajectory is updated to the new trajectory. Local optimization is iteratively performed until the change in cumulative cost is less than a preset threshold (e.g., 1%), at which point convergence is considered achieved, yielding the final optimal planned trajectory.

[0121] In this embodiment, by constructing basic transition probabilities and state transition probability distributions within the trajectory search space, sampling nodes can be dynamically guided to cluster in the gradient descent direction, improving path search efficiency. Layered setting of node search radii and motion constraint verification ensures that the generated trajectory meets the UAV's maneuverability limitations in terms of turning angle and acceleration. Multi-layered cost weight accumulation calculation of candidate trajectories helps to comprehensively consider path smoothness, traversability, and environmental complexity, achieving a balance between global optimality and local feasibility. Iterative optimization based on local sampling and trajectory smoothing further refines the optimal trajectory, significantly improving path continuity and safety margin, making the final planned trajectory more adaptable to efficient tracking execution under dynamic environments and multiple constraints.

[0122] In one optional implementation, the flight path is divided into multiple segments based on the curvature variation of the optimal planned trajectory, and online trajectory optimization is performed on the flight segments to obtain a continuous attitude sequence, including:

[0123] Obtain the curvature change value of adjacent trajectory points on the optimal planned trajectory, determine the trajectory points with curvature greater than the preset curvature threshold as curvature change points, and divide the optimal planned trajectory into multiple flight segments with the curvature change points as the boundary;

[0124] A fixed-length sliding time window is established for each flight segment. The initial position of the sliding time window is set at the beginning of the flight segment. Within the sliding time window, the expected overload command range and expected roll rate range of the control point are determined based on the curvature gradient sequence of the current position, and multiple sets of candidate control input sequences are generated.

[0125] From multiple candidate control input sequences, the control input sequence that satisfies flight dynamics constraints and has the minimum curvature change is selected as the optimal control input, and a continuous attitude sequence for the current time window is generated based on the optimal control input.

[0126] The final state of the continuous attitude sequence is used as the initial state of the next sliding time window. The sliding time window is slid forward and the optimization process is repeated until the trajectory optimization of the current flight segment to be optimized is completed. After the trajectory optimization of all flight segments is completed, the optimized continuous attitude sequences of each flight segment are combined to obtain the final continuous attitude sequence.

[0127] For example, before trajectory optimization, the optimal planned trajectory needs to be analyzed and processed first. The curvature change values ​​of adjacent trajectory points on the optimal planned trajectory are obtained, calculated as the absolute value of the difference between the curvature of the current trajectory point and the curvature of the previous trajectory point. In practical applications, a preset curvature threshold of 0.05 rad / m can be set. When the curvature change value of a trajectory point exceeds this threshold, it is marked as a curvature abrupt change point. For example, for a trajectory containing 100 trajectory points, calculations might reveal that the curvature change values ​​of points 25, 47, and 78 are 0.063 rad / m, 0.072 rad / m, and 0.058 rad / m, respectively, all exceeding the preset threshold. Therefore, these three points are identified as curvature abrupt change points. Using these curvature abrupt change points as boundaries, the entire optimal planned trajectory is divided into four flight segments: points 1 to 25, 26 to 47, 48 to 78, and 79 to 100.

[0128] For each segmented flight path, a fixed-length sliding time window is established for online trajectory optimization. The length of the sliding time window can be set to 2 seconds, with a sampling interval of 0.1 seconds, meaning each window contains 21 time points. Initially, the starting position of the sliding time window is set at the first trajectory point of the current flight path to be optimized. Based on the curvature gradient sequence at the current position within the window, the desired overload command range and desired roll rate range of the control point are determined.

[0129] When determining the control input range, based on the aircraft's dynamic characteristics and flight mission requirements, the overload command range can be set to 1g to 5g, where g represents the standard gravitational acceleration (9.81 m / s²). 2 The roll rate range is set to -60° / s to 60° / s. Multiple candidate control input sequences are generated discretely within this range. For example, the overload command can be uniformly sampled into nine values ​​within the range of 1g to 5g: 1g, 1.5g, 2g, 2.5g, 3g, 3.5g, 4g, 4.5g, and 5g; the roll rate can be uniformly sampled into seven values ​​within the range of -60° / s to 60° / s: -60° / s, -40° / s, -20° / s, 0° / s, 20° / s, 40° / s, and 60° / s. By combining these discrete values, 9 × 7 = 63 candidate control input sequences can be generated.

[0130] For each generated candidate control input sequence, a forward simulation is performed using a flight dynamics model to calculate the corresponding flight trajectory and attitude changes. Specifically, each candidate control input sequence (such as thrust, control surface angle, overload command, roll rate, etc.) is used as the driving signal for the model and input into the flight dynamics model. The flight dynamics model calculates the forces and moments acting on the aircraft based on rigid body kinematics and dynamic equations (such as the Newton-Euler equations), and performs numerical integration based on the current state (position, velocity, attitude, angular velocity, etc.) to obtain the motion state at the next moment. Through iterative time steps, the control input sequence is applied to the model to simulate the trajectory, position, and attitude changes of the aircraft over continuous time, forming a complete flight trajectory and attitude time series. During the simulation, it is necessary to verify whether the trajectory meets the flight dynamics constraints, including maximum overload constraints, maximum roll rate constraints, and attitude change continuity constraints. For candidate sequences that meet the constraints, their curvature change value is calculated, specifically by calculating the sum of squares of the curvature differences between adjacent points in the sequence. The smaller the curvature change value, the smoother the trajectory.

[0131] From the candidate control input sequences that meet the constraints, the sequence with the smallest curvature change is selected as the optimal control input. Assume that within a certain time window, calculations show that the control input sequence with an overload of 3g and a roll rate of 20° / s produces the smallest curvature change, which is 0.012 rad. 2 / m 2 Therefore, this is determined as the optimal control input for the current window. Based on this optimal control input, a continuous attitude sequence within the current time window is generated through the flight dynamics model, including parameters such as position, velocity, acceleration, heading angle, pitch angle, and roll angle.

[0132] After optimizing the current time window, the final state of the continuous attitude sequence is used as the initial state for the next sliding time window. For example, if the optimized final state of the current window is: position coordinates (1000m, 500m, 200m), speed 300m / s, heading angle 45°, pitch angle 10°, and roll angle 5°, these parameter values ​​will be used as the initial state for the next window. The sliding time window is moved forward by a certain step, such as 0.5 seconds, and the above optimization process is repeated. Sliding window optimization continues until the entire trajectory of the current flight segment is optimized. When optimizing to the end of the segment, the length of the last sliding window may need to be adjusted to ensure coverage of the entire segment. After completing the above optimization process for all flight segments, the optimized continuous attitude sequences of each segment are combined sequentially to form a complete, smoothly curvatured continuous attitude sequence, which serves as the final flight command for the aircraft.

[0133] Based on the above technical solutions, fine-grained dynamic adjustments and continuous attitude control of complex trajectories at the flight execution level can be achieved. By detecting and segmenting the curvature changes of the optimal planned trajectory, regions of varying maneuver intensity can be effectively identified and segmented optimized control can be implemented, enhancing adaptability to highly dynamic flight missions. Utilizing a sliding time window for segmented online optimization, combined with dynamic adjustment of the control input range based on curvature gradient characteristics, a smoother and more stable attitude change sequence can be generated while meeting flight dynamics constraints, reducing the load impact of abrupt attitude commands on the flight system. A window-based recursive optimization approach achieves continuity and real-time performance of the attitude sequence across the entire flight segment, contributing to improved path tracking accuracy, flight stability, and control response efficiency. This is particularly suitable for highly dynamic obstacle avoidance flight and autonomous navigation missions in complex terrain.

[0134] In one optional implementation, an adaptive thrust allocation strategy and a real-time attitude control strategy are generated based on a continuous attitude sequence combined with environmental disturbance compensation, and then combined into segmented execution commands. Controlling the UAV to complete trajectory tracking for a flight segment according to these segmented execution commands includes:

[0135] The desired attitude command and desired trajectory information are calculated based on a continuous attitude sequence. The current flight state and environmental disturbance information of the UAV are collected, and the attitude deviation value is obtained by comparing the current flight state with the desired attitude command.

[0136] The thrust compensation is calculated based on the attitude deviation value, and the thrust compensation is adaptively adjusted in combination with environmental disturbance information to generate an adaptive thrust allocation strategy. A real-time attitude control strategy is generated based on the desired attitude command and attitude deviation value.

[0137] Based on the feature points of the desired trajectory information, the flight segment is divided into multiple continuous control intervals, and the adaptive thrust allocation strategy and real-time attitude control strategy are combined to form segmented execution commands.

[0138] Configure corresponding control parameters in each control interval, and use a smooth transition function at the boundary point of adjacent control intervals to achieve continuous switching of the segmented execution commands. Control the UAV to complete the trajectory tracking of the flight segment according to the switched segmented execution commands.

[0139] In this embodiment, a continuous attitude sequence is first acquired. A third-order spline interpolation algorithm is then used to process these discrete attitude points, generating a smooth and continuous desired attitude curve to ensure the continuity of the UAV's motion. Based on this curve, the desired attitude command and desired trajectory information at each time point are calculated. Based on the generated smooth and continuous attitude curve, the curve is discretized according to a preset sampling time interval to obtain the attitude angle values ​​at each sampling moment, including roll, pitch, and yaw angles. These attitude angles serve as the desired attitude command for that moment, providing real-time reference and control for the flight control system. Simultaneously, combined with the UAV's flight speed information, the desired attitude angles are converted into flight direction vectors. By calculating the change in the flight direction vector between adjacent sampling moments and combining it with the speed magnitude, the corresponding spatial position change is derived, thereby accumulating and calculating the desired trajectory point sequence of the UAV in three-dimensional space. For example, in a certain flight mission, the original 20 discrete attitude points might be interpolated into 100 smooth and continuous attitude points with a time interval of 50 milliseconds, covering the entire 5-second flight segment.

[0140] Simultaneously, the system collects the UAV's current flight status, including position coordinates, flight speed, attitude angles, and angular rates, using sensors such as accelerometers, gyroscopes, and GPS. Environmental disturbance information is acquired through barometers, wind speed sensors, and a disturbance observer, which can estimate wind speed, direction, and turbulence. The system compares the current attitude angles with the desired attitude commands to calculate attitude deviations, including roll, pitch, and yaw angle deviations.

[0141] Based on the aforementioned attitude deviation values, a nonlinear feedback control algorithm is used to calculate the thrust compensation. Specifically, the basic thrust requirement is first calculated, and then the compensation coefficient is determined according to the magnitude and rate of change of the attitude deviation. For example, when a roll angle deviation exceeding 5 degrees is detected, a 10% thrust adjustment is added in the corresponding direction according to a preset compensation rule. These basic compensation amounts are then adaptively adjusted in conjunction with environmental disturbance information.

[0142] In the environmental disturbance compensation stage, an adaptive disturbance compensation algorithm is used to handle external interference. When a lateral wind speed of 5 m / s is detected, a 15% counter-thrust is added to the original thrust compensation to counteract the wind's influence. Simultaneously, to cope with sudden airflow changes, a rapid response mechanism is implemented, capable of completing disturbance assessment and thrust reallocation within 200 milliseconds. This adaptive mechanism ensures the stability of the UAV under various environmental conditions.

[0143] After combining the adaptive thrust allocation strategy with the real-time attitude control strategy, the flight segment needs to be divided into multiple consecutive control zones. The division is based on characteristic points of the desired trajectory, such as sharp turns, acceleration / deceleration points, and hovering points. For example, a trajectory comprising three phases—uniform straight-line flight, a 90-degree turn, and a climb—will be divided into three control zones, each configured with different control parameters. The first zone uses a parameter combination prioritizing speed control, the second zone uses a parameter combination prioritizing attitude accuracy, and the third zone uses a parameter combination prioritizing altitude stability.

[0144] To achieve a smooth transition between control intervals, a weighting function is used for gradual switching. Near the boundary between two adjacent control intervals, the control strategy is not switched immediately, but rather the parameter weights are gradually adjusted within a transition interval (typically 500 milliseconds). Specifically, a sigmoid function is used as the weighting transition function, ensuring that the control quantity changes smoothly over time and avoiding system oscillations caused by abrupt changes. For example, when the UAV enters a turning region from straight flight, the weight of the turning control parameters is gradually increased starting 250 milliseconds before the boundary, while the weight of the straight flight parameters is decreased. The entire transition process lasts 500 milliseconds, ensuring the smoothness of the UAV's movements.

[0145] Finally, based on the specific parameters of each control interval and the segmented execution instructions processed by the smooth transition function, real-time motor control signals are generated. In each control cycle (typically 10 milliseconds), the current state is reassessed, thrust distribution and attitude control strategies are updated, and corresponding control commands are executed, forming a closed-loop control system to ensure the UAV can accurately track the predetermined trajectory. Tests show that this method enables the UAV to maintain a trajectory deviation of less than 0.5 meters and an attitude angle deviation of less than 3 degrees under Force 6 wind conditions, significantly improving the accuracy and reliability of flight missions.

[0146] Based on the above technical solutions, high-precision trajectory tracking and stable flight control of UAVs in complex dynamic environments can be achieved. By comparing continuous attitude sequences with real-time flight status, attitude deviations are accurately obtained, effectively guiding the adaptive adjustment of thrust compensation, and enhancing the responsiveness to environmental disturbances and control robustness. Segmented command execution divides the flight segment into multiple control intervals, and combined with a smooth transition function, seamless switching of control parameters is achieved, ensuring the continuity and stability of flight control and avoiding abrupt changes and oscillations caused by control switching. The overall strategy integrates thrust distribution and attitude control, improving the UAV's dynamic adaptability and trajectory tracking accuracy in changing environments, enhancing flight safety and mission execution efficiency.

[0147] A second aspect of this invention provides a deep learning-based autonomous obstacle avoidance and path planning system for unmanned aerial vehicles (UAVs), the system comprising:

[0148] The first unit is used to acquire the drone's location information and environmental perception data;

[0149] The second unit is used to construct a spatiotemporal feature matrix from environmental perception data, extract target motion feature vectors and background feature vectors from the spatiotemporal feature matrix, calculate motion trajectory projection based on target motion feature vectors, construct environmental constraint boundaries based on background feature vectors, and map motion trajectory projections and environmental constraint boundaries onto a three-dimensional mesh space to obtain a target-environment fusion feature field.

[0150] The third unit is used to calculate the reachability matrix of grid points based on the target-environment fusion feature field, calculate the state transition cost between adjacent grid points to generate the cost matrix, combine the reachability matrix and the cost matrix to construct the trajectory search space, generate a set of candidate trajectories that meet the motion constraints in the trajectory search space, and determine the optimal planned trajectory by calculating the cumulative cost of each trajectory in the candidate trajectory set.

[0151] The fourth unit is used to divide the flight path into multiple flight segments based on the curvature change of the optimal planned trajectory, perform online trajectory optimization on the flight segments to obtain a continuous attitude sequence, generate an adaptive thrust allocation strategy and a real-time attitude control strategy based on the continuous attitude sequence and environmental disturbance compensation, and combine them into segmented execution commands. The UAV is then controlled to complete the trajectory tracking of the flight segments according to the segmented execution commands.

[0152] A third aspect of the present invention provides an electronic device, comprising:

[0153] processor;

[0154] Memory used to store processor-executable instructions;

[0155] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0156] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0157] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A deep learning-based autonomous obstacle avoidance and path planning method for unmanned aerial vehicles (UAVs), characterized in that, include: Acquire the drone's location information and environmental perception data; A spatiotemporal feature matrix is ​​constructed from environmental perception data. Target motion feature vectors and background feature vectors are extracted from the spatiotemporal feature matrix. The motion trajectory projection is calculated based on the target motion feature vectors. An environmental constraint boundary is constructed based on the background feature vectors. The motion trajectory projection and the environmental constraint boundary are mapped onto a three-dimensional mesh space to obtain a target-environment fusion feature field. The reachability matrix of grid points is calculated based on the target-environment fusion feature field. The state transition cost between adjacent grid points is calculated to generate the cost matrix. The reachability matrix and the cost matrix are combined to construct the trajectory search space. A set of candidate trajectories that meet the motion constraints is generated in the trajectory search space. The optimal planned trajectory is determined by calculating the cumulative cost of each trajectory in the candidate trajectory set. The flight path is divided into multiple segments based on the curvature change of the optimal planned trajectory. The flight path segments are optimized online to obtain a continuous attitude sequence. The desired attitude command and desired trajectory information are calculated based on the continuous attitude sequence. The current flight state and environmental disturbance information of the UAV are collected. The current flight state is compared with the desired attitude command to obtain the attitude deviation value. The thrust compensation is calculated based on the attitude deviation value, and the thrust compensation is adaptively adjusted in combination with environmental disturbance information to generate an adaptive thrust allocation strategy. A real-time attitude control strategy is generated based on the desired attitude command and attitude deviation value. Based on the feature points of the desired trajectory information, the flight segment is divided into multiple continuous control intervals, and the adaptive thrust allocation strategy and real-time attitude control strategy are combined to form segmented execution commands. Configure corresponding control parameters in each control interval, and use a smooth transition function at the boundary point of adjacent control intervals to achieve continuous switching of the segmented execution commands. Control the UAV to complete the trajectory tracking of the flight segment according to the switched segmented execution commands.

2. The method according to claim 1, characterized in that, A spatiotemporal feature matrix is ​​constructed from the environmental perception data. Target motion feature vectors and background feature vectors are extracted from this matrix, including: Multiple sets of environmental data are sampled at preset time intervals for environmental perception data. The environmental data are arranged in the time dimension to form a data stream. The data stream is subjected to temporal filtering and amplitude normalization to obtain a time-series data sequence. Multi-scale convolution kernels are used to extract features from the time-series data sequence to obtain a multi-scale feature map. The multi-scale feature maps are fused to obtain a spatiotemporal feature matrix. The data differences between adjacent time points are calculated in the spatiotemporal feature matrix and threshold segmentation is performed to obtain the moving target region and the background region; The cross-correlation coefficient is calculated for data from adjacent time points in the moving target region to obtain a correlation coefficient matrix. The position offset corresponding to the maximum correlation is extracted from the correlation coefficient matrix to obtain the motion direction vector. Morphological processing is performed on the moving target region to extract boundary coordinates to obtain the target contour. The motion feature vector is obtained based on the centroid displacement of the target contour along the motion direction and the motion direction vector. Feature points are extracted from the background region to calculate disparity and obtain a depth map. Bilateral filtering is applied to the depth map to obtain depth information. Edge detection is performed on the background region to obtain an edge point set. Edge information is obtained by connecting the edge points using a double threshold method. The depth information and edge information are spatially aligned and combined to obtain a background feature vector.

3. The method according to claim 1, characterized in that, The motion trajectory projection is calculated based on the target motion feature vector, and the environmental constraint boundary is constructed based on the background feature vector. The motion trajectory projection and the environmental constraint boundary are mapped onto the three-dimensional mesh space to obtain the target-environment fusion feature field, which includes: Based on the target motion feature vector, a state equation is established to calculate the target position and velocity changes. Based on the target position and velocity changes, a recursive iterative method is used to predict the target's future position sequence to obtain the target motion trajectory. The trajectory projection is obtained by projecting it onto the observation space through coordinate transformation. Point cloud data is constructed using depth information in background feature vectors, and the point cloud data is segmented and clustered to obtain the three-dimensional spatial structure of the environment. Obstacle contours are extracted from the three-dimensional spatial structure of the environment to obtain obstacle boundaries, and passable areas are calculated to obtain passable boundaries. The obstacle boundaries and passable boundaries are combined to obtain environmental constraint boundaries. The motion trajectory projection is divided into grid cells according to a preset resolution and mapped to a three-dimensional grid space to obtain trajectory grid features. At the same time, the environmental constraint boundary is divided into grid cells according to a preset resolution and mapped to a three-dimensional grid space to obtain boundary grid features. The motion continuity score is obtained by calculating the state consistency between adjacent grid cells of the trajectory grid features. The structural integrity score is obtained by calculating the spatial coverage and boundary integrity of the boundary grid features. The motion continuity score and structural integrity score are normalized to obtain the feature fusion weight. The trajectory grid features and boundary grid features are adaptively weighted and superimposed according to the feature fusion weight to obtain the target-environment fusion feature field.

4. The method according to claim 1, characterized in that, The reachability matrix of grid points is calculated based on the target-environment fusion feature field. The state transition cost between adjacent grid points is calculated to generate the cost matrix. The reachability matrix and the cost matrix are combined to construct the track search space, including: Calculate the eigenvalue gradient of each grid point in the target-environment fusion feature field, decompose the eigenvalue gradient in three-dimensional space to obtain gradient components, determine the gradient principal direction based on the gradient components, calculate the gradient projection value of the grid point according to the gradient principal direction, divide adjacent grid points into multi-level gradient descent domains based on the gradient projection value, and select adjacent points with eigenvalues ​​less than the current grid point as reachable candidate points in the multi-level gradient descent domains. The detection step size is determined based on the rate of change of the feature values ​​of the region where the line connecting the current grid point and the reachable candidate point is located. The detection step size is used to detect the line segment by segment to determine whether it intersects with the obstacle boundary. The continuity of the feature values ​​on the line is checked. Reachable candidate points that do not intersect with obstacles and whose feature values ​​change continuously are determined as reachable points. A reachability matrix is ​​established based on the reachability distribution of each grid point. The reachability is obtained by calculating the proportion of reachable points of each grid point in the reachability matrix to the total number of adjacent points. The reachability is then graded and weighted according to the level of the gradient descent domain in which the grid point is located to obtain the weight coefficient. The weight coefficient is combined with the distance between grid points, the change in heading, and the change rate of the feature field to obtain the state transition cost, and a state transition cost matrix is ​​constructed. The reachability matrix and the state transition cost matrix are combined to construct the track search space.

5. The method according to claim 1, characterized in that, Within the trajectory search space, a set of candidate trajectories satisfying motion constraints is generated. The optimal planned trajectory is determined by calculating the cumulative cost of each trajectory in the candidate trajectory set, including: Within the trajectory search space, the weight coefficients of the grid points are normalized to obtain the basic transition probability. Based on the basic transition probability and the gradient principal direction, the state transition probability distribution is calculated, and an initial set of sampling nodes is generated in each gradient descent domain. The node search radius is determined based on the level of the gradient descent domain where the sampling node is located. A nearest neighbor search is performed within the search radius to obtain a set of connectable nodes. Based on the reachability matrix, it is determined whether the connectable nodes meet the motion constraints. Nodes that meet the motion constraints are connected and given a state transition cost to construct a candidate trajectory search tree. Motion constraint verification is performed on the paths in the candidate track search tree, and path nodes that satisfy the steering angle constraint and steering angle acceleration constraint are extracted as control point sequences. The control point sequences are then smoothed to generate a candidate track set. Based on the state transition cost and gradient descent domain level, hierarchical cost weights are set, the cumulative cost of each track in the candidate track set is calculated, the track with the minimum cumulative cost is selected as the current optimal track, local sampling is performed with the control point of the optimal track as the center to obtain a new control point sequence, the new control point sequence is smoothed to generate a new candidate track, the cumulative cost of the new candidate track is calculated and the optimal track is updated, and local optimization is iteratively performed until the cumulative cost converges to obtain the optimal planned track.

6. The method according to claim 1, characterized in that, The flight path is divided into multiple segments based on the curvature change of the optimal planned trajectory. Online trajectory optimization is then performed on these segments to obtain a continuous attitude sequence, including: Obtain the curvature change value of adjacent trajectory points on the optimal planned trajectory, determine the trajectory points with curvature greater than the preset curvature threshold as curvature change points, and divide the optimal planned trajectory into multiple flight segments with the curvature change points as the boundary; A fixed-length sliding time window is established for each flight segment. The initial position of the sliding time window is set at the beginning of the flight segment. Within the sliding time window, the expected overload command range and expected roll rate range of the control point are determined based on the curvature gradient sequence of the current position, and multiple sets of candidate control input sequences are generated. From multiple candidate control input sequences, the control input sequence that satisfies flight dynamics constraints and has the minimum curvature change is selected as the optimal control input, and a continuous attitude sequence for the current time window is generated based on the optimal control input. The final state of the continuous attitude sequence is used as the initial state of the next sliding time window. The sliding time window is slid forward and the optimization process is repeated until the trajectory optimization of the current flight segment to be optimized is completed. After the trajectory optimization of all flight segments is completed, the optimized continuous attitude sequences of each flight segment are combined to obtain the final continuous attitude sequence.

7. A deep learning-based autonomous obstacle avoidance and path planning system for unmanned aerial vehicles (UAVs), used to implement the method described in any one of claims 1-6, characterized in that, include: The first unit is used to acquire the drone's location information and environmental perception data; The second unit is used to construct a spatiotemporal feature matrix from environmental perception data, extract target motion feature vectors and background feature vectors from the spatiotemporal feature matrix, calculate motion trajectory projection based on target motion feature vectors, construct environmental constraint boundaries based on background feature vectors, and map motion trajectory projections and environmental constraint boundaries onto a three-dimensional mesh space to obtain a target-environment fusion feature field. The third unit is used to calculate the reachability matrix of grid points based on the target-environment fusion feature field, calculate the state transition cost between adjacent grid points to generate the cost matrix, combine the reachability matrix and the cost matrix to construct the trajectory search space, generate a set of candidate trajectories that meet the motion constraints in the trajectory search space, and determine the optimal planned trajectory by calculating the cumulative cost of each trajectory in the candidate trajectory set. The fourth unit is used to divide the flight path into multiple flight segments based on the curvature change of the optimal planned trajectory, perform online trajectory optimization on the flight segments to obtain a continuous attitude sequence, generate an adaptive thrust allocation strategy and a real-time attitude control strategy based on the continuous attitude sequence and environmental disturbance compensation, and combine them into segmented execution commands. The UAV is then controlled to complete the trajectory tracking of the flight segments according to the segmented execution commands.

8. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 6.

Images