Time distribution method of unmanned aerial vehicle track, electronic equipment and storage medium

Abstract

The invention provides an unmanned aerial vehicle track time distribution method, electronic equipment and a storage medium, and relates to the technical field of unmanned aerial vehicles, and the method comprises the steps: inputting an initial state, a termination state and a flight corridor sequence of a target unmanned aerial vehicle into a pre-trained track optimization network structure, obtaining a trajectory result of the target unmanned aerial vehicle output by the trajectory optimization network structure; and based on the trajectory result, generating a corresponding control instruction and sending the control instruction to the target unmanned aerial vehicle to control the target unmanned aerial vehicle to fly in the target route, the trajectory result is used for indicating the time spent by the unmanned aerial vehicle on each section of sub-trajectory under the target route and the position coordinate, the speed value, the pitching angle value, the rolling angle value and the yaw angle speed value of each section of sub-trajectory under the preset moment. The safety and planning efficiency of the track of the unmanned aerial vehicle in a complex environment are improved.

Description

Technical Field

[0001] This application relates to the technical field of unmanned aerial vehicles, and more particularly, to a method for time allocation of an unmanned aerial vehicle trajectory, an electronic device, and a storage medium. Background Art

[0002] Traditional unmanned aerial vehicle trajectory planning methods mostly adopt static optimization techniques and cannot fully cope with the challenges of dynamic changes in complex environments. In addition, time allocation is also important for the trajectory optimization of unmanned aerial vehicles. Reasonable time allocation can significantly reduce flight time, save energy, and improve the task completion efficiency. However, currently, the time allocation optimization of unmanned aerial vehicle trajectories is basically a simple average allocation, and simple time allocation under complex conditions may affect the quality of the generated trajectory. Summary of the Invention

[0003] The purpose of the embodiments of this application is to provide a method for time allocation of an unmanned aerial vehicle trajectory, an electronic device, and a storage medium, so as to improve the safety and planning efficiency of unmanned aerial vehicle trajectories in complex environments.

[0004] In a first aspect, the present invention provides a method for time allocation of an unmanned aerial vehicle trajectory, the method including:

[0005] Input the initial state, the terminal state, and the flight corridor sequence of the target unmanned aerial vehicle into a pre-trained trajectory optimization network structure to obtain the trajectory result of the target unmanned aerial vehicle output by the trajectory optimization network structure;

[0006] Generate corresponding control instructions based on the trajectory result and send them to the target unmanned aerial vehicle to control the target unmanned aerial vehicle to fly along the target route;

[0007] wherein, the initial state is used to indicate the position coordinates, speed value, pitch angle value, roll angle value, and yaw angular velocity value of the unmanned aerial vehicle at the initial moment, the terminal state is used to indicate the position coordinates, speed value, pitch angle value, roll angle value, and yaw angular velocity value of the unmanned aerial vehicle at the terminal moment, the flight corridor sequence is used to indicate the safe flight range of the unmanned aerial vehicle, and the trajectory result is used to indicate the time spent on each sub-trajectory of the unmanned aerial vehicle along the target route, as well as the position coordinates, speed value, pitch angle value, roll angle value, and yaw angular velocity value at a preset moment in each sub-trajectory;

[0008] The target route is the route between the position coordinates at the initial moment and the position coordinates at the terminal moment.

[0009] In an alternative embodiment, the trajectory optimization network structure includes a first feature extraction layer, a second feature extraction layer, an LSTM layer, a time allocation layer, and a solver. The first feature extraction layer is used to extract the first feature vector of the flight corridor sequence and output it to the LSTM layer. The second feature extraction layer is used to extract the second feature vectors of the initial state and the termination state of the target unmanned aerial vehicle (UAV) and output them to the LSTM layer. The LSTM layer is used to output the hidden state to the time allocation layer based on the first feature vector and the second feature vector. The time allocation layer is used to map the hidden state to time allocation to output the time spent by the UAV on each sub-trajectory under the target route. The solver is used to solve the flat output at the preset moment in each sub-trajectory according to the optimization conditions for each sub-trajectory.

[0010] In an alternative embodiment, for each sub-trajectory, the solver solves for the position coordinates, speed value, pitch angle value, roll angle value, and yaw angular velocity value at the preset moment in the sub-trajectory in the following manner:

[0011] The solver takes the trajectory smoothness and flight time of the target UAV as the optimization objectives and solves the piecewise polynomial of the sub-trajectory. Among them, the optimization objective function is expressed as:

[0012]

[0013]

[0014] β(t) = [1, t, t 2 ,... t N T ;

[0015] where, σ i (t) is the flat output of the UAV sub-trajectory i, σ i (k) (t) is the k-th derivative of σ i (t), Δt k represents the time spent on the k-th segment of the trajectory, c i is the coefficient matrix of the UAV sub-trajectory i, β(t) is the polynomial basis function vector, M is the total number of piecewise polynomials, dt is the differential symbol of the time parameter t, N is the order of the piecewise polynomial, and T is the transpose operation symbol;

[0016] Based on the flat output of the UAV sub-trajectory i, the position coordinates, speed value, pitch angle value, roll angle value, and yaw angular velocity value at the preset moment in the sub-trajectory are determined.

[0017] In an alternative embodiment, the constraint conditions of the optimization objective function include initial state constraint, termination state constraint, continuity constraint between each sub-trajectory, safe flight range of each sub-trajectory, and dynamic constraint.​

[0018] In an alternative embodiment, the trajectory optimization network structure further includes an implicit micro layer, and the trajectory optimization network structure is trained to be generated in the following manner:

[0019] Input the sample data into the original trajectory optimization network structure to obtain the time allocation result output by the time allocation layer;

[0020] The implicit micro layer solves based on the time spent by the UAV on each sub-trajectory under the target route output by the time allocation layer and the KKT conditions to obtain a preset coefficient matrix;

[0021] Calculate the gradient of the preset coefficient matrix with respect to the time allocation result;

[0022] Adjust the time allocation result and the parameters of the trajectory optimization network structure based on the calculated gradient to obtain the trained trajectory optimization network structure.

[0023] In an alternative embodiment, the objective function of the KKT condition is:

[0024]

[0025] where Γ represents the function of the KKT condition, c * is the preset coefficient matrix, v * and λ * are Lagrange multipliers, b is the constant term vector of the equality constraint, h is the constant term vector of the inequality constraint, diag is the diagonal matrix construction operation symbol, λ is the Lagrange multiplier related to the inequality constraint, Q(t) is the quadratic term coefficient matrix, A(t) is the linear constraint matrix, and G(t) is the inequality constraint matrix.

[0026] In an alternative embodiment, both the first feature extraction layer and the second feature extraction layer include a convolutional layer, a pooling layer, and a fully connected layer connected in sequence.

[0027] In a second aspect, the present invention provides a time allocation device for a UAV trajectory, and the device includes:

[0028] A prediction module, configured to input the initial state, the termination state, and the flight corridor sequence of the target UAV into a pre-trained trajectory optimization network structure to obtain the trajectory result of the target UAV output by the trajectory optimization network structure;

[0029] A control module, configured to generate a corresponding control instruction based on the trajectory result and send it to the target UAV to control the target UAV to fly under the target route;

[0030] Among them, the initial state is used to indicate the position coordinates, speed value, pitch angle value, roll angle value, and yaw angular velocity value of the UAV at the initial moment, the termination state is used to indicate the position coordinates, speed value, pitch angle value, roll angle value, and yaw angular velocity value of the UAV at the termination moment, and the flight corridor sequence is used to indicate the safe flight range of the UAV.

[0031] The trajectory result is used to indicate the time spent by each sub-trajectory of the UAV under the target route, as well as the position coordinates, speed value, pitch angle value, roll angle value, and yaw angular velocity value at the preset moment in each sub-trajectory.

[0032] The target route is the route between the position coordinates at the initial moment and the position coordinates at the termination moment.

[0033] In a third aspect, the present invention provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus, and the processor executes the machine-readable instructions to perform the steps of any of the methods for time allocation of the UAV trajectory in the foregoing embodiments.

[0034] In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it performs the steps of any of the methods for time allocation of the UAV trajectory in the foregoing embodiments.

[0035] A method for time allocation of an unmanned aerial vehicle (UAV) trajectory, an electronic device, and a storage medium provided by the present application. The method includes inputting the initial state, termination state, and flight corridor sequence of a target UAV into a pre-trained trajectory optimization network structure to obtain the trajectory result of the target UAV output by the trajectory optimization network structure; generating corresponding control instructions based on the trajectory result and sending them to the target UAV to control the target UAV to fly along a target route. The initial state is used to indicate the position coordinates, speed value, pitch angle value, roll angle value, and yaw angular velocity value of the UAV at the initial moment. The termination state is used to indicate the position coordinates, speed value, pitch angle value, roll angle value, and yaw angular velocity value of the UAV at the termination moment. The flight corridor sequence is used to indicate the safe flight range of the UAV. The trajectory result is used to indicate the time spent on each sub-trajectory of the UAV along the target route, as well as the position coordinates, speed value, pitch angle value, roll angle value, and yaw angular velocity value at a preset moment in each sub-trajectory. The target route is the route between the position coordinates at the initial moment and the position coordinates at the termination moment. By adjusting the flight trajectory and time allocation in real time according to environmental changes, the intelligence and real-time performance of trajectory planning are realized. Combining deep learning technology to optimize time allocation enables the UAV to generate smooth and safe flight trajectories in complex environments. Through a dynamic trajectory time allocation mechanism, the risk of collision with obstacles is effectively avoided, and the flight safety and the efficiency of trajectory planning are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] To more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required to be used in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0037] Figure 1 It is a flowchart of a method for time allocation of an unmanned aerial vehicle (UAV) trajectory provided by an embodiment of the present application;

[0038] Figure 2 It is a schematic structural diagram of a trajectory optimization network structure provided by an embodiment of the present application;

[0039] Figure 3 It is a schematic structural diagram of a device for time allocation of an unmanned aerial vehicle (UAV) trajectory provided by an embodiment of the present application;

[0040] Figure 4 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0041] First, the application scenario of this application is described. The technical solution of this application is applicable to the trajectory planning during the flight of an unmanned aerial vehicle (UAV).

[0042] Next, the technical solutions in the embodiments of this application will be described with reference to the accompanying drawings in the embodiments of this application.

[0043] Figure 1 It is a flowchart of a method for time allocation of a UAV trajectory provided for an embodiment of this application. As Figure 1 shown, a method for time allocation of a UAV trajectory provided for an embodiment of this application includes:

[0044] S1. Input the initial state, terminal state, and flight corridor sequence of the target UAV into a pre-trained trajectory optimization network structure to obtain the trajectory result of the target UAV output by the trajectory optimization network structure.

[0045] Here, the initial state is used to indicate the position coordinates, speed value, pitch angle value, roll angle value, and yaw angular velocity value of the UAV at the initial moment, the terminal state is used to indicate the position coordinates, speed value, pitch angle value, roll angle value, and yaw angular velocity value of the UAV at the terminal moment, and the flight corridor sequence is used to indicate the safe flight range of the UAV. The safe flight range (safe flight corridor) here is composed of a convex polyhedron formed by a series of hyperplanes.

[0046] The trajectory result is used to indicate the time spent by the UAV on each sub-trajectory under the target route, as well as the position coordinates, speed value, pitch angle value, roll angle value, and yaw angular velocity value at a preset moment in each sub-trajectory.

[0047] The target route is the route between the position coordinates at the initial moment and the position coordinates at the terminal moment.

[0048] S2. Generate corresponding control commands based on the trajectory result and send them to the target UAV to control the target UAV to fly under the target route.

[0049] Specifically, the polynomial curve of the UAV trajectory can be obtained through the following steps:

[0050] Represent the differential flat output of the UAV trajectory with the [x, y, z] of the UAV in the Cartesian coordinate system and the yaw angle of the UAV, and represent the system variables of the UAV with the flat output and its multi-order derivatives, including the position P of the UAV, the speed of the UAV the pitch angle θ of the UAV, the roll angle of the UAV the yaw angular velocity of the UAV Represent the trajectory σ(t) of the UAV with the relationship between the flat output and time. Each trajectory σ(t) is composed of multiple sub-items and multi-segments σ i (t) connected together, σi (t) is represented by the coefficient matrix of the i-th segment of the trajectory and the basis function vector.

[0051] The UAV trajectory through the flat output includes the UAV's [x, y, z] in the Cartesian coordinate system and the yaw angle of the UAV and is composed of. The differential flat output of the trajectory is expressed as:

[0052]

[0053] Through the differential flat output σ and its higher-order derivatives, the system variables of the UAV can be represented:

[0054] p = [x, y, z] T ;

[0055]

[0056] Through the differential flat representation, the relationship between the system variables of the UAV and the trajectory can be fully described.

[0057] A polynomial curve is used to represent the UAV's trajectory. σ(t) is used to represent the correspondence between the UAV's trajectory and time, and the flat output is used to represent the UAV's trajectory:

[0058]

[0059] Each sub-trajectory σ(t) of the UAV is composed of multiple sub-items multi-segment σ i (t) connected together:

[0060]

[0061] where σ i (t) represents the i-th sub-trajectory, and Δt k represents the time taken for the k-th sub-trajectory. Each sub-trajectory is only valid within a specific time interval.

[0062] The piecewise polynomial σ i (t) corresponding to each sub-trajectory is expressed as:

[0063]

[0064] where c i represents the coefficient matrix of the i-th sub-trajectory, and β(t) is the polynomial basis function vector.

[0065] The basis function β(t) is defined as:

[0066] β(t) = [1, t, t 2 ,... t N T ;

[0067] This vector is used to represent the polynomial form of this segment of the sub-trajectory.

[0068] In this embodiment, the UAV trajectory is represented in segments by a polynomial curve, and deep learning technology is combined to optimize the time allocation, enabling the UAV to generate a smooth and safe flight trajectory in a complex environment. The dynamic trajectory time allocation mechanism effectively avoids the risk of collision with obstacles, improving the flight safety and the efficiency of trajectory planning.

[0069] In an embodiment of the present application, deep learning is used to predict and optimize the time allocation of the UAV trajectory. The initial state, terminal state, and flight corridor sequence of the UAV are used as the input layer; the LSTM layer is used to process the time series input, and the hidden state h at the current moment is calculated recursively t and the cell state c t , and the hidden state at the next moment is output. The fully connected layer maps the output of the LSTM to the time allocation prediction. The Softplus activation layer is used for smooth activation and outputs the time allocation t * .

[0070] Figure 2 It is a schematic structural diagram of a trajectory optimization network structure provided by an embodiment of the present application. Specifically, as Figure 2 shown, the trajectory optimization network structure includes a first feature extraction layer, a second feature extraction layer, an LSTM layer, a time allocation layer, and a solver. The first feature extraction layer and the second feature extraction layer are the outputs of the trajectory optimization network structure. The outputs of the first feature extraction layer and the second feature extraction layer are both connected to the input of the LSTM layer. The output of the LSTM layer is connected to the input of the time allocation layer. The output of the time allocation layer is connected to the input of the solver, and the output of the solver is used as the output of the trajectory optimization network structure. The first feature extraction layer and the second feature extraction layer both include a convolutional layer, a pooling layer, and a fully connected layer connected in sequence.

[0071] The first feature extraction layer is used to extract the first feature vector of the flight corridor sequence and output it to the LSTM layer. The second feature extraction layer is used to extract the second feature vector of the initial state and terminal state of the target UAV and output it to the LSTM layer. The LSTM layer is used to output the hidden state to the time allocation layer based on the first feature vector and the second feature vector. The time allocation layer is used to map the hidden state to the time allocation to output the time spent by the UAV on each segment of the sub-trajectory under the target route. The solver is used to solve the flat output (the functional relationship between the position coordinates and yaw angle of the flat UAV with respect to time) for each segment of the sub-trajectory according to the optimization conditions. The input includes the initial state, terminal state, and flight corridor sequence, and can be expressed as:

[0072]

[0073] Among them, is the initial state, is the termination state, and P i is the flight corridor sequence.

[0074] The convolutional layer is specifically a two-dimensional convolutional layer, which is respectively used for the first feature vector of the flight corridor sequence, and the second feature extraction layer is used to extract the second feature vectors of the initial state and the termination state of the target UAV:

[0075] Con2d(x) = σ(W1 * x + b1);

[0076] Among them, W1 is the convolutional kernel, b1 is the bias, and σ is the activation function.

[0077] The role of the pooling layer is to reduce the size of the features, reduce the complexity and retain the important features:

[0078] MaxPool2d(x) = max(x);

[0079] The fully connected layer is used to extract the previous features and perform a linear mapping to generate a feature vector of a fixed size:

[0080] Linear(x) = W2x + b2;

[0081] Among them, W2 is the weight network of the fully connected layer, and b2 is the bias.

[0082] The LSTM layer is used to process the time series input, and calculates the hidden state h t and the cell state c t at the current moment through recursion, and outputs the hidden state at the next moment:

[0083] i t = sigmoid(W i X t + U i h t-1 + b i );

[0084] f t = sigmoid(W f X t + U f h t-1 + b f );

[0085] o t = sigmoid(W o X t + U o h t-1 + b o );

[0086]

[0087] h t = o t ⊙tanh(c t );

[0088] Among them, W i , W f , W o , W c , U i , U f , U o , U c are weight matrices, b i , b f , b o , b c are bias vectors, i t , f t , o t are the activation values of the update gate, forget gate, and output gate respectively.

[0089] The fully connected layer maps the output of the LSTM to the time allocation prediction:

[0090] t * = W h h t + b h ;

[0091] Among them, W h and b h are the weight matrix and bias vector respectively, h t is the hidden state of the LSTM, and t * is the time spent by the UAV on each path segment, expressed as:

[0092] t * = [△t1, △t2,..., △t M T .

[0093] In an embodiment of the present application, an optimized objective function for ensuring the smoothness of the flight trajectory and minimizing the flight time can be defined to ensure that the UAV is within a safe flight corridor at each time period and avoid collisions with obstacles.

[0094] For each sub-trajectory segment, the solver solves for the flat output at a preset moment in this sub-trajectory segment in the following manner:

[0095] The solver takes the smoothness of the trajectory of the target UAV as the optimization objective and solves the piecewise polynomial of this sub-trajectory segment. Among them, the optimization objective function is expressed as: ​

[0096]

[0097] β(t) = [1, t, t 2 ,...t N T ;

[0098] where σ i (t) is the piecewise polynomial (flat output) of the UAV sub-trajectory i, and σ i (k) (t) is the k-th derivative of σ i (t) (used to measure the smoothness of the trajectory), c i is the coefficient matrix of the UAV sub-trajectory i, β(t) is the polynomial basis function vector, M is the total number of piecewise polynomials, dt is the differential symbol of the time parameter t, N is the order of the piecewise polynomial, and t is the transpose operation symbol;

[0099] Based on this piecewise polynomial, the flat output at a preset moment in this sub-trajectory is determined.

[0100] Specifically, the flat output at a preset moment in this sub-trajectory can be calculated by the following formula:

[0101]

[0102] Specifically, the constraint conditions of the optimization objective function include initial state constraint, terminal state constraint, continuity constraint between each sub-trajectory, safe flight range of each sub-trajectory, and dynamic constraint.

[0103] Among them, the initial state constraint can be expressed as:

[0104]

[0105] The terminal state constraint can be expressed as:

[0106]

[0107] This condition constrains the terminal state of the trajectory at the final moment t.

[0108] The continuity constraint between each sub-trajectory is expressed as:

[0109]

[0110] This condition constrains the continuity between each sub-trajectory, ensuring that at the end of the i-th sub-trajectory, the state of the trajectory is the same as the start state of the (i + 1)-th sub-trajectory, guaranteeing the smooth connection of the trajectory.

[0111] The dynamic constraint can be expressed as:​

[0112]

[0113] d κ represents the maximum value for the k-th derivative dynamic feasibility of the UAV trajectory, which constrains the dynamic range of each order derivative of the trajectory and serves as the dynamic constraint for the UAV.

[0114] The safe flight range of each sub-trajectory is also the safe flight range of the target route (total trajectory). This constraint ensures that the trajectory is within the safe flight corridor at each time period, avoiding collisions with obstacles.

[0115] Among them, G i is composed of a convex polyhedron formed by a series of hyperplanes. For each polyhedron, it can be described in the following form:

[0116]

[0117] G i is the hyperplane normal vector matrix defining the i-th polyhedron, F i is the number of faces of the polyhedron, h i is a distance vector defining the right boundary value of the inequality constraint, which represents the specific position range of the target UAV in the flight corridor at time t. G i and h i together form the flight corridor sequence P i of the i-th flight corridor.

[0118] Therefore, the trajectory planning with given time allocation is constructed as a quadratic programming problem (QP):

[0119] Objective function:

[0120] Constraints: A t c = b;

[0121] G t c ≤ h;

[0122] Among them, c is the coefficient of each trajectory (i.e., the optimization variable in the optimization problem), Q t is a positive semi-definite matrix, directly determined by the time allocation t, and is used to measure the smoothness of the trajectory and the control cost; the equality constraint A t c = b corresponds to the initial state constraint, the terminal state constraint, and the continuity constraint; the inequality constraint G t c ≤ h corresponds to the dynamic constraint and the flight range constraint.

[0123] By constructing a quadratic programming problem, the trajectory smoothness and the total flight time are used as optimization objectives, and at the same time, deep learning optimization is carried out on the time allocation. It is possible to reduce the total flight time while maintaining the trajectory smoothness, achieve efficient utilization of energy, and improve the endurance and task completion efficiency of the unmanned aerial vehicle.

[0124] In an embodiment of the present application, the trajectory optimization network structure further includes an implicit micro layer. The implicit micro layer solves the optimal solution of the trajectory coefficient c in this optimization problem and passes the gradient of this solution back to the previous layer of the neural network to optimize the time allocation t during the training process. * and other network parameters.

[0125] Furthermore, the trajectory optimization network structure can be trained in the following way:

[0126] Input the sample data into the original trajectory optimization network structure to obtain the time allocation result output by the time allocation layer;

[0127] During offline training, the implicit micro layer performs quadratic programming to solve based on the time spent by the unmanned aerial vehicle on each sub-trajectory under the target route output by the time allocation layer and the KKT conditions to obtain a preset coefficient matrix;

[0128] Calculate the gradient of the preset coefficient matrix with respect to the time allocation result;

[0129] Based on the calculated gradient, adjust the time allocation result and the parameters of the trajectory optimization network structure to obtain the trained trajectory optimization network structure. It should be noted that the implicit micro layer takes the time allocation t * as the input, solves the corresponding trajectory optimization as a quadratic programming problem, and for a given time allocation, hopes to minimize a quadratic objective function about the trajectory coefficient c:

[0130]

[0131] The constraint conditions can refer to the constraint conditions of the aforementioned solver optimization problem.

[0132] Here, the optimization problem can be obtained by solving the Karush-Kuhn-Tucker (KKT) conditions. The KKT conditions are sorted into a linear system and can be written in the following form:

[0133]

[0134] where Γ represents the function of the KKT conditions, v * and λ *is the Lagrange multiplier, Q(t) is the quadratic coefficient matrix, which is used to describe the quadratic coefficient of the objective function in the trajectory optimization problem. A(t) is the linear constraint matrix, which is used to describe the boundary conditions and continuity constraints of the UAV trajectory. G(t) is the inequality constraint matrix, which is used to describe the k-th derivative dynamics constraint and safe flight range constraint of the UAV trajectory. b is the constant term vector of the equality constraint, h is the constant term vector of the inequality constraint, diag is the operation symbol for constructing a diagonal matrix, and λ is the Lagrange multiplier related to the inequality constraint.

[0135] Solving the above conditions gives the optimal solution c * . The core of the implicit micro-layer is to calculate the optimal solution c * for the gradient of the input parameter, i.e., the time allocation t. Assume that we have obtained the optimal solution c * , and we need to calculate it to pass this gradient to the neural network.

[0136]

[0137] where

[0138] is the partial derivative of the KKT condition with respect to the time allocation t, including the partial derivatives of Q(t), G i (t), A j (t) with respect to the time allocation t.

[0139] This formula gives the gradient of c * with respect to the time allocation t. Using this gradient, the error can be backpropagated from the trajectory optimization problem to the time allocation layer of the neural network.

[0140] The loss function of the time allocation network here can be expressed as:

[0141]

[0142]

[0143] where, are the loss functions representing the trajectory performance, energy consumption, and flight time respectively, and ω traj , ω time_allocation are the corresponding weight coefficients.

[0144] where, can be calculated by the following formula:

[0145] where, Q t is the smoothness matrix of the trajectory, which is determined by the time allocation t * . Optimizing the acceleration directly determines the power demand of the UAV and indirectly achieves the purpose of energy consumption optimization.

[0146] The trajectory optimization gradient can be calculated by the following formula

[0147]

[0148] where has been obtained from the implicit differential layer

[0149] Neural network gradient can be calculated through the following steps

[0150] Use PyTorch to calculate the gradient of the time allocation network

[0151]

[0152] Calculate the total gradient

[0153] Multiply the gradients chain by chain to obtain the total gradient

[0154]

[0155] Update the neural network parameter θ

[0156]

[0157] where η is the learning rate

[0158] Figure 3 This is a schematic structural diagram of a time allocation device for an unmanned aerial vehicle (UAV) trajectory provided by an embodiment of the present application. Based on the same inventive concept, an embodiment of the present application also provides a time allocation device for an unmanned aerial vehicle trajectory. The device 30 includes

[0159] A prediction module 310, configured to input the initial state, the termination state, and the flight corridor sequence of the target UAV into a pre-trained trajectory optimization network structure to obtain the trajectory result of the target UAV output by the trajectory optimization network structure

[0160] A control module 320, configured to generate a corresponding control instruction based on the trajectory result and send it to the target UAV to control the target UAV to fly along the target route

[0161] where the initial state is used to indicate the position coordinates, speed value, pitch angle value, roll angle value, and yaw angular velocity value of the UAV at the initial moment, the termination state is used to indicate the position coordinates, speed value, pitch angle value, roll angle value, and yaw angular velocity value of the UAV at the termination moment, and the flight corridor sequence is used to indicate the safe flight range of the UAV

[0162] The trajectory result is used to indicate the time spent by the drone on each sub-trajectory under the target route, as well as the position coordinates, speed value, pitch angle value, roll angle value, and yaw angular velocity value at a preset moment in each sub-trajectory;

[0163] The target route is the route between the position coordinates at the initial moment and the position coordinates at the termination moment.

[0164] In an alternative embodiment, the trajectory optimization network structure includes a first feature extraction layer, a second feature extraction layer, an LSTM layer, a time allocation layer, and a solver. The first feature extraction layer is used to extract the first feature vector of the flight corridor sequence and output it to the LSTM layer. The second feature extraction layer is used to extract the second feature vectors of the initial state and the termination state of the target drone and output them to the LSTM layer. The LSTM layer is used to output the hidden state to the time allocation layer based on the first feature vector and the second feature vector; the time allocation layer is used to map the hidden state to the time allocation to output the time spent by the drone on each sub-trajectory under the target route; the solver is used to solve the position coordinates, speed value, pitch angle value, roll angle value, and yaw angular velocity value at a preset moment in each sub-trajectory according to the optimization conditions.

[0165] In an alternative embodiment, for each sub-trajectory, the solver solves the position coordinates, speed value, pitch angle value, roll angle value, and yaw angular velocity value at a preset moment in the sub-trajectory in the following manner:

[0166] The solver takes the trajectory smoothness of the target drone as the optimization goal and solves the piecewise polynomial of the sub-trajectory. Among them, the optimization objective function is expressed as:

[0167]

[0168] β(t) = [1, t, t 2 ,...t N T ;

[0169] Among them, σ i (t) is the piecewise polynomial of the drone sub-trajectory i, ω t is the weight coefficient of the time cost, f(t) is the cost related to the total flight time of the sub-trajectory, Δt k represents the time spent on the k-th segment of the trajectory, c i is the coefficient matrix of the drone sub-trajectory i, and β(t) is the polynomial basis function vector;

[0170] Based on this piecewise polynomial, the position coordinates, speed value, pitch angle value, roll angle value, and yaw angular velocity value at a preset moment in the sub-trajectory are determined.

[0171] ​In an alternative embodiment, the constraint conditions for optimizing the objective function include initial state constraints, terminal state constraints, continuity constraints between each sub-trajectory, the safe flight range of each sub-trajectory, and dynamic constraints.

[0172] In an alternative embodiment, the trajectory optimization network structure further includes an implicit differentiation layer, and the trajectory optimization network structure is trained and generated in the following manner:

[0173] Input the sample data into the original trajectory optimization network structure to obtain the time allocation result output by the time allocation layer;

[0174] The implicit differentiation layer performs a solution based on the time spent by the UAV on each sub-trajectory under the target route output by the time allocation layer and the KKT conditions to obtain a preset coefficient matrix;

[0175] Calculate the gradient of the preset coefficient matrix with respect to the time allocation result;

[0176] Based on the calculated gradient, adjust the time allocation result and the parameters of the trajectory optimization network structure to obtain the trained trajectory optimization network structure.

[0177] In an alternative embodiment, the KKT conditions are expressed as:

[0178]

[0179] where c * is the preset coefficient matrix, v * and λ * are Lagrange multipliers.

[0180] In an alternative embodiment, both the first feature extraction layer and the second feature extraction layer include a convolutional layer, a pooling layer, and a fully connected layer connected in sequence.

[0181] Please refer to Figure 4 , Figure 4 which is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As shown in Figure 4 , the electronic device 400 includes a processor 410, a memory 420, and a bus 430.

[0182] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 runs, the processor 410 communicates with the memory 420 through the bus 430. When the machine-readable instructions are executed by the processor 410, the steps of a method for time allocation of a UAV trajectory in the above method embodiment can be executed. The specific implementation manner can refer to the method embodiment and will not be elaborated here.

[0183] The embodiments of the present application also provide a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it can execute the steps of a method for time allocation of an unmanned aerial vehicle trajectory in the above method embodiments. For the specific implementation manner, reference can be made to the method embodiments and will not be elaborated here.

[0184] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated here.

[0185] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there can be other division methods in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces. The indirect coupling or communication connection of the devices or units can be in an electrical, mechanical, or other form.

[0186] In addition, the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0187] Furthermore, in each embodiment of the present application, the functional modules can be integrated together to form an independent part, or each module can exist alone, or two or more modules can be integrated to form an independent part.

[0188] It should be noted that if a function is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or a part of this technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0189] In this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0190] The above are only embodiments of the present application and are not used to limit the protection scope of the present application. For those skilled in the art, the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for allocating the trajectory time of an unmanned aerial vehicle, characterized in that, The method includes: Inputting the initial state, termination state, and flight corridor sequence of the target unmanned aerial vehicle (UAV) into a pre-trained trajectory optimization network structure to obtain the trajectory result of the target UAV output by the trajectory optimization network structure; Generating corresponding control instructions based on the trajectory result and sending them to the target UAV to control the target UAV to fly along the target route; Wherein, the initial state is used to indicate the position coordinates, speed value, pitch angle value, roll angle value, and yaw angular velocity value of the UAV at the initial moment, the termination state is used to indicate the position coordinates, speed value, pitch angle value, roll angle value, and yaw angular velocity value of the UAV at the termination moment, and the flight corridor sequence is used to indicate the safe flight range of the UAV, The trajectory result is used to indicate the time spent on each sub-trajectory of the UAV along the target route, as well as the position coordinates, speed value, pitch angle value, roll angle value, and yaw angular velocity value at a preset moment in each sub-trajectory; The target route is the route between the position coordinates at the initial moment and the position coordinates at the termination moment.

2. The method according to claim 1, characterized in that, The trajectory optimization network structure includes a first feature extraction layer, a second feature extraction layer, an LSTM layer, a time allocation layer, and a solver, The first feature extraction layer is used to extract the first feature vector of the flight corridor sequence and output it to the LSTM layer, The second feature extraction layer is used to extract the second feature vectors of the initial state and termination state of the target UAV and output them to the LSTM layer, The LSTM layer is used to output a hidden state to the time allocation layer based on the first feature vector and the second feature vector; The time allocation layer is used to map the hidden state to time allocation to output the time spent on each sub-trajectory of the UAV along the target route; The solver is used to solve the flat output of each sub-trajectory according to the optimization conditions for each sub-trajectory.

3. The method according to claim 2, characterized in that, For each sub-trajectory, the solver solves for the position coordinates, speed value, pitch angle value, roll angle value, and yaw angular velocity value at a preset moment in this sub-trajectory in the following manner: The solver takes the trajectory smoothness and flight time of the target UAV as the optimization objective and solves the piecewise polynomial of this sub-trajectory, where the optimization objective function is expressed as: β(t) = [1, t, t 2 ,... t N T ;​ where, σ i (t) is the flat output of the UAV sub-trajectory i, σ i (k) (t) is the k-th derivative of σ i (t), Δt k represents the time taken for the k-th segment of the trajectory, c i is the coefficient matrix of the UAV sub-trajectory i, β(t) is the polynomial basis function vector, M is the total number of piecewise polynomials, dt is the differential symbol of the time parameter t, N is the order of the piecewise polynomial, and T is the transpose operation symbol; Based on the flat output of the UAV sub-trajectory i, determine the position coordinates, speed value, pitch angle value, roll angle value, and yaw angular velocity value at a preset moment in this sub-trajectory.

4. The method according to claim 3, wherein The constraint conditions of the optimization objective function include initial state constraint, termination state constraint, continuity constraint between each sub-trajectory, safe flight range of each sub-trajectory, and dynamic constraint.

5. The method according to claim 2, characterized in that, The trajectory optimization network structure further includes an implicit differentiation layer, and the trajectory optimization network structure is trained and generated in the following manner: Inputting sample data into the original trajectory optimization network structure to obtain the time allocation result output by the time allocation layer; The implicit differentiation layer performs a solution based on the time spent on each sub-trajectory of the UAV along the target route output by the time allocation layer and the KKT condition to obtain a preset coefficient matrix; Calculate the gradient of the preset coefficient matrix with respect to the time allocation result; Based on the calculated gradient, adjust the time allocation result and the parameters of the trajectory optimization network structure to obtain the trained trajectory optimization network structure.

6. The method according to claim 5, wherein The objective function of the KKT condition is: Among them, Γ represents the function of the KKT conditions, c * is a preset coefficient matrix, v * and λ * are Lagrange multipliers, b is the constant term vector of the equality constraint, h is the constant term vector of the inequality constraint, diag is the operation symbol for constructing a diagonal matrix, λ is the Lagrange multiplier related to the inequality constraint, Q(t) is the quadratic term coefficient matrix, A(t) is the linear constraint matrix, and G(t) is the inequality constraint matrix.

7. The method according to claim 2, characterized in that, Both the first feature extraction layer and the second feature extraction layer include a convolutional layer, a pooling layer, and a fully connected layer connected in sequence.

8. A time allocation device for an unmanned aerial vehicle trajectory, characterized in that, The device includes: A prediction module, configured to input the initial state, the termination state, and the flight corridor sequence of the target unmanned aerial vehicle into a pre-trained trajectory optimization network structure to obtain the trajectory result of the target unmanned aerial vehicle output by the trajectory optimization network structure; A control module, configured to generate a corresponding control instruction based on the trajectory result and send it to the target unmanned aerial vehicle to control the target unmanned aerial vehicle to fly along the target route; Wherein, the initial state is used to indicate the position coordinates, speed value, pitch angle value, roll angle value, and yaw angular velocity value of the unmanned aerial vehicle at the initial moment, the termination state is used to indicate the position coordinates, speed value, pitch angle value, roll angle value, and yaw angular velocity value of the unmanned aerial vehicle at the termination moment, and the flight corridor sequence is used to indicate the safe flight range of the unmanned aerial vehicle, The trajectory result is used to indicate the time spent by the unmanned aerial vehicle on each sub-trajectory along the target route, as well as the position coordinates, speed value, pitch angle value, roll angle value, and yaw angular velocity value at a preset moment in each sub-trajectory; The target route is the route between the position coordinates at the initial moment and the position coordinates at the termination moment.

9. An electronic device, characterized in that, Includes: A processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. The processor executes the machine-readable instructions to perform the steps of the method for time allocation of the unmanned aerial vehicle trajectory according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium. When the computer program is run by the processor, it performs the steps of the method for time allocation of the unmanned aerial vehicle trajectory according to any one of claims 1 to 7.

Images