A trajectory planning method, device, equipment and medium of a variant near space vehicle

Abstract

The application provides a trajectory planning method, device, equipment and medium of a variant near space vehicle, and relates to the field of aerospace guidance control. The application first constructs a three-dimensional center of mass motion model of the variant near space vehicle; then uses an initial value generator to obtain a suboptimal initial trajectory of the variant near space vehicle according to the three-dimensional center of mass motion model; and then uses an improved Newton trajectory planning method to plan the trajectory based on the suboptimal initial trajectory to obtain a trajectory planning result. The initial value generator designed by the application improves the problem that the initial value sensitive trajectory does not converge and directly diverges in the existing trajectory planning method, realizes the generation of a good initial value under the condition that the on-board computing resources are limited, and improves the convergence speed. The application adjusts the damping (step size) dynamically, ensures the robustness of the iteration process and improves the applicability of the method, which is conducive to the fast time-varying, nonlinear and complex battlefield environment.

Description

Technical Field

[0001] This application relates to the field of aerospace guidance and control, and in particular to a trajectory planning method, apparatus, equipment and medium for a variant near-space vehicle. Background Technology

[0002] To meet the needs of future highly dynamic and time-varying warfare, the flight missions of variant near-space vehicles (NSVs) have become more complex. Existing NSVs only have reaching the target position as their sole constraint; however, in future highly contested and complex combat scenarios, vehicles must simultaneously fulfill terminal position, velocity, and angle constraints. Therefore, these additional complex constraints place higher demands on the vehicle's guidance system. Variant NSVs typically achieve precise control of six terminal states (position, velocity, track angle, and heading angle) through normal and lateral overload, representing a typical underactuated guidance problem. The optimal control problem is generally solved using trajectory planning methods, but existing methods have the following drawbacks: 1) Trajectory planning methods generally have a small convergence radius, and sensitivity to initial values ​​may lead to non-convergence or even divergence during iteration. 2) Existing model-based static programming methods converge slowly during iteration. Summary of the Invention

[0003] The purpose of this application is to provide a trajectory planning method, apparatus, device, and medium for a variant near-space vehicle, so as to improve the convergence speed in the trajectory planning process of the variant near-space vehicle.

[0004] To achieve the above objectives, this application provides the following solution.

[0005] In a first aspect, this application provides a trajectory planning method for a variant of a near-space vehicle, including:

[0006] Construct a three-dimensional center-of-mass motion model for a variant near-space vehicle;

[0007] The suboptimal initial trajectory of the variant near-space vehicle is obtained by using an initial value generator based on the three-dimensional centroid motion model; the initial value generator is obtained through reinforcement learning of a two-layer deep Q-network.

[0008] Based on the suboptimal initial trajectory, an improved Newton-like trajectory planning method is used to plan the trajectory and obtain the trajectory planning result. The improved Newton-like trajectory planning method is obtained by introducing a damping coefficient based on adaptive law dynamic adjustment into the Newton-like trajectory planning method.

[0009] Optionally, the three-dimensional centroid motion model is:

[0010] ;

[0011] in, Indicates the speed of the variant near-space vehicle; Indicates the track angle; Indicates the heading angle; Indicates the distance traveled; Indicates altitude; Indicates lateral position; Indicates axial overload; and These represent normal overload and lateral overload in the vertical velocity direction, respectively; It represents the acceleration due to gravity.

[0012] Optionally, based on the three-dimensional centroid motion model, an initial value generator is used to obtain the suboptimal initial trajectory of the variant near-space spacecraft, specifically including:

[0013] Two optimization parameters are generated based on the initial value generator;

[0014] Based on two optimized parameters, the guidance law for each time step is obtained using the following formula;

[0015] ;

[0016] ;

[0017] in, For guidance laws based on altitude, For lateral position guidance law, and All are optimized parameters. Indicates the speed of the variant near-space vehicle. Indicates the track angle, It represents the acceleration due to gravity.

[0018] The suboptimal initial trajectory is generated based on the guidance law at each time step.

[0019] Optionally, the reward function used in obtaining the initial value generator through reinforcement learning on a two-layer deep Q-network is:

[0020] ;

[0021] in, For the reward function, The position term penalty coefficient, This is the penalty coefficient for the speed term. For the track angle penalty coefficient, This is the heading angle penalty coefficient. The guidance law penalty coefficient, This is the cumulative speed penalty coefficient. To accumulate the track angle penalty coefficient, To accumulate the heading angle penalty factor, , , These are the terminal time points. The range, altitude, and lateral position, , , These are the terminal time points. The expected range, expected altitude, and expected lateral position. Terminal time point speed, Terminal time point track angle, Terminal time point The heading angle, For nodes Guidance law based on altitude, For nodes Lateral guidance law, For nodes speed, Terminal time point Expected speed, For nodes track angle, Terminal time point Expected track angle, For nodes The heading angle, Terminal time point The expected heading angle.

[0022] Optionally, based on the suboptimal initial trajectory, an improved Newton-like trajectory planning method is used for trajectory planning, and the formula for obtaining the trajectory planning result is:

[0023] ;

[0024] in, Indicates the damping coefficient. This indicates the updated control history. This represents the control history in the suboptimal initial trajectory; This indicates the history of deviation control.

[0025] Optionally, the damping coefficient is obtained through a trained neural network model.

[0026] Optionally, both the suboptimal initial trajectory and the trajectory planning result satisfy the following constraints:

[0027] ;

[0028] in, For the angle of attack, and These are the minimum and maximum angles of attack, respectively. The tilt angle, and These are the minimum and maximum values ​​of the tilt angle, respectively. The deformation ratio, and These are the minimum and maximum values ​​of the deformation ratio, respectively. , and All of these are actual control quantities, derived from the guidance laws for altitude and lateral positions.

[0029] Secondly, this application provides a trajectory planning device for a variant near-space vehicle, which applies the aforementioned trajectory planning method for variant near-space vehicles. The trajectory planning device for the variant near-space vehicle includes:

[0030] The 3D center-of-mass motion model construction module is used to construct a 3D center-of-mass motion model for a variant near-space vehicle.

[0031] The suboptimal initial trajectory generation module is used to obtain the suboptimal initial trajectory of the variant near-space vehicle by using an initial value generator based on the three-dimensional centroid motion model; the initial value generator is obtained by reinforcement learning of a two-layer deep Q-network.

[0032] The trajectory planning module is used to perform trajectory planning based on the suboptimal initial trajectory using an improved Newton-like trajectory planning method to obtain the trajectory planning result. The improved Newton-like trajectory planning method is obtained by introducing a damping coefficient based on adaptive law dynamic adjustment into the Newton-like trajectory planning method.

[0033] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described variant near-space vehicle trajectory planning method.

[0034] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described variant near-space vehicle trajectory planning method.

[0035] According to the specific embodiments provided in this application, this application has the following technical effects.

[0036] This application provides a trajectory planning method, apparatus, device, and medium for a variant near-space vehicle. First, a three-dimensional centroid motion model of the variant near-space vehicle is constructed. Then, based on the three-dimensional centroid motion model, an initial value generator is used to obtain a suboptimal initial trajectory for the variant near-space vehicle. Next, based on the suboptimal initial trajectory, an improved Newton-like trajectory planning method is used to perform trajectory planning, obtaining the trajectory planning result. The initial value generator designed in this application improves upon the problems of non-convergence and direct divergence of initial value-sensitive trajectories in existing trajectory planning methods, achieving the generation of good initial values ​​under limited onboard computing resources and improving the convergence speed. This application ensures the robustness of the iteration process and improves the applicability of the method by dynamically adjusting the damping (step size), which is beneficial for complex battlefield environments with fast time-varying and nonlinear characteristics. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a flowchart illustrating a variant of the trajectory planning method for a near-space vehicle provided in one embodiment of this application.

[0039] Figure 2 This is a schematic diagram illustrating the principle of a trajectory planning method for a variant near-space vehicle provided in one embodiment of this application.

[0040] Figure 3 A trajectory coordinate system diagram provided for an embodiment of this application.

[0041] Figure 4 This is a diagram illustrating the neural network design structure provided in one embodiment of this application.

[0042] Figure 5 A simulation diagram of the trajectory curve provided in one embodiment of this application.

[0043] Figure 6 A simulation diagram of the time-normal overload curve provided for an embodiment of this application.

[0044] Figure 7 A simulation diagram of the time-lateral overload curve provided for an embodiment of this application.

[0045] Figure 8 A simulation diagram of the time-velocity curve provided for an embodiment of this application.

[0046] Figure 9A simulation diagram of the time-track angle curve provided for an embodiment of this application.

[0047] Figure 10 A simulation diagram of the time-heading angle curve provided for an embodiment of this application.

[0048] Figure 11 A simulation graph of the time-performance index curve provided for an embodiment of this application.

[0049] Figure 12 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0050] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0051] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] In one exemplary embodiment, such as Figure 1 and Figure 2 As shown, this application embodiment provides a trajectory planning method for a variant near-space vehicle, including the following steps 101-103.

[0053] Step 101: Construct a three-dimensional center-of-mass motion model of the variant near-space spacecraft.

[0054] Step 102: Based on the three-dimensional centroid motion model, an initial value generator is used to obtain the suboptimal initial trajectory of the variant near-space spacecraft; the initial value generator is obtained by reinforcement learning on a two-layer deep Q-network.

[0055] Step 103: Based on the suboptimal initial trajectory, an improved Newton-like trajectory planning method is used to perform trajectory planning and obtain the trajectory planning result; the improved Newton-like trajectory planning method is obtained by introducing a damping coefficient based on adaptive law dynamic adjustment into the Newton-like trajectory planning method.

[0056] Implementing steps 101-103 above can improve the problems of initial value-sensitive trajectories failing to converge and directly diverging in existing trajectory planning methods, enabling the generation of good initial values ​​under conditions of limited onboard computing resources and improving convergence speed. Simultaneously, it ensures the robustness of the iterative process and enhances the applicability of the method, making it suitable for complex battlefield environments that are rapidly time-varying and nonlinear.

[0057] In another exemplary embodiment, step 101 above constructs a three-dimensional centroid motion model for a variant near-space vehicle, providing a motion model basis for the initial value generator in step 102 to generate a suboptimal initial trajectory and for the adaptive static programming method in step 103.

[0058] The specific implementation method of step 101 above is as follows: To study the trajectory change of the variant near-space vehicle, it is only necessary to study the three-degree-of-freedom center of mass model, without considering attitude motion. Therefore, the motion model only needs to study the center of mass dynamics model and kinematic model. Among them, (1) the influence of airflow speed on actual flight is not considered; (2) gravitational acceleration g It can be assumed to be a constant value; (3) the Earth's rotation is not considered.

[0059] To construct a three-dimensional model of the center of mass motion, the first step is to define a coordinate system. The trajectory coordinate system of a variant near-space vehicle is as follows: Figure 3 As shown. Wherein: The axis points north, indicating the distance traveled. The axis points upwards, indicating height. The axis points east according to the right-hand screw rule, indicating the lateral position; Indicates the speed of the variant near-space vehicle; and These represent the track angle and heading angle, respectively.

[0060] Therefore, the motion model equations can be constructed as follows:

[0061] (1)

[0062] in, Indicates the speed of the variant near-space vehicle; Indicates the track angle; Indicates the heading angle; Indicates the distance traveled; Indicates altitude; Indicates lateral position; Indicates axial overload; and These represent normal overload and lateral overload in the vertical velocity direction, respectively; It represents the acceleration due to gravity.

[0063] The first three lines of equation (1) represent the three-degree-of-freedom center-of-mass dynamic equations, and the last three lines represent the three-degree-of-freedom center-of-mass kinematic equations. The state vector is expressed as follows: The virtual control vector is expressed as It is important to note that near-space vehicles have variable configurations, therefore, virtual control variables... and The actual control quantity angle of attack can be derived. Tilt angle With deformation ratio .

[0064] In another exemplary embodiment, constraints need to be placed on the actual control quantities described above. To ensure the effectiveness of the research and to make the guidance commands generated by the trajectory planning method conform to the characteristics of a real maneuvering model, factors such as the internal structural strength of the near-space vehicle, servo characteristics, and normal operating conditions of sensors need to be considered during the design phase. Therefore, constraints need to be placed on the control quantities:

[0065] (2)

[0066] in, and These represent the minimum and maximum usable angle of attack, respectively. and These represent the minimum and maximum available tilt angles, respectively. and These represent the minimum and maximum values ​​of the deformation ratio, respectively. Considering the physical limits that near-space vehicles can withstand, we can assume: , , , , , .

[0067] In another exemplary embodiment, step 102 above constructs a reinforcement learning initial value generator based on a two-layer deep Q-network, which quickly provides a suboptimal trajectory that approximately satisfies some of the constraints for the trajectory planning method in the subsequent step 103, making the iterative process of the trajectory planning method more robust.

[0068] The specific implementation method of step 102 above is as follows: In the trajectory planning problem, the choice of initial value determines whether the trajectory can converge quickly. Traditionally, the initial guess for controlling the history is selected based on a linear approximation of the initial and terminal guess values. Due to the small convergence radius, traditional methods will lead to the problem of the trajectory not converging and directly diverging. In order to obtain better initial values ​​in the trajectory planning problem, an initial value generator based on reinforcement learning is introduced to generate a suboptimal trajectory that approximately satisfies the initial and terminal constraints.

[0069] Based on the variable coefficient proportional guidance law, a reinforcement learning initial value generator is proposed to ensure rapid trajectory convergence during iteration. The guidance law is expressed as follows: the elevation angle and its derivative are represented as:

[0070] (3)

[0071] (4)

[0072] in, Indicates the angle of elevation; The derivative of the angle of elevation; subscript Indicates the terminal time point; , , These are the terminal time points. The range, altitude, and lateral position.

[0073] The proportionality coefficient is expressed as:

[0074] (5)

[0075] in, Represents the proportionality coefficient. Terminal time point The track angle.

[0076] The track angle can be expressed as:

[0077] (6)

[0078] in, The derivative of the track angle; The derivative of the angle of elevation.

[0079] Using equation (1), the normal overload formula can be constructed as follows:

[0080] (7)

[0081] in, For guidance laws based on altitude, It represents the acceleration due to gravity.

[0082] It should be noted that the above derivation achieves precise control of the terminal's longitudinal position and track angle, while the lateral position is controlled by the traditional proportional guidance law:

[0083] (8)

[0084] in, For lateral position guidance law, Indicates the heading angle. This represents the derivative of the heading angle.

[0085] Heading angle The derivative is expressed as:

[0086] (9)

[0087] in, It is a positive number greater than 0.

[0088] The trajectory obtained by the above method cannot satisfy the terminal velocity and heading angle constraints. The purpose of introducing reinforcement learning is to establish a suboptimal initial trajectory that approximately satisfies the constraints, so as to avoid the initial value divergence problem in subsequent iterations.

[0089] The guidance laws in equations (7) and (8) are modified as follows:

[0090] (10)

[0091] (11)

[0092] in, and All of these are optimized parameters.

[0093] train The purpose is to adjust the terminal velocity and track angle. However, due to the coupling relationship, the above process leads to a loss of terminal state accuracy; therefore, a correction factor is introduced. As an adjustment parameter.

[0094] The reward function is designed considering terminal constraints and trajectory smoothness. Reward Function R 3 is set as:

[0095] (12)

[0096] Among them, subscript This represents the desired terminal state value. Indicates the penalty coefficient. For the reward function, The position term penalty coefficient, This is the penalty coefficient for the speed term. For the track angle penalty coefficient, This is the heading angle penalty coefficient. The guidance law penalty coefficient, This is the cumulative speed penalty coefficient. To accumulate the track angle penalty coefficient, To accumulate the heading angle penalty factor, , , These are the terminal time points. The range, altitude, and lateral position, , , These are the terminal time points. The expected range, expected altitude, and expected lateral position. Terminal time point speed, Terminal time point track angle, Terminal time point The heading angle, For nodes Guidance law based on altitude, For nodes Lateral guidance law, For nodes speed, Terminal time point Expected speed, For nodes track angle, Terminal time point Expected track angle, For nodes The heading angle, Terminal time point The expected heading angle.

[0097] In reinforcement learning initial value generators, Q-learning algorithms are used as an iterative method to update the action value function, while two-layer deep Q-networks (Dueling DQN) are used to transform the action value function from a discrete form to a continuous form and to model the state value function and reward function separately.

[0098] The iterative equation in the Q-learning algorithm can be expressed as:

[0099] (13)

[0100] in, A function representing an action; Indicates state; Indicates the action (i.e., the control quantity); Indicates the learning rate; This represents the discount rate parameter; This represents the reward resulting from the current state transition; Subscript represents the maximum value function. Indicates the current time, subscript Indicates the next time step after the current time.

[0101] Deep networks are introduced to transform state variables from discrete to continuous forms. Then, to better handle states with low correlation to actions in variant near-space vehicles, a two-layer DQN architecture is introduced to model the state value function and reward function separately, enabling better learning of the state value function. In the two-layer DQN, The network is modeled as follows:

[0102] (14)

[0103] Subscript Indicates the parameters in the state network; subscript Indicates parameters in the reward network; subscript This represents parameters that are simultaneously related to both networks; State value function:

[0104] (15)

[0105] in, It represents the expected value of a random variable.

[0106] Value function:

[0107] (16)

[0108] The only difference between Dueling DQN and DQN lies in their network architecture. The advantage of Dueling DQN is that it... The relevant network connection parameters can remain unaffected by any actions. It learns gradually due to the influence of [the system / mechanism]. Therefore, it requires fewer learning iterations than DQN, and this advantage becomes more pronounced as the number of action choices increases.

[0109] It should be noted that the above-described reinforcement learning initialization generator method may not be able to obtain an optimal flight trajectory that precisely satisfies the initial and terminal constraints. The purpose of using an initialization generator is not to create a completely accurate and optimal trajectory, but to ensure that subsequent static programming methods do not diverge directly during iteration.

[0110] In another exemplary embodiment, step 103 above constructs an adaptive static planning method based on a neural network, which improves the problems of poor robustness of the initial large step-size iteration and low terminal state control accuracy of the existing method, thus ensuring the task objective of high-precision and fast trajectory planning for mid-range computational guidance.

[0111] The specific implementation method of S3 is as follows: The general form of the dynamic equation is defined as:

[0112] (17)

[0113] in, and These represent the state and control vectors, respectively. The derivative of the state vector; Represents the dynamic constraint vector; Represents the current time.

[0114] Based on the linear superposition theory in optimal control, the deviation dynamics equation is expressed as:

[0115] (18)

[0116] in, This represents taking the derivative of the state vector; and These represent the state matrix and the control partial derivative matrix, respectively. Represents the differential of the state vector; Represents the derivative of the control vector; It represents the differential of the derivative of the state vector.

[0117] The model predictive static programming method employs the small-step Euler discretization method. To improve computational efficiency and reduce optimization time, a large-step flipped Radau pseudospectral discretization method is used for equation (18). Therefore, the time interval under the original continuous optimal control problem is reduced from... Transform to The conversion formula is:

[0118] (19)

[0119] in, and These represent the initial and final time points, respectively. This represents the standardized time in the pseudospectral method.

[0120] The collocation points in the inverted Radau pseudospectral method are Legendre-Gauss-Radau (LGR) points, ranging from... . The LGR point of order is a polynomial The root of the function. Expressed as:

[0121] (20)

[0122] in, Indicates the number of matching points; the symbol " " represents differentiation; This represents the standardized time in the pseudospectral method.

[0123] To ensure that the nodes cover the endpoints of the interval, the first node in the flipped Radau pseudospectrum is the initial time point. The last node is the terminal time point. . Represents the number of nodes. Used The first-order Lagrange interpolation polynomial is used as the basis function to approximate the state vector:

[0124] (twenty one)

[0125] in: express The state vector at time, Represents the approximate state vector after discretization; Denotes the interpolation basis function. Represents a node Standardized time.

[0126] Based on the weighted residual rule, the dynamic constraints in the continuous time frame are transformed into numerical algebraic constraints:

[0127] (twenty two)

[0128] in: Represents an algebraic constraint vector. ; Represents the elements in the differential matrix. ; Represents a node Standardized time, where nodes For nodes The matching points, Represents a node The state vector at time, Represents a node The state vector at time, Represents a node The control vector at that time.

[0129] Based on equations (18) and (22), the algebraic constraints are transformed into:

[0130] (twenty three)

[0131] in, It is the identity matrix. , , , Representing nodes 1, 2, and 3 respectively. , The state matrix at time, , , , Representing nodes 1, 2, and 3 respectively. , The control partial derivative matrix at time, symbol " "" indicates differentiation.

[0132] To perform the inverse operation, equation (23) is modified as follows:

[0133] (twenty four)

[0134] Among them, in formula (24) , , , Corresponding to formula (23) , , , In formula (24) , , Corresponding to formula (23) , , In formula (24) , , Corresponding to formula (23) , , In formula (24) , , Corresponding to formula (23) , , .

[0135] Thus, the original dynamic constraints in equation (18) are transformed into the algebraic constraints in equation (24).

[0136] Based on the discretization of the flipped Radau pseudospectrum, an improved model prediction static programming method is proposed to handle initial and fixed terminal state constraints. The output (observation) equation is expressed as:

[0137] (25)

[0138] in, Represents the observation matrix. This represents the output state equation.

[0139] The state transition equation in equation (24) is established as follows:

[0140] (26)

[0141] in, and All are elements in the first matrix on the right side of equation (24).

[0142] Taking the differential of equation (25):

[0143] (27)

[0144] if k =K =f:

[0145] (28)

[0146] in, This indicates the error in the terminal output status.

[0147] According to equations (26) to (28), we can obtain:

[0148] (29)

[0149] Initial state perturbation in trajectory planning problems is typically... It can be set to 0:

[0150] (30)

[0151] Discrete performance indicators are designed as follows:

[0152] (31)

[0153] in, This indicates the inversion of the integral weights in the Radau pseudospectrum; Represents a function related to the state; Representation function The penalty coefficient; Represents the performance index after discretization; This represents the control penalty weight matrix.

[0154] To obtain the optimal solution through multiple iterations of the sequence, performance metrics It can be rewritten as:

[0155] (32)

[0156] in, Represents a node Control vector at time; symbol " "" indicates differentiation.

[0157] Based on the Lagrange multipliers, the constrained optimization problem is transformed into an unconstrained optimization problem through equations (30) and (32):

[0158] (33)

[0159] in, Indicates augmented performance metrics; Represents the Lagrange multipliers; This indicates the error in the terminal output status.

[0160] According to optimal control theory, the first-order necessary condition is expressed as:

[0161] (34)

[0162] in, The process vector is represented as:

[0163] (35)

[0164] After the iteration stabilizes, For function The partial derivative function with respect to the state vector, Approximately equal to a function Therefore, equation (35) can be transformed into:

[0165] (36)

[0166] in,

[0167] (37)

[0168] Using equation (34), the deviation control history can be expressed as:

[0169] (38)

[0170] Based on equations (29) and (38), we can obtain:

[0171] (39)

[0172] Through equation (39), the Lagrange multipliers can be expressed as:

[0173] (40)

[0174] By combining equations (38) and (40), the deviation control history is expressed as:

[0175] (41)

[0176] Finally, combining equation (41), the iterative update law is expressed as:

[0177] (42)

[0178] in, This indicates the updated control history.

[0179] To match the discretization of the flipped Radau pseudospectrum, the approximate fast integral equation is expressed as:

[0180] (43)

[0181] in, This represents the dynamic constraint vector.

[0182] The trajectory planning method described above can be proven to be a Newton-like method. To simplify the proof, we will not consider state-related terms here, but only prove... The simplified form. Under the premise of satisfying the initial and final state constraints, the proposed method also needs to construct the following constraints according to the requirements of Newton-type methods. By combining equation (26), the constraint vector can be obtained as:

[0183] (44)

[0184] in, express The constraint vector, This represents the extended control vector. , This is the partial derivative matrix of the dynamic equations with respect to the control quantity in the reference trajectory. Represents the terminal state vector. This represents the desired terminal state vector.

[0185] Based on the performance indicators, equation (44) can be rewritten as:

[0186] (45)

[0187] in, This represents the extended control vector in the reference trajectory. express The constraint vector, express The derivative, Represents the extended control weight matrix:

[0188] (46)

[0189] in, Representation matrix The first element on the middle diagonal; Representation matrix The second element on the diagonal.

[0190] Expressed as:

[0191] (47)

[0192] This can be expressed as:

[0193] (48)

[0194] Using equation (45), the control vector for each discrete point can be expressed as:

[0195] (49)

[0196] Finally, equation (49) can be rewritten as:

[0197] (50)

[0198] The above demonstrates that the proposed static planning method is an improved Newton-type trajectory planning method. Newton-type methods suffer from poor robustness in initial large-step iterations and low accuracy in terminal state control in subsequent small-step iterations due to their small convergence radius. Therefore, the damping coefficient needs to be dynamically adjusted using an adaptive law. Equation (42) can be rewritten as:

[0199] (51)

[0200] in, This represents the damping coefficient, ranging from... , This indicates the updated control history. This represents the control history in the suboptimal initial trajectory; This represents the history of deviation control. Based on training data, damping is dynamically adjusted using a multi-layer adaptive neural network technique with structural adjustment capabilities: First, by defining a neural network model with three hidden layers, the training and prediction of the dynamic adjustment of the damping coefficient are achieved, such as... Figure 4 As shown;

[0201] Then, during training, the cross-entropy loss function and the Adam optimizer are used to update the model parameters. The input and output of the neural network model are expressed as follows:

[0202] (52)

[0203] in, This represents the training dataset; Indicates the number of iterations.

[0204] Finally, based on the trained model, dynamic damping is adjusted according to equation (52).

[0205] In another exemplary embodiment, an example is provided to illustrate the technical effects of the various method embodiments described above.

[0206] To demonstrate the effectiveness and feasibility of the adaptive static programming method for a variant near-space vehicle based on a reinforcement learning initialization generator, this application conducts simulation verification for mid-course guidance of a near-space vehicle. The initial state and desired terminal state constraints of the vehicle are shown in Table 1.

[0207] Table 1 Initial and Desired Terminal States of Near-Space Vehicles

[0208]

[0209] The flight trajectory of a near-space vehicle is as follows Figure 5 As shown; the control quantity normal overload and lateral overload curves are as follows: Figures 6-7 As shown; the velocity curve is as follows Figure 8 As shown; the curves of track angle and heading angle are as follows Figures 9-10 As shown; the performance index curves are as follows Figure 11 As shown. The terminal status control errors are as follows: range error -0.0355 meters; altitude error 0.0002 meters; lateral position error -0.0349 meters; speed error 3.1964 × 10⁻⁶ meters. -6 m / s; track angle error is -9.8841×10 -7 The heading angle error is -1.3122 × 10⁻⁶ degrees; -5 Spend.

[0210] Figure 5 This indicates that the initial values ​​output by the reinforcement learning initial value generator ensure that the initial trajectory does not diverge significantly, guaranteeing that the subsequent trajectory planning method can achieve robust and fast convergence within the convergence radius, thereby achieving a high-precision trajectory planning task. Figures 6-7 This indicates that the normal and lateral overload curves are relatively smooth, ensuring high control accuracy for the subsequent inner-loop attitude tracking control system. For the more sensitive terminal speed control, Figure 8 This indicates that the velocity curve basically converges by the fifth iteration, demonstrating high terminal velocity control accuracy. Regarding the track angle and heading angle curves, Figures 9-10 This indicates that the two angle curves basically converged and had high accuracy by the sixth iteration. Figure 11 This indicates that through multiple iterations, the performance index curve achieved rapid stabilization and convergence.

[0211] Based on the same inventive concept, this application also provides a trajectory planning device for a variant near-space vehicle to implement the trajectory planning method for the variant near-space vehicle described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the trajectory planning device for a variant near-space vehicle provided below can be found in the limitations of the trajectory planning method for the variant near-space vehicle described above, and will not be repeated here.

[0212] In one exemplary embodiment, a trajectory planning device for a variant near-space vehicle is provided, comprising:

[0213] The 3D center-of-mass motion model construction module is used to construct a 3D center-of-mass motion model for a variant near-space vehicle.

[0214] The suboptimal initial trajectory generation module is used to obtain the suboptimal initial trajectory of the variant near-space vehicle by using an initial value generator based on the three-dimensional centroid motion model; the initial value generator is obtained by reinforcement learning of a two-layer deep Q-network.

[0215] The trajectory planning module is used to perform trajectory planning based on the suboptimal initial trajectory using an improved Newton-like trajectory planning method to obtain the trajectory planning result. The improved Newton-like trajectory planning method is obtained by introducing a damping coefficient based on adaptive law dynamic adjustment into the Newton-like trajectory planning method.

[0216] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 12 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and databases. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media to run. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a trajectory planning method for a variant of a near-space vehicle.

[0217] Those skilled in the art will understand that Figure 12 The structures shown are merely block diagrams of some structures related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0218] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0219] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0220] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0221] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0222] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0223] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0224] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A trajectory planning method for a variant near-space vehicle, characterized in that, include: Construct a three-dimensional center-of-mass motion model for a variant near-space vehicle; Based on the three-dimensional centroid motion model, an initial value generator is used to obtain the suboptimal initial trajectory of the variant near-space spacecraft. The initial value generator is obtained through reinforcement learning of a two-layer deep Q-network; Based on the suboptimal initial trajectory, an improved Newton-like trajectory planning method is used to perform trajectory planning and obtain the trajectory planning result. The improved Newton-like trajectory planning method is obtained by introducing a damping coefficient based on adaptive law dynamic adjustment into the Newton-like trajectory planning method; The three-dimensional centroid motion model is as follows: ； in, Indicates the speed of the variant near-space vehicle; Indicates the track angle; Indicates the heading angle; Indicates the distance traveled; Indicates altitude; Indicates lateral position; Indicates axial overload; and These represent normal overload and lateral overload in the vertical velocity direction, respectively; Represents gravitational acceleration; Based on the aforementioned three-dimensional centroid motion model, an initial value generator is used to obtain the suboptimal initial trajectory of the variant near-space spacecraft, specifically including: Two optimization parameters are generated based on the initial value generator; Based on two optimized parameters, the guidance law for each time step is obtained using the following formula; ； ； in, For guidance laws based on altitude, For lateral position guidance law, and All are optimized parameters. Indicates the speed of the variant near-space vehicle. Indicates the track angle, Indicates the heading angle. Represents gravitational acceleration; The suboptimal initial trajectory is generated based on the guidance law at each time step.

2. The trajectory planning method for a variant near-space vehicle according to claim 1, characterized in that, The reward function used in obtaining the initial value generator through reinforcement learning of a two-layer deep Q-network is: ； in, For the reward function, The position term penalty coefficient, This is the penalty coefficient for the speed term. For the track angle penalty coefficient, This is the heading angle penalty coefficient. The guidance law penalty coefficient, This is the cumulative speed penalty coefficient. To accumulate the track angle penalty coefficient, To accumulate the heading angle penalty factor, , , These are the terminal time points. The range, altitude, and lateral position, , , These are the terminal time points. The expected range, expected altitude, and expected lateral position. Terminal time point speed, Terminal time point track angle, Terminal time point The heading angle, For nodes Guidance law based on altitude, For nodes Lateral guidance law, For nodes speed, Terminal time point Expected speed, For nodes track angle, Terminal time point Expected track angle, For nodes The heading angle, Terminal time point The expected heading angle.

3. The trajectory planning method for a variant near-space vehicle according to claim 1, characterized in that, Based on the suboptimal initial trajectory, an improved Newton-like trajectory planning method is used for trajectory planning, and the formula for obtaining the trajectory planning result is as follows: ； in, Indicates the damping coefficient. This indicates the updated control history. This represents the control history in the suboptimal initial trajectory; This indicates the history of deviation control.

4. The trajectory planning method for a variant near-space vehicle according to claim 1 or 3, characterized in that, The damping coefficient is obtained through a trained neural network model.

5. The trajectory planning method for a variant near-space vehicle according to claim 1, characterized in that, Both the suboptimal initial trajectory and the trajectory planning result satisfy the following constraints: ； in, For the angle of attack, and These are the minimum and maximum angles of attack, respectively. The tilt angle, and These are the minimum and maximum values ​​of the tilt angle, respectively. The deformation ratio, and These are the minimum and maximum values ​​of the deformation ratio, respectively. , and All of these are actual control quantities, derived from the guidance laws for altitude and lateral positions.

6. A trajectory planning device for a variant near-space vehicle, characterized in that, The trajectory planning device for the variant near-space vehicle applies the trajectory planning method for the variant near-space vehicle according to any one of claims 1-5, and the trajectory planning device for the variant near-space vehicle comprises: The 3D center-of-mass motion model construction module is used to construct a 3D center-of-mass motion model for a variant near-space vehicle. The suboptimal initial trajectory generation module is used to obtain the suboptimal initial trajectory of the variant near-space vehicle by using an initial value generator based on the three-dimensional centroid motion model; the initial value generator is obtained by reinforcement learning of a two-layer deep Q-network. The trajectory planning module is used to perform trajectory planning based on the suboptimal initial trajectory using an improved Newton-like trajectory planning method to obtain the trajectory planning result. The improved Newton-like trajectory planning method is obtained by introducing a damping coefficient based on adaptive law dynamic adjustment into the Newton-like trajectory planning method.

7. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the trajectory planning method for a variant near-space vehicle according to any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the trajectory planning method for the variant near-space vehicle as described in any one of claims 1-5.

Images