Learning type unmanned tracked vehicle trajectory tracking prediction control method

Abstract

The invention relates to the technical field of unmanned tracked vehicle trajectory tracking, in particular to a learning type unmanned tracked vehicle trajectory tracking prediction control method, which comprises the following steps of: establishing an interactive training virtual environment oriented to tracked vehicle trajectory tracking controller learning; receiving vehicle global expected trajectory point information output by a tracked vehicle trajectory planning module; acquiring the motion states of the crawler and the obstacle at the current moment; a controller expected parameter instruction and a crawler longitudinal acceleration and yaw angle acceleration expected instruction are determined; a tracked vehicle longitudinal lateral motion tracking controller is constructed, and a vehicle longitudinal lateral motion bottom layer control instruction is determined; motion states of the crawler and the obstacle at the next moment are acquired; data are collected while interaction between the crawler and the environment is carried out, and a data experience pool for controller parameter learning is constructed; and constructing a predictive controller parameter reinforcement learning algorithm. According to the invention, through an autonomous exploration learning technology, the trajectory tracking precision of the unmanned tracked vehicle in a dynamic complex environment is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned tracked vehicle trajectory tracking, and in particular relates to a learning-type unmanned tracked vehicle trajectory tracking prediction control method. Background Art

[0002] Trajectory tracking is a key technology for unmanned tracked vehicles, and its adaptability is crucial for them to navigate dynamic, open, and complex operating environments. However, existing rule-based or optimization-based trajectory tracking technologies are unable to accurately model the kinematics and dynamics of tracked vehicles during motion, resulting in low trajectory tracking accuracy and even collisions with obstacles. Furthermore, existing tracked vehicle trajectory tracking controllers have fixed parameters, resulting in poor adaptability. Therefore, mitigating the negative impact of uncertainty in modeling the kinematics and dynamics of unmanned tracked vehicles while improving the adaptability of trajectory tracking control to achieve efficient, safe, and highly adaptable trajectory tracking control for unmanned tracked vehicles is a significant challenge. Summary of the Invention

[0003] The purpose of the present invention is to solve the problem of reducing the negative impact of uncertainty in the kinematic and dynamic modeling of unmanned tracked vehicles proposed in the background technology, and to propose a learning-based unmanned tracked vehicle trajectory tracking prediction control method.

[0004] The present invention provides a learning-based unmanned tracked vehicle trajectory tracking prediction control method, the method comprising:

[0005] S1: Establish an interactive training virtual environment for learning tracked vehicle trajectory tracking controllers. The interactive training virtual environment includes a road surface physics model, a vehicle sensor physics model, a tracked vehicle kinematic and dynamic physics model, and an obstacle kinematic physics model.

[0006] S2: Receive the vehicle global expected trajectory point information output by the tracked vehicle trajectory planning module;

[0007] S3: Based on the road surface physical model, the onboard sensor physical model, the tracked vehicle kinematic and dynamic physical model, the obstacle kinematic physical model, and the global expected trajectory points obtained in step S2, the motion state of the tracked vehicle and the obstacle at the current moment is acquired;

[0008] S4: Construct the parameter instruction set, state parameter value network, and parameter selection exploration mechanism of the trajectory tracking prediction controller parameter interactive learning algorithm to determine the controller's expected parameter instructions;

[0009] S5: Based on the tracked caterpillar vehicle trajectory controller expected parameter instruction, the vehicle global expected trajectory point obtained in step S2, and the motion state of the tracked caterpillar vehicle and the obstacle at the current time obtained in step S3, a trajectory tracking predictive controller is constructed by using a multi-constraint model predictive control algorithm to determine the longitudinal acceleration and yaw angle acceleration expected instructions of the tracked caterpillar vehicle;

[0010] S6: A tracked caterpillar vehicle longitudinal and lateral motion tracking controller is constructed to track the vehicle longitudinal acceleration and yaw angle acceleration expected instructions to determine the vehicle longitudinal and lateral motion bottom layer control instructions;

[0011] S7: The vehicle longitudinal and lateral motion bottom layer control instructions are output to the tracked caterpillar vehicle kinematics and dynamics physical model to control the motion while performing single-step motion state transition, and the motion state of the tracked caterpillar vehicle and the obstacle at the next time is collected;

[0012] S8: A state space, a parameter space, and a reward function for the tracked caterpillar vehicle trajectory tracking predictive controller interactive learning are constructed;

[0013] S9: Based on the state space, the parameter space, and the reward function for the tracked caterpillar vehicle controller interactive learning, and the parameter selection exploration mechanism in step S4, the tracked caterpillar vehicle and the environment are interacted while collecting data, and a data experience pool for the controller parameter learning is constructed;

[0014] S10: Based on the data experience pool for the controller parameter learning, a predictive controller parameter reinforcement learning algorithm is constructed to continuously update the state parameter value network in step S4.

[0015] Further, the step S3 specifically comprises:

[0016] S3-1: The motion state of the tracked caterpillar vehicle and the obstacle at the current time is collected, including: the projection coordinates (x traj,i ,y traj,i ,φ traj,i ) of the global expected trajectory point sequence in the vehicle coordinate system xoy, wherein the three coordinates respectively represent the longitudinal position, the lateral position, and the yaw angle of a certain global expected trajectory point, and i is the time step index; the longitudinal velocity u and the lateral velocity v of the tracked caterpillar vehicle mass center in the vehicle body coordinate system xoy; the yaw rate ω of the tracked caterpillar vehicle at the mass center; the yaw angle φ of the tracked caterpillar vehicle; the longitudinal acceleration a of the tracked caterpillar vehicle mass center in the vehicle body coordinate system; the yaw angle acceleration ω d of the tracked caterpillar vehicle at the mass center; the relative longitudinal distance Δx b , the relative lateral distance Δy b , the relative longitudinal velocity Δu b , and the relative lateral velocity Δv b of the tracked caterpillar vehicle and the obstacle in the vehicle body coordinate system.

[0017] Furthermore, the step S4 specifically includes:

[0018] S4-1: Constructing a parameter instruction set for the interactive learning algorithm of trajectory tracking predictive controller parameters;

[0019] The parameter instruction set includes N act,Q Trajectory tracking weight parameter action instructions, Control variable amplitude suppression weight parameter action instruction, control variable fluctuation suppression weight parameter action instruction, and N act,W Obstacle avoidance weight parameter action instructions; among them, the trajectory tracking weight parameter action instruction ranges from 0 to the maximum value Q max Equally spaced discrete, the control variable amplitude suppresses the weight parameter action instruction from 0 to the maximum value R u,max Equally spaced discrete, control variable fluctuation suppression weight parameter action instruction from 0 to the maximum value R Δu,max The obstacle avoidance weight parameter action instruction is discrete at equal intervals and ranges from 0 to the maximum value W. max Equally spaced discretization;

[0020] S4-2: Based on the parameter instruction set of the trajectory tracking prediction controller, a state parameter value network is constructed. The state parameter value network is a fully connected network of N1×N2×N3×N4; where N1, N2, N3, and N4 are the number of neurons in the 1st, 2nd, 3rd, and 4th layers of the neural network, respectively. At the same time, N1 is set equal to the sum of the motion state dimensions of the crawler vehicle and the obstacle in step S3-1 and the dimension of any column vector in the controller parameter instruction set. The activation function is designed to be a hyperbolic tangent function;

[0021] S4-3: Based on the tracked vehicle state-parameter value network, a controller parameter selection exploration mechanism is constructed: a positive decimal p1 is randomly initialized. If p1 is greater than or equal to the threshold p0, a column vector is randomly selected from the controller parameter instruction set as the controller expected parameter instruction. Otherwise, the motion state of the tracked vehicle and the obstacle at the current moment in step S3-1, and each column vector in the controller parameter instruction set are respectively augmented and constructed into a single column vector, and are respectively input into the tracked vehicle state-parameter value network to obtain the output value, and the controller parameter instruction corresponding to the maximum network output value is selected as the tracked vehicle controller expected parameter instruction.

[0022] Furthermore, the step S5 specifically includes:

[0023] S5-1: Receive the desired parameter instruction of the crawler vehicle controller obtained in step S4-3 and define it as the desired trajectory tracking weight parameter action instruction Q act , expected control variable amplitude suppression weight parameter action instruction R u,act , expected control variable fluctuation suppression weight parameter action instruction RΔu,act , expected obstacle avoidance weight parameter action instruction W act , and the projection coordinates (x traj,i ,y traj,i ,φ traj,i );

[0024] S5-2: Constructing a differential model of the longitudinal and lateral coupled kinematics and dynamic motion control of a tracked vehicle:

[0025]

[0026] Among them, x and y are the longitudinal and lateral positions of the tracked vehicle's center of mass in the vehicle body coordinate system, respectively; x, y, The first-order derivatives of u, v, and ω; C is the lateral sliding friction coefficient of the track; g is the acceleration of gravity; tanh is the hyperbolic tangent function; γ is a predefined small real number; and L is the longitudinal length of the tracked vehicle's contact patch.

[0027] S5-3: Determine the state variables of the tracked vehicle predictive control model Output variables and control variables for:

[0028]

[0029] S5-4: Construct a time discrete prediction model based on the crawler vehicle motion control differential model in step S5-2:

[0030]

[0031] Where i is the index of the prediction time step; is the state variable at time i; are the state variables and output variables at time i+1 respectively; is the control variable at time i; I is the unit matrix; f(·) represents the crawler vehicle motion control differential model in step S5-2; is the partial differential symbol; is the reference working point; Z is the transformation matrix; T p is the prediction time domain step; N p is the total number of steps in the prediction time domain;

[0032] S5-5: Constructing a multi-constraint predictive optimization control problem for tracked vehicle trajectory tracking:

[0033]

[0034] Among them, J tis the optimization target at time t; is the output variable at time i;

[0035] is the projection coordinate vector of the global expected trajectory point sequence in step S3-1 in the vehicle coordinate system xoy; N c To control the time domain; are the longitudinal and lateral positions of the obstacle at time i, respectively; The state variable constraints of the crawler vehicle predictive control model in step S5-3; Output variable constraints for predictive control models; Control variable constraints for predictive control models; Incremental constraints on control variables for predictive control models; To control the vector increment sequence, define it as follows:

[0036]

[0037] in, is the difference between the control variable at the current moment and the control variable at the previous moment;

[0038] S5-6: Solve the optimization control problem in step S5-5 to obtain And The first column vector of is summed with the control variable of the previous moment to obtain the desired longitudinal acceleration instruction a of the crawler vehicle ref and the desired yaw angular acceleration command ω d,ref .

[0039] Furthermore, the step S6 specifically includes:

[0040] S6-1: Based on the expected longitudinal acceleration instruction a of the crawler vehicle ref and the desired yaw angular acceleration command ω d,ref , determine the desired speed of the left and right motors:

[0041]

[0042] Among them, s left,ref 、s right,ref are the desired speeds of the left and right motors respectively; E is the lateral spacing between the two tracks; R d is the rolling radius of the tracked vehicle's driving wheel;

[0043] S6-2: Construct a motor speed tracking PID controller, calculate the driving torque, and obtain the underlying control instructions for the longitudinal and lateral motion of the vehicle, where: They are the proportional, integral and differential parameters of the left motor controller respectively. are the proportional, integral, and differential parameters of the right motor controller.

[0044] Furthermore, the step S8 specifically includes:

[0045] S8-1: Based on the current motion states of the tracked vehicle and the obstacle obtained in step S3-1 and the parameter instruction set of the trajectory tracking prediction controller parameter interactive learning algorithm obtained in step S4-1, construct a state space and an action space for interactive learning respectively;

[0046] S8-2: Constructing interactive learning reward function:

[0047] R total =R b +R tra

[0048] Among them, R b is the reward function for collision with an obstacle:

[0049]

[0050] Among them, r b is the collision reward function value; R tra The reward function for trajectory tracking is:

[0051]

[0052] Among them, w tra is the weight coefficient, x traj,t 、y traj,t 、φ traj,t are respectively the longitudinal position, lateral position, and yaw angle of the desired trajectory point closest to the vehicle's center of mass in the vehicle body coordinate system at the current time t.

[0053] Furthermore, the step S9 specifically includes:

[0054] S9-1: constructing an interactive learning state vector based on the motion states of the crawler vehicle and the obstacle at the current moment obtained in step S3-1;

[0055] S9-2: constructing the controller expected parameter instruction obtained in step S4 into an action vector;

[0056] S9-3: constructing the motion states of the crawler vehicle and the obstacle at the next moment obtained in step S7 into the interactive learning state vector at the next moment;

[0057] S9-4: For state transition, calculate the action selection reward based on the reward function in step S8-2;

[0058] S9-5: Construct the state transition sequence and reward function into a single interactive learning training data and store it in the experience pool.

[0059] Furthermore, the step S10 specifically includes:

[0060] S10-1: Set the state parameter value network in step S4-2 as the Q network and copy it as the target Q network;

[0061] S10-2: Sample from the experience pool obtained in step S9 and construct the interactive learning loss function:

[0062]

[0063] Among them, Loss(w t ) is the loss function; s t is the state of the interactive learning agent at time t; a t 、a t+1 The actions of the interactive learning agent at time t and time t+1 respectively; t+1 is the state at time t+1; is the state transfer reward; B is the experience pool; w t 、w target are the parameters of the Q network and the target Q network respectively; Q(·) represents the calculation of the neural network output value; λ is the discount factor;

[0064] Use the gradient descent method to update the Q network parameters, where γ is the learning rate, and each training n train The Q network parameters are copied to the target Q network in the next step, thereby continuously updating the state-parameter value network until convergence.

[0065] Compared with the prior art, the present invention has the following beneficial effects:

[0066] 1. Through autonomous exploration and learning technology, the negative impact of uncertainty in the kinematics and dynamics modeling of unmanned tracked vehicles on trajectory tracking can be reduced, thereby improving trajectory tracking accuracy.

[0067] 2. It can realize the self-adaptation of trajectory tracking prediction controller parameters, thereby improving the adaptability of trajectory tracking control and realizing efficient, safe and highly adaptable trajectory tracking control for unmanned tracked vehicles.

[0068] In order to more clearly illustrate the functional characteristics and structural parameters of the present invention, further description is given below with reference to the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Fig. 1 A schematic diagram of the trajectory tracking prediction control process of a learning unmanned tracked vehicle provided in an embodiment of the present application;

[0070] Fig. 2 Schematic diagram of the trajectory tracking and obstacle avoidance principles provided in an embodiment of the present application. DETAILED DESCRIPTION

[0071] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0072] See also Figs. 1-2 The present invention provides a technical solution: a learning-based unmanned tracked vehicle trajectory tracking prediction control method, comprising the following steps:

[0073] S1: Establish an interactive training virtual environment for tracked vehicle trajectory tracking controller learning, including road surface physics model, vehicle sensor physics model, tracked vehicle kinematic and dynamic physics model, obstacle kinematic physics model, etc.

[0074] S2: Receive the vehicle global expected trajectory point information output by the tracked vehicle trajectory planning module;

[0075] S3: Based on the road surface physical model, the onboard sensor physical model, the tracked vehicle kinematic and dynamic physical model, the obstacle kinematic physical model, and the global expected trajectory points obtained in step S2, the motion state of the tracked vehicle and the obstacle at the current moment is acquired;

[0076] S4: Construct the parameter instruction set, state-parameter value network, and parameter selection exploration mechanism of the trajectory tracking prediction controller parameter interactive learning algorithm, and calculate the controller's expected parameter instructions;

[0077] S5: Based on the desired parameter instructions of the tracked vehicle trajectory tracking controller, the global desired trajectory point of the vehicle obtained in step S2, and the motion state of the tracked vehicle and the obstacle at the current moment obtained in step S3, a trajectory tracking predictive controller is constructed using a multi-constraint model predictive control algorithm to calculate the desired instructions for the longitudinal acceleration and yaw angular acceleration of the tracked vehicle;

[0078] S6: Construct a tracked vehicle longitudinal and lateral motion tracking controller to track the desired longitudinal acceleration and yaw angular acceleration of the vehicle and calculate the underlying control instructions for the longitudinal and lateral motion of the vehicle;

[0079] S7: Outputting the vehicle's longitudinal and lateral motion control instructions to the tracked vehicle's kinematic and dynamic physical models to control its motion while performing single-step motion state transfer. Based on this, the motion state of the tracked vehicle and the obstacle at the next moment is acquired.

[0080] S8: Construct the state space, parameter space, and reward function for interactive learning of tracked vehicle trajectory tracking predictive controller;

[0081] S9: Based on the state space, parameter space, and reward function of the tracked vehicle controller interactive learning, and the parameter selection exploration mechanism in step S4, the tracked vehicle and the environment interact and collect data, and on this basis, a data experience pool for controller parameter learning is constructed;

[0082] S10: Based on the data experience pool of controller parameter learning, a predictive controller parameter reinforcement learning algorithm is constructed to continuously update the state-parameter value network in step S4.

[0083] The step S3 specifically includes:

[0084] S3-1: Collect the motion state of the tracked vehicle and the obstacle at the current moment, including: the projection coordinates (x traj,i ,y traj,i ,φ traj,i ), where the three coordinates represent the longitudinal position, lateral position, and yaw angle of a global desired trajectory point, respectively; i is the time step index; the longitudinal velocity u and lateral velocity v of the tracked vehicle's center of mass in the vehicle body coordinate system xoy; the yaw angular velocity ω of the tracked vehicle at the center of mass; the yaw angle φ of the tracked vehicle; the longitudinal acceleration a of the tracked vehicle's center of mass in the vehicle body coordinate system; the yaw angular acceleration ω of the tracked vehicle at the center of mass d ; The relative longitudinal distance Δx between the tracked vehicle and the obstacle in the vehicle body coordinate system b , relative lateral distance Δy b , relative longitudinal velocity Δu b , relative lateral velocity Δv b .

[0085] The step S4 specifically includes:

[0086] S4-1: Construct the parameter instruction set of the trajectory tracking predictive controller parameter interactive learning algorithm, including N act,Q =5 trajectory tracking weight parameter action instructions, Control variable amplitude suppression weight parameter action instruction, control variable fluctuation suppression weight parameter action instruction, and N act,W = 5 obstacle avoidance weight parameter action instructions, among which the trajectory tracking weight parameter action instruction ranges from 0 to the maximum value Q max = 10 equally spaced discrete values, the control variable amplitude suppresses the weight parameter action instruction from 0 to the maximum value R u,max = 10 equally spaced discrete, control variable fluctuation suppression weight parameter action instruction from 0 to the maximum value R Δu,max= 10 equally spaced discrete, obstacle avoidance weight parameter action instruction ranges from 0 to the maximum value W max =10 equally spaced discrete;

[0087] S4-2: Based on the parameter instruction set of the trajectory tracking prediction controller, a state-parameter value network is constructed. The network is designed as a fully connected network of N1×N2×N3×N4, where N1=34, N2=256, N3=256, and N4=1 are the number of neurons in the 1st, 2nd, 3rd, and 4th layers of the neural network, respectively. At the same time, N1 is set equal to the sum of the motion state dimensions of the crawler vehicle and the obstacle in step S3-1 and the dimension of any column vector in the controller parameter instruction set. The activation function is designed to be a hyperbolic tangent function;

[0088] S4-3: Based on the tracked vehicle state-parameter value network, a controller parameter selection exploration mechanism is constructed: a positive decimal p1 is randomly initialized. If p1 is greater than or equal to the threshold p0 = 0.85, a column vector is randomly selected from the controller parameter instruction set as the controller expected parameter instruction. Otherwise, the motion state of the tracked vehicle and the obstacle at the current moment in step S3-1, and each column vector in the controller parameter instruction set are respectively augmented and constructed into a single column vector, and are respectively input into the tracked vehicle state-parameter value network to obtain the output value, and the controller parameter instruction corresponding to the maximum network output value is further selected as the tracked vehicle controller expected parameter instruction.

[0089] The step S5 specifically includes:

[0090] S5-1: Receive the desired parameter instruction of the crawler vehicle controller obtained in step S4-3 and define it as the desired trajectory tracking weight parameter action instruction Q act , expected control variable amplitude suppression weight parameter action instruction R u,act , expected control variable fluctuation suppression weight parameter action instruction R Δu,act , expected obstacle avoidance weight parameter action instruction W act , and the projection coordinates (x traj,i ,y traj,i ,φ traj,i );

[0091] S5-2: Constructing a differential model of the longitudinal and lateral coupled kinematics and dynamic motion control of a tracked vehicle:

[0092]

[0093] Among them, x and y are the longitudinal and lateral positions of the tracked vehicle's center of mass in the vehicle body coordinate system, respectively; x, y, The first derivative of u, v, and ω; C = 0.7 is the lateral sliding friction coefficient of the belt soil; g = 9.8 m / s 2 is the acceleration due to gravity; γ = 0.01 is a predefined small real number; L = 1.6m is the longitudinal length of the tracked vehicle's contact patch.

[0094] S5-3: Design state variables of the predictive control model for tracked vehicles Output variables and control variables

[0095]

[0096] S5-4: Construct a time discrete prediction model based on the crawler vehicle motion control differential model in step S5-2:

[0097]

[0098] Where i is the prediction time step index; is the state variable at time i; are the state variables and output variables at time i+1 respectively; is the control variable at time i; I is the unit matrix; f(·) represents the crawler vehicle motion control differential model in step S5-2; is the partial differential symbol; is the reference working point; Z is the transformation matrix; T p =0.05s is the prediction time domain step; N p =10 is the total number of steps in the prediction time domain. S5-5: Constructing the multi-constraint predictive optimization control problem of tracked vehicle trajectory tracking:

[0099]

[0100] Among them, J t is the optimization target at time t; is the output variable at time i;

[0101] is the projection coordinate vector of the global expected trajectory point sequence in step S3-1 in the vehicle coordinate system xoy; N c =5 is the control time domain; are the longitudinal and lateral positions of the obstacle at time i, respectively; The state variable constraints of the crawler vehicle predictive control model in step S5-3; Output variable constraints for predictive control models; Control variable constraints for predictive control models; Incremental constraints on control variables for predictive control models; To control the vector increment sequence, define it as follows:

[0102]

[0103] in, is the difference between the control variable at the current moment and the control variable at the previous moment.

[0104] S5-6: Solve the optimization control problem in step S5-5 to obtain And The first column vector of is summed with the control variable of the previous moment to obtain the desired longitudinal acceleration instruction a of the crawler vehicle ref and the desired yaw angular acceleration command ω d,ref .

[0105] The step S6 specifically includes:

[0106] S6-1: Based on the expected longitudinal acceleration instruction a of the crawler vehicle ref and the desired yaw angular acceleration command ω d,ref , calculate the expected speed of the left and right motors:

[0107]

[0108] Among them, s left,ref 、s right,ref are the desired speeds of the left and right motors respectively; E = 1.3m is the lateral distance between the two tracks; R d =0.16m is the rolling radius of the crawler vehicle's driving wheel;

[0109] S6-2: Construct a motor speed tracking PID controller, calculate the driving torque, and obtain the underlying control instructions for the longitudinal and lateral motion of the vehicle, where: They are the proportional, integral and differential parameters of the left motor controller respectively. are the proportional, integral, and differential parameters of the right motor controller.

[0110] The step S8 specifically includes:

[0111] S8-1: Based on the current motion states of the tracked vehicle and the obstacle obtained in step S3-1 and the parameter instruction set of the trajectory tracking prediction controller parameter interactive learning algorithm obtained in step S4-1, construct a state space and an action space for interactive learning respectively;

[0112] S8-2: Constructing interactive learning reward function:

[0113] R total =R b +R tra

[0114] Among them, R bis the reward function for collision with an obstacle:

[0115]

[0116] Among them, r b is the collision reward function value; R tra The reward function for trajectory tracking is:

[0117]

[0118] Among them, w tra =-10 is the weight coefficient, x traj,t 、y traj,t 、φ traj,t are respectively the longitudinal position, lateral position, and yaw angle of the desired trajectory point closest to the vehicle's center of mass in the vehicle body coordinate system at the current time t.

[0119] The step S9 specifically includes:

[0120] S9-1: constructing an interactive learning state vector based on the motion states of the crawler vehicle and the obstacle at the current moment obtained in step S3-1;

[0121] S9-2: constructing the controller expected parameter instruction obtained in step S4 into an action vector;

[0122] S9-3: constructing the motion states of the crawler vehicle and the obstacle at the next moment obtained in step S7 into the interactive learning state vector at the next moment;

[0123] S9-4: For the above state transition, the action selection reward is calculated based on the reward function in step S8-2;

[0124] S9-5: Construct the above state transition sequence and reward function into a single interactive learning training data and store it in the experience pool.

[0125] The step S10 specifically includes:

[0126] S10-1: Set the state-parameter value network in step S4-2 as a Q network and copy it as the target Q network;

[0127] S10-2: Sample from the experience pool obtained in step S9 and construct the interactive learning loss function:

[0128]

[0129] Among them, Loss(w t ) is the loss function; s t is the state of the interactive learning agent at time t; a t 、a t+1The actions of the interactive learning agent at time t and time t+1 respectively; t+1 is the state at time t+1; is the state transfer reward; B is the experience pool; w t 、w target are the parameters of the Q network and the target Q network respectively; Q(·) represents the calculation of the neural network output value; λ=0.9 is the discount factor. On this basis, the Q network parameters are updated using the gradient descent method, where γ=0.001 is the learning rate, and each training n train =50 steps to copy the Q network parameters to the target Q network, thereby continuously updating the state-parameter value network until convergence.

[0130] Fig. 2 where u and v are the longitudinal and lateral velocities of the tracked vehicle's center of mass in the vehicle body coordinate system xoy; r tk 、r b are the envelope radii of the tracked vehicle and the obstacle respectively; (x traj,i ,y traj,i ,φ traj,i ) is the projection coordinate of the global expected trajectory point sequence in the vehicle coordinate system xoy; (x b,i ,y b,i ) are the predicted coordinates of the obstacle trajectory points.

[0131] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A learning-based unmanned tracked vehicle trajectory tracking prediction control method, characterized in that: The method comprises: S1: Establish an interactive training virtual environment for tracked vehicle trajectory tracking controller learning; the interactive training virtual environment includes a road surface physical model, a vehicle sensor physical model, a tracked vehicle kinematic and dynamic physical model, and an obstacle kinematic physical model; S2, receiving the vehicle global expected trajectory point information output by the tracked vehicle trajectory planning module; S3, based on the road surface physical model, the onboard sensor physical model, the tracked vehicle kinematic and dynamic physical model, the obstacle kinematic physical model, and the global expected trajectory points obtained in step S2, the motion state of the tracked vehicle and the obstacle at the current moment is acquired; S4, construct the parameter instruction set, state parameter value network, and parameter selection exploration mechanism of the trajectory tracking prediction controller parameter interactive learning algorithm to determine the controller's expected parameter instructions; S5, based on the tracked vehicle trajectory tracking controller desired parameter instructions, the vehicle global desired trajectory point obtained in step S2, and the current motion state of the tracked vehicle and the obstacle obtained in step S3, a trajectory tracking predictive controller is constructed using a multi-constraint model predictive control algorithm to determine the desired instructions for the tracked vehicle longitudinal acceleration and yaw angular acceleration; S6, constructing a tracked vehicle longitudinal and lateral motion tracking controller, tracking the vehicle longitudinal acceleration and yaw angular acceleration desired instructions, and determining the vehicle longitudinal and lateral motion underlying control instructions; S7, outputting the vehicle longitudinal and lateral motion bottom-level control instructions to the tracked vehicle kinematic and dynamic physical models, controlling the motion while performing single-step motion state transfer, and acquiring the motion state of the tracked vehicle and the obstacle at the next moment; S8, construct the state space, parameter space, and reward function for interactive learning of tracked vehicle trajectory tracking prediction controller; S9, based on the state space, parameter space, and reward function of the tracked vehicle controller interactive learning, and the parameter selection exploration mechanism in step S4, the tracked vehicle and the environment interact while collecting data, and building a data experience pool for controller parameter learning; S10, based on the data experience pool of controller parameter learning, construct a predictive controller parameter reinforcement learning algorithm to continuously update the state parameter value network in step S4.

2. The method according to claim 1, characterized in that S3, based on the road surface physical model, the onboard sensor physical model, the tracked vehicle kinematic and dynamic physical model, the obstacle kinematic physical model, and the global expected trajectory points obtained in step S2, the motion state of the tracked vehicle and the obstacle at the current moment is collected, including: S3-1: Collect the motion state of the tracked vehicle and the obstacle at the current moment, including: the projection coordinates (x traj,i ,y traj,i ,φ traj,i ), where the three coordinates represent the longitudinal position, lateral position, and yaw angle of a global desired trajectory point, respectively; i is the time step index; the longitudinal velocity u and lateral velocity v of the tracked vehicle's center of mass in the vehicle body coordinate system xoy; the yaw angular velocity ω of the tracked vehicle at the center of mass; the yaw angle φ of the tracked vehicle; the longitudinal acceleration a of the tracked vehicle's center of mass in the vehicle body coordinate system; the yaw angular acceleration ω of the tracked vehicle at the center of mass d ; The relative longitudinal distance Δx between the tracked vehicle and the obstacle in the vehicle body coordinate system b , relative lateral distance Δy b , relative longitudinal velocity Δu b , relative lateral velocity Δv b .

3. The method according to claim 2, characterized in that S4: Construct the parameter instruction set, state parameter value network, and parameter selection exploration mechanism of the trajectory tracking prediction controller parameter interactive learning algorithm to determine the controller's expected parameter instructions, including: S4-1: Constructing a parameter instruction set for the interactive learning algorithm of trajectory tracking predictive controller parameters; The parameter instruction set includes N act,Q Trajectory tracking weight parameter action instructions, Control variable amplitude suppression weight parameter action instruction, control variable fluctuation suppression weight parameter action instruction, and N act,W Obstacle avoidance weight parameter action instructions; among them, the trajectory tracking weight parameter action instruction ranges from 0 to the maximum value Q max Equally spaced discrete, the control variable amplitude suppresses the weight parameter action instruction from 0 to the maximum value R u,max Equally spaced discrete, control variable fluctuation suppression weight parameter action instruction from 0 to the maximum value R Δu,max The obstacle avoidance weight parameter action instruction is discrete at equal intervals and ranges from 0 to the maximum value W. max Equally spaced discretization; S4-2: Based on the parameter instruction set of the trajectory tracking prediction controller, a state parameter value network is constructed. The state parameter value network is a fully connected network of N1×N2×N3×N4; where N1, N2, N3, and N4 are the number of neurons in the 1st, 2nd, 3rd, and 4th layers of the neural network, respectively. At the same time, N1 is set equal to the sum of the motion state dimensions of the crawler vehicle and the obstacle in step S3-1 and the dimension of any column vector in the controller parameter instruction set. The activation function is designed to be a hyperbolic tangent function; S4-3: Based on the tracked vehicle state-parameter value network, a controller parameter selection exploration mechanism is constructed: a positive decimal p1 is randomly initialized. If p1 is greater than or equal to the threshold p0, a column vector is randomly selected from the controller parameter instruction set as the controller expected parameter instruction. Otherwise, the motion state of the tracked vehicle and the obstacle at the current moment in step S3-1, and each column vector in the controller parameter instruction set are respectively augmented and constructed into a single column vector, and are respectively input into the tracked vehicle state-parameter value network to obtain the output value, and the controller parameter instruction corresponding to the maximum network output value is selected as the tracked vehicle controller expected parameter instruction.

4. The method according to claim 3, characterized in that S5, based on the tracked vehicle trajectory tracking controller desired parameter instructions, the vehicle global desired trajectory point obtained in step S2, and the current motion state of the tracked vehicle and the obstacle obtained in step S3, a trajectory tracking predictive controller is constructed using a multi-constraint model predictive control algorithm to determine the desired instructions for the tracked vehicle longitudinal acceleration and yaw angular acceleration, including: S5-1: Receive the desired parameter instruction of the crawler vehicle controller obtained in step S4-3 and define it as the desired trajectory tracking weight parameter action instruction Q act , expected control variable amplitude suppression weight parameter action instruction R u,act , expected control variable fluctuation suppression weight parameter action instruction R Δu,act , expected obstacle avoidance weight parameter action instruction W act , and the projection coordinates (x traj,i ,y traj,i ,φ traj,i ); S5-2: Constructing a differential model of the longitudinal and lateral coupled kinematics and dynamic motion control of a tracked vehicle: Among them, x and y are the longitudinal and lateral positions of the tracked vehicle's center of mass in the vehicle body coordinate system, respectively; x, y, The first-order derivatives of u, v, and ω; C is the lateral sliding friction coefficient of the track; g is the acceleration of gravity; tanh is the hyperbolic tangent function; γ is a predefined small real number; and L is the longitudinal length of the tracked vehicle's contact patch. S5-3: Determine the state variables of the tracked vehicle predictive control model Output variables and control variables for: S5-4: Construct a time discrete prediction model based on the crawler vehicle motion control differential model in step S5-2: Where i is the prediction time step index; is the state variable at time i; are the state variables and output variables at time i+1 respectively; is the control variable at time i; I is the unit matrix; f(·) represents the crawler vehicle motion control differential model in step S5-2; is the partial differential symbol; is the reference working point; Z is the transformation matrix; T p is the prediction time domain step; N p is the total number of steps in the prediction time domain; S5-5: Constructing a multi-constraint predictive optimization control problem for tracked vehicle trajectory tracking: Among them, J t is the optimization target at time t; is the output variable at time i; is the projection coordinate vector of the global expected trajectory point sequence in step S3-1 in the vehicle coordinate system xoy; N c To control the time domain; are the longitudinal and lateral positions of the obstacle at time i, respectively; The state variable constraints of the crawler vehicle predictive control model in step S5-3; Output variable constraints for predictive control models; Control variable constraints for predictive control models; Incremental constraints on control variables for predictive control models; To control the vector increment sequence, define it as follows: in, is the difference between the control variable at the current moment and the control variable at the previous moment; S5-6: Solve the optimization control problem in step S5-5 to obtain And The first column vector of is summed with the control variable of the previous moment to obtain the desired longitudinal acceleration instruction a of the crawler vehicle ref and the desired yaw angular acceleration command ω d,ref .

5. The method according to claim 4, characterized in that S6: Build a tracked vehicle longitudinal and lateral motion tracking controller to track the vehicle's longitudinal acceleration and yaw angular acceleration desired instructions and determine the underlying control instructions for the vehicle's longitudinal and lateral motion, including: S6-1: Based on the expected longitudinal acceleration instruction a of the crawler vehicle ref and the desired yaw angular acceleration command ω d,ref , determine the desired speed of the left and right motors: Among them, s left,ref 、s right,ref are the desired speeds of the left and right motors respectively; E is the lateral distance between the two tracks; R d is the rolling radius of the tracked vehicle's driving wheel; S6-2: Construct a motor speed tracking PID controller, calculate the driving torque, and obtain the underlying control instructions for the longitudinal and lateral motion of the vehicle, where: They are the proportional, integral and differential parameters of the left motor controller respectively. are the proportional, integral, and differential parameters of the right motor controller.

6. The method according to claim 5, characterized in that S8: Constructing the state space, parameter space, and reward function for interactive learning of the tracked vehicle trajectory tracking prediction controller; including: S8-1: Based on the current motion states of the tracked vehicle and the obstacle obtained in step S3-1 and the parameter instruction set of the trajectory tracking prediction controller parameter interactive learning algorithm obtained in step S4-1, construct a state space and an action space for interactive learning respectively; S8-2: Constructing interactive learning reward function: R total =R b +R tra Among them, R b is the reward function for collision with an obstacle: Among them, r b is the collision reward function value; R tra The reward function for trajectory tracking is: Among them, w tra is the weight coefficient, x traj,t 、y traj,t 、φ traj,t are respectively the longitudinal position, lateral position, and yaw angle of the desired trajectory point closest to the vehicle's center of mass in the vehicle body coordinate system at the current time t.

7. The method according to claim 6, characterized in that S9, based on the state space, parameter space, and reward function of the tracked vehicle controller interactive learning, and the parameter selection exploration mechanism in step S4, the tracked vehicle and the environment interact while collecting data, and constructing a data experience pool for controller parameter learning; including: S9-1: constructing an interactive learning state vector based on the motion states of the crawler vehicle and the obstacle at the current moment obtained in step S3-1; S9-2: constructing the controller expected parameter instruction obtained in step S4 into an action vector; S9-3: constructing the motion states of the crawler vehicle and the obstacle at the next moment obtained in step S7 into the interactive learning state vector at the next moment; S9-4: For state transition, calculate the action selection reward based on the reward function in step S8-2; S9-5: Construct the state transition sequence and reward function into a single interactive learning training data and store it in the experience pool.

8. The method according to claim 7, characterized in that S10, based on the data experience pool of controller parameter learning, construct a predictive controller parameter reinforcement learning algorithm to continuously update the state parameter value network in step S4, including: S10-1: Set the state parameter value network in step S4-2 as the Q network and copy it as the target Q network; S10-2: Sample from the experience pool obtained in step S9 and construct the interactive learning loss function: Among them, Loss(w t ) is the loss function; s t is the state of the interactive learning agent at time t; a t 、a t+1 The actions of the interactive learning agent at time t and time t+1 respectively; t+1 is the state at time t+1; is the state transfer reward; B is the experience pool; w t 、w target are the parameters of the Q network and the target Q network respectively; Q(·) represents the calculation of the neural network output value; λ is the discount factor; Use the gradient descent method to update the Q network parameters, where γ is the learning rate, and each training n train The Q network parameters are copied to the target Q network in the next step, thereby continuously updating the state-parameter value network until convergence.

Images