Trajectory tracking control system and method for paddy field vehicle

Abstract

The invention aims to provide a trajectory tracking control system and method for a paddy field vehicle. The method comprises the following steps: acquiring position information and operation path requirements of the vehicle; generating a vehicle track according to the position information of the vehicle and the operation path requirement; controlling the vehicle to work according to the vehicle track and the paddy field parameters by adopting an agricultural machinery kinetic model; and adopting a reinforcement learning model to optimize the agricultural machinery dynamical model according to the actual trajectory and the predicted trajectory of the vehicle. The agricultural machinery dynamic model can be dynamically adjusted through a reinforcement learning method, a more accurate model is provided for MPC, the trajectory tracking precision of the agricultural machinery in the paddy field environment is improved, and technical guarantee is provided for efficient paddy field agricultural operation.

Description

Technical Field

[0001] The invention relates to the technical field of farmland vehicle control, and in particular to a track tracking control system and method for a paddy field vehicle. Background Art

[0002] In recent years, the full implementation of the rural revitalization strategy has significantly accelerated the modernization process of agriculture and rural areas, and the application of modern information technology in the agricultural field has become more and more in-depth. As a result, smart agriculture and digital agriculture have received widespread attention. Against this background, the new agricultural production model has put forward higher requirements for agricultural machinery, and promoted the development of agricultural machinery towards intelligence and digitalization. As an important part of it, the automatic driving technology of agricultural machinery is in a rapid development stage, and trajectory tracking technology is one of the key links in automatic driving technology.

[0003] At present, the mainstream trajectory tracking control algorithms include PID control, fuzzy control, linear quadratic regulator (LQR) control and model predictive control (MPC). The PID control algorithm does not rely on an accurate mathematical model, but its parameter adjustment process is cumbersome and requires experienced parameter adjustment personnel. Fuzzy control also does not require a mathematical model of the controlled object, but relies on a rule base formulated by experts and has shortcomings in data stability.

[0004] In contrast, LQR control and model predictive control (MPC) rely on the mathematical model of the controlled object. LQR control is mainly applicable to linear systems and achieves optimal control by solving the optimal gain matrix, but its control effect is highly dependent on the accuracy of the model. MPC can better handle nonlinear systems, dynamically adjust the control strategy through rolling optimization, and has strong adaptability to system changes.

[0005] Compared with structured roads, paddy fields are uneven, irregular, muddy and slippery. In particular, due to the deformability of paddy field soil (i.e., the plasticity and rheological properties of the soil), when the vehicle turns, the deformed soil will exert lateral force on the wheels, resulting in a phenomenon similar to tire sideways deviation. This phenomenon can easily cause the agricultural machinery to deviate from the track and form a large lateral error. Summary of the invention

[0006] The purpose of the present invention is to provide a trajectory tracking control system and method for paddy field vehicles. The method can dynamically adjust the agricultural machinery dynamics model through the reinforcement learning method to provide a more accurate model for MPC, improve the accuracy of agricultural machinery trajectory tracking in the paddy field environment, and provide technical guarantee for efficient paddy field agricultural operations.

[0007] A track tracking control method for a paddy field vehicle, comprising:

[0008] Obtain vehicle location information and operation path requirements;

[0009] Generate a vehicle trajectory according to the position information of the vehicle and the operation path requirement;

[0010] Using an agricultural machinery dynamics model to control vehicle operation according to the vehicle trajectory and paddy field parameters;

[0011] A reinforcement learning model is used to optimize the agricultural machinery dynamics model based on the actual and predicted trajectories of the vehicle.

[0012] Preferably, before using the agricultural machinery dynamics model to control the vehicle operation according to the vehicle trajectory and paddy field parameters, it also includes constructing an agricultural machinery dynamics model, specifically:

[0013] The agricultural machinery dynamics model is constructed according to the parameters of paddy fields and is expressed as:

[0014]

[0015] Where y is the displacement in the y-axis direction of the vehicle body coordinate system, is the yaw angle, m is the vehicle mass, v x is the speed of the vehicle along the x-axis in the body coordinate system, a is the front wheelbase of the vehicle, b is the rear wheelbase of the vehicle, I is the moment of inertia of the vehicle, δ f represents the front wheel turning angle, is the soil response stiffness of the front wheel, R ar The soil response stiffness for the rear wheels.

[0016] Preferably, the adopting of the agricultural machinery dynamics model to control the vehicle operation according to the vehicle trajectory and paddy field parameters comprises:

[0017] Optimize the cost function and constraints, and send the optimal front wheel steering angle control to the vehicle controller:

[0018] The cost function is expressed as:

[0019]

[0020] Where, N p represents the prediction time domain, e rr represents the trajectory error sequence from k to k+1 predicted at time k, Q is the positive definite matrix of error weights, N c is the control time domain, Δδ is the control increment sequence from time k to time k+1 predicted at time k, R is the positive definite matrix of control increment weights, F is the positive definite matrix of terminal error weights, ε is the relaxation factor, and ρ is the weight of the relaxation factor.

[0021] Preferably, the adopting of the agricultural machinery dynamics model to control the vehicle operation according to the vehicle trajectory and paddy field parameters comprises:

[0022] The constraints are expressed as:

[0023]

[0024] Where δ represents the front wheel turning angle, δ min Indicates the left limit of the front wheel turning angle, δ max represents the right limit of the front wheel steering angle, Δδ is the front wheel steering angle increment, Δδ min is the minimum value of the increment, Δδ max is the maximum value of the increment, and the constraint is determined by the actual vehicle's turning angle limit and angular velocity.

[0025] Preferably, before the use of the reinforcement learning model to optimize the agricultural machinery dynamics model according to the actual trajectory and predicted trajectory of the vehicle, it also includes constructing a reinforcement learning model, specifically:

[0026] Build an Actor network: use a three-dimensional state input layer, two fully connected layers as hidden layers, and a two-dimensional action output layer. The first hidden layer consists of 128 neurons and the second hidden layer consists of 64 neurons. The hidden layer activation function of the Actor network is LeakyReLU, and the output layer activation function is Tanh;

[0027] The Critic network is constructed with two input layers, two fully connected hidden layers and one output layer. The first hidden layer is behind the three-dimensional state input layer. The first hidden layer is composed of 128 neurons and is a fully connected layer. The two-dimensional action input layer is connected to the first hidden layer and the second hidden layer is composed of 64 neurons. The output layer outputs the value function of the action. The hidden layer activation function of the Critic network is LeakyReLU, and the output layer activation function is Identity.

[0028] Preferably, the method of optimizing the agricultural machinery dynamics model according to the actual trajectory and predicted trajectory of the vehicle using a reinforcement learning model includes:

[0029] Initialize the weights and biases of the Actor network and Critic network, and initialize the experience replay buffer;

[0030] Collect the vehicle longitudinal speed, reference trajectory curvature, the error between the model state estimated position and the actual vehicle position, the front wheel soil response stiffness, the rear wheel soil response stiffness, the reward, and the next state and store them in the experience playback buffer;

[0031] Randomly sample a small batch of data from the experience replay buffer and calculate the target Q value, that is: y = r w +γQ Target (s t+1 ,πTarget (s t+1 )), where r w is the reward, γ is the discount factor, s t+1 is the next state, π target (s t+1 ) is the target Actor network in the next state s t+1 Next action.

[0032] Calculate the loss of the Critic network;

[0033] Calculate the loss of the Actor network;

[0034] Update the target network.

[0035] Preferably, the calculation of the loss of the Critic network includes:

[0036]

[0037] Back propagate and update the parameters of the Critic network. N is the number of data randomly extracted from the experience replay buffer, which is a hyperparameter. i is the target Q value, Q(s i ,a i ) Current Critic network state s i and action a i Q-value prediction.

[0038] Preferably, the calculating the loss of the Actor network includes:

[0039]

[0040] The Actor network loss is obtained by backpropagating the output of the Critic network. The negative sign indicates that the loss function is minimized, which is actually maximizing the output of the Critic network. The gradient descent method is used to minimize the loss function and update the weights of the Actor network.

[0041] Preferably, the update target network includes:

[0042] The parameter update of the target network is expressed as: θ target ←τθ+(1-τ)θ target ;

[0043] Among them, θ represents the current network parameters, θ target represents the target network parameters, and τ is a constant used to control the update speed.

[0044] A track tracking control system for a paddy field vehicle, comprising:

[0045] A data acquisition module is used to obtain the vehicle's location information and operation path requirements;

[0046] A trajectory prediction module, used to generate a vehicle trajectory according to the position information of the vehicle and the operation path requirements;

[0047] A trajectory adjustment module, for controlling the vehicle operation according to the vehicle trajectory and paddy field parameters using an agricultural machinery dynamics model;

[0048] The reinforcement learning module is used to optimize the agricultural machinery dynamics model based on the actual trajectory and predicted trajectory of the vehicle using the reinforcement learning model.

[0049] The beneficial effects of the present invention are: 1. The present invention establishes an agricultural machinery dynamics model that takes into account paddy field soil deformation, providing a more accurate model for model predictive control; 2. Since the soil response stiffness is difficult to calibrate and is significantly affected by the driving road surface, the present invention uses the idea of ​​reinforcement learning to continuously improve the soil response characteristic parameter table, so that the vehicle can be driven more and more accurately, and can provide initial values ​​with better performance for other scenes with similar working conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0052] Figure 1 is a flow chart of the present invention;

[0053] Figure 2 A schematic diagram of the Actor neural network of the present invention;

[0054] Figure 3 Schematic diagram of the Critic neural network of the present invention. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0057] In addition, the descriptions of "first", "second", etc. in the present invention are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0058] Compared with structured roads, paddy fields are uneven, irregular, muddy and slippery. In particular, due to the deformability of paddy field soil (i.e., the plasticity and rheological properties of the soil), when the vehicle turns, the deformed soil will exert lateral force on the wheels, resulting in a phenomenon similar to tire sideways deviation. This phenomenon can easily cause the agricultural machinery to deviate from the track and form a large lateral error.

[0059] The present invention establishes an agricultural machinery dynamics model that takes paddy field soil deformation into consideration, and provides a more accurate model for model predictive control. Since soil response stiffness is difficult to calibrate and is significantly affected by the driving road surface, the present invention utilizes the idea of ​​reinforcement learning to continuously improve the soil response characteristic parameter table, so that the vehicle can be driven more and more accurately, and can provide initial values ​​with better performance for other scenes with similar working conditions.

[0060] Example 1

[0061] A track tracking control method for a paddy field vehicle, comprising:

[0062] S100, obtaining vehicle location information and operation path requirements;

[0063] S200, generating a vehicle trajectory according to the vehicle's location information and the operation path requirement;

[0064] The trajectory planning module uses a quintic polynomial to fit and optimize the local path that cannot be tracked by the vehicle hardware performance and provides the local path curvature composed of discrete trajectory points, the coordinates of each point, and nearby trajectory points.

[0065] S300, which uses a dynamic model of agricultural machinery to control vehicle operations based on vehicle trajectory and paddy field parameters;

[0066] The vehicle information obtained is the vehicle's current longitudinal speed and front wheel turning angle, and the trajectory information is the curvature of the reference trajectory point at the current moment.

[0067] S400, uses a reinforcement learning model to optimize the agricultural machinery dynamics model based on the vehicle's actual and predicted trajectories.

[0068] The present invention replaces the tire cornering stiffness in the conventional vehicle dynamics model with the soil response stiffness. However, since the soil response stiffness is difficult to calibrate and is significantly affected by the driving road surface, the present invention uses reinforcement learning technology to dynamically obtain this parameter, thereby establishing a more accurate vehicle mathematical model.

[0069] Preferably, S300, before using the agricultural machinery dynamics model to control the vehicle operation according to the vehicle trajectory and paddy field parameters, also includes constructing the agricultural machinery dynamics model, specifically:

[0070] The agricultural machinery dynamics model is constructed according to the parameters of paddy fields and is expressed as:

[0071]

[0072] Where y is the displacement in the y-axis direction of the vehicle body coordinate system, is the yaw angle, m is the vehicle mass, v x is the speed of the vehicle along the x-axis in the body coordinate system, a is the front wheelbase of the vehicle, b is the rear wheelbase of the vehicle, I is the moment of inertia of the vehicle, δ f represents the front wheel turning angle, is the soil response stiffness of the front wheel, R ar The soil response stiffness for the rear wheels.

[0073] Preferably, S300, using the agricultural machinery dynamics model to control the vehicle operation according to the vehicle trajectory and paddy field parameters includes:

[0074] Optimize the cost function and constraints, and send the optimal front wheel steering angle control to the vehicle controller:

[0075] The cost function is expressed as:

[0076]

[0077] Where, N p represents the prediction time domain, e rr represents the trajectory error sequence from k to k+1 predicted at time k, Q is the positive definite matrix of error weights, N c is the control time domain, Δδ is the control increment sequence from time k to time k+1 predicted at time k, R is the positive definite matrix of control increment weights, F is the positive definite matrix of terminal error weights, ε is the relaxation factor, and ρ is the weight of the relaxation factor.

[0078] Preferably, S300, using the agricultural machinery dynamics model to control the vehicle operation according to the vehicle trajectory and paddy field parameters includes:

[0079] The constraints are expressed as:

[0080]

[0081] Where δ represents the front wheel turning angle, δ min Indicates the left limit of the front wheel turning angle, δ max represents the right limit of the front wheel steering angle, Δδ is the front wheel steering angle increment, Δδ min is the minimum value of the increment, Δδ max is the maximum value of the increment, and the constraint is determined by the actual vehicle's turning angle limit and angular velocity.

[0082] Preferably, S400, before using the reinforcement learning model to optimize the agricultural machinery dynamics model according to the actual trajectory and predicted trajectory of the vehicle, also includes constructing a reinforcement learning model, specifically:

[0083] Build an Actor network: use a three-dimensional state input layer, two fully connected layers as hidden layers, and a two-dimensional action output layer. The first hidden layer consists of 128 neurons and the second hidden layer consists of 64 neurons. The hidden layer activation function of the Actor network is LeakyReLU, and the output layer activation function is Tanh;

[0084] The Critic network is constructed with two input layers, two fully connected hidden layers and one output layer. The first hidden layer is behind the three-dimensional state input layer. The first hidden layer is composed of 128 neurons and is a fully connected layer. The two-dimensional action input layer is connected to the first hidden layer and the second hidden layer is composed of 64 neurons. The output layer outputs the value function of the action. The hidden layer activation function of the Critic network is LeakyReLU, and the output layer activation function is Identity.

[0085] The observation states of the reinforcement learning module are defined as the vehicle longitudinal speed, the reference trajectory curvature, and the error between the model state estimated position and the actual vehicle position. The action set is the soil response stiffness of the front wheel and the soil response stiffness of the rear wheel. The soil response characteristic parameter table is constructed by saving the soil response stiffness at different longitudinal speeds of the vehicle and the reference trajectory curvature.

[0086] Using the Actor-Critic method, such as Figure 2As shown in the figure, the Actor network consists of a three-dimensional state input layer, two fully connected layers as hidden layers, and a two-dimensional action output layer. The first hidden layer consists of 128 neurons, and the second hidden layer consists of 64 neurons. The hidden layer activation function of the Actor network is LeakyReLU, and the output layer activation function is Tanh.

[0087] like Figure 3 As shown in the figure, the Critic network is built. The Critic network consists of two input layers, two fully connected hidden layers and an output layer. The first hidden layer is behind the three-dimensional state input layer. This layer consists of 128 neurons and is a fully connected layer. The two-dimensional action input layer is connected to the first hidden layer and the second hidden layer consists of 64 neurons. The output layer outputs the value function of the action. The hidden layer activation function of the Critic network is LeakyReLU, and the output layer activation function is Identity.

[0088] The reward function is shown in Table 1, which divides the error range between the model state estimated position and the actual vehicle position into less than,0.03m, less than 0.05m, less than 0.08m, less than 0.10m, less than 0.15m, less than 0.20m, greater than 0.20m, greater than 0.30m, greater than 0.40m, and greater than 0.50m.

[0089] Table 1 Reward Distribution Table

[0090]

[0091] Preferably, in S400, optimizing the agricultural machinery dynamics model according to the actual trajectory and predicted trajectory of the vehicle using a reinforcement learning model includes:

[0092] Initialize the weights and biases of the Actor network and Critic network, and initialize the experience replay buffer;

[0093] Initialize the weights and biases of the Actor network and the Critic network, initialize the target network, which is a copy of the Actor network and the Critic network to stabilize the training process. Initialize the experience replay buffer to store the sampled data.

[0094] Collect the vehicle longitudinal speed, reference trajectory curvature, the error between the model state estimated position and the actual vehicle position, the front wheel soil response stiffness, the rear wheel soil response stiffness, the reward, and the next state and store them in the experience playback buffer;

[0095] Randomly sample a small batch of data from the experience replay buffer and calculate the target Q value, that is: y = r w +γQ Target (s t+1 ,πTarget (s t+1 )), where r w is the reward, γ is the discount factor, s t+1 is the next state, π target (s t+1 ) is the target Actor network in the next state s t+1 Next action.

[0096] Randomly sample a small batch of data from the experience replay buffer, including the current state, action, reward, and next state. Calculate the target Q value. The target Q value is calculated by the target critic network, that is: y = r w +γQ Target (s t+1 ,π Target (s t+1 )), where r w is the reward, γ is the discount factor, s t+1 is the next state, π target (s t+1 ) is the target Actor network in the next state s t+1 Next action.

[0097] Calculate the loss of the Critic network;

[0098] Calculate the loss of the Actor network;

[0099] Update the target network.

[0100] Preferably, calculating the loss of the Critic network includes:

[0101]

[0102] Back propagate and update the parameters of the Critic network. N is the number of data randomly extracted from the experience replay buffer, which is a hyperparameter. i is the target Q value, Q(s i ,a i ) Current Critic network state s i and action a i Q-value prediction.

[0103] Calculate the loss of the Critic network, back-propagate and update the parameters of the Critic network.

[0104] Preferably, calculating the loss of the Actor network includes:

[0105]

[0106] The Actor network loss is obtained by backpropagating the output of the Critic network. The negative sign indicates that the loss function is minimized, which is actually maximizing the output of the Critic network. The gradient descent method is used to minimize the loss function and update the weights of the Actor network.

[0107] Preferably, updating the target network includes:

[0108] The parameter update of the target network is expressed as: θ target ←τθ+(1-τ)θ target ;

[0109] Among them, θ represents the current network parameters, θ target represents the target network parameters, and τ is a constant used to control the update speed.

[0110] In order to stabilize the training process, it is necessary to use a soft update method to update the target network. τ is a very small value (such as 0.001) used to control the speed of the update.

[0111] Example 2

[0112] A track tracking control system for a paddy field vehicle, comprising:

[0113] A data acquisition module is used to obtain the vehicle's location information and operation path requirements;

[0114] A trajectory prediction module is used to generate a vehicle trajectory based on the vehicle's location information and operation path requirements;

[0115] A trajectory adjustment module is used to control vehicle operation according to vehicle trajectory and paddy field parameters using an agricultural machinery dynamics model;

[0116] The reinforcement learning module is used to optimize the agricultural machinery dynamics model based on the actual trajectory and predicted trajectory of the vehicle using the reinforcement learning model.

[0117] The present invention establishes an agricultural machinery dynamics model that takes paddy field soil deformation into consideration, and provides a more accurate model for model predictive control. Since soil response stiffness is difficult to calibrate and is significantly affected by the driving road surface, the present invention utilizes the idea of ​​reinforcement learning to continuously improve the soil response characteristic parameter table, so that the vehicle can be driven more and more accurately, and can provide initial values ​​with better performance for other scenes with similar working conditions.

[0118] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A track tracking control method for a paddy field vehicle, characterized in that: include: Obtain vehicle location information and operation path requirements; Generate a vehicle trajectory according to the position information of the vehicle and the operation path requirement; Using an agricultural machinery dynamics model to control vehicle operation according to the vehicle trajectory and paddy field parameters; A reinforcement learning model is used to optimize the agricultural machinery dynamics model based on the actual and predicted trajectories of the vehicle.

2. The track tracking control method for a paddy field vehicle according to claim 1, characterized in that: Before the agricultural machinery dynamics model is used to control the vehicle operation according to the vehicle trajectory and paddy field parameters, the agricultural machinery dynamics model is constructed, specifically: The agricultural machinery dynamics model is constructed according to the parameters of paddy fields and is expressed as: Where y is the displacement in the y-axis direction of the vehicle body coordinate system, is the yaw angle, m is the vehicle mass, v x is the speed of the vehicle along the x-axis in the body coordinate system, a is the front wheelbase of the vehicle, b is the rear wheelbase of the vehicle, I is the moment of inertia of the vehicle, δ f represents the front wheel turning angle, is the soil response stiffness of the front wheel, R ar The soil response stiffness for the rear wheels.

3. The track tracking control method for a paddy field vehicle according to claim 1, characterized in that: The use of the agricultural machinery dynamics model to control the vehicle operation according to the vehicle trajectory and paddy field parameters includes: Optimize the cost function and constraints, and send the optimal front wheel steering angle control to the vehicle controller: The cost function is expressed as: Where, N p represents the prediction time domain, e rr represents the trajectory error sequence from k to k+1 predicted at time k, Q is the positive definite matrix of error weights, N c is the control time domain, Δδ is the control increment sequence from time k to time k+1 predicted at time k, R is the positive definite matrix of control increment weights, F is the positive definite matrix of terminal error weights, ε is the relaxation factor, and ρ is the weight of the relaxation factor.

4. The track tracking control method for a paddy field vehicle according to claim 3, characterized in that: The use of the agricultural machinery dynamics model to control the vehicle operation according to the vehicle trajectory and paddy field parameters includes: The constraints are expressed as: Where δ represents the front wheel turning angle, δ min Indicates the left limit of the front wheel turning angle, δ max represents the right limit of the front wheel steering angle, Δδ is the front wheel steering angle increment, Δδ min is the minimum value of the increment, Δδ max is the maximum value of the increment, and the constraint is determined by the actual vehicle's turning angle limit and angular velocity.

5. The track tracking control method for a paddy field vehicle according to claim 3, characterized in that: Before the reinforcement learning model is used to optimize the agricultural machinery dynamics model according to the actual trajectory and predicted trajectory of the vehicle, the reinforcement learning model is also constructed, specifically: Build an Actor network: use a three-dimensional state input layer, two fully connected layers as hidden layers, and a two-dimensional action output layer. The first hidden layer consists of 128 neurons, and the second hidden layer consists of 64 neurons. The hidden layer activation function of the Actor network is LeakyReLU, and the output layer activation function is Tanh. The Critic network is constructed with two input layers, two fully connected hidden layers and one output layer. The first hidden layer is behind the three-dimensional state input layer. The first hidden layer is composed of 128 neurons and is a fully connected layer. The two-dimensional action input layer is connected to the first hidden layer and the second hidden layer is composed of 64 neurons. The output layer outputs the value function of the action. The hidden layer activation function of the Critic network is LeakyReLU, and the output layer activation function is Identity.

6. A paddy field vehicle trajectory tracking control method according to claim 5, characterized in that: The use of a reinforcement learning model to optimize the agricultural machinery dynamics model according to the actual trajectory and predicted trajectory of the vehicle includes: Initialize the weights and biases of the Actor network and Critic network, and initialize the experience replay buffer; Collect the vehicle longitudinal speed, reference trajectory curvature, the error between the model state estimated position and the actual vehicle position, the front wheel soil response stiffness, the rear wheel soil response stiffness, the reward, and the next state and store them in the experience playback buffer; Randomly sample a small batch of data from the experience replay buffer and calculate the target Q value, that is: y = r w +γQ Target (s t+1 ,π Target (s t+1 )), where r w is the reward, γ is the discount factor, s t+1 is the next state, π target (s t+1 ) is the target Actor network in the next state s t+1 The action below; Calculate the loss of the Critic network; Calculate the loss of the Actor network; Update the target network.

7. A paddy field vehicle trajectory tracking control method according to claim 6, characterized in that: The calculation of the loss of the Critic network includes: Back propagate and update the parameters of the Critic network. N is the number of data randomly extracted from the experience replay buffer, which is a hyperparameter. i is the target Q value, Q(s i ,a i ) Current Critic network state s i and action a i Q-value prediction.

8. The track tracking control method for a paddy field vehicle according to claim 7, characterized in that: The calculation of the loss of the Actor network includes: The Actor network loss is obtained by backpropagating the output of the Critic network. The negative sign indicates that the loss function is minimized, which is actually maximizing the output of the Critic network. The gradient descent method is used to minimize the loss function and update the weights of the Actor network.

9. A paddy field vehicle trajectory tracking control method according to claim 6, characterized in that: The update target network includes: The parameter update of the target network is expressed as: θ target ←τθ+(1-τ)θ target ; Among them, θ represents the current network parameters, θ target represents the target network parameters, and τ is a constant used to control the update speed.

10. A track tracking control system for paddy field vehicles, characterized in that: include: A data acquisition module is used to obtain the vehicle's location information and operation path requirements; A trajectory prediction module, used to generate a vehicle trajectory according to the position information of the vehicle and the operation path requirements; A trajectory adjustment module, for controlling the vehicle operation according to the vehicle trajectory and paddy field parameters using an agricultural machinery dynamics model; The reinforcement learning module is used to optimize the agricultural machinery dynamics model based on the actual trajectory and predicted trajectory of the vehicle using the reinforcement learning model.

Images