A trajectory tracking control system and method for a paddy field vehicle

Abstract

The application aims to provide a track tracking control system and method of a paddy field vehicle, the method comprising: obtaining position information and operation path requirements of the vehicle; generating a vehicle track according to the position information and operation path requirements of the vehicle; controlling vehicle operation according to the vehicle track and paddy field parameters by using a farm machine dynamics model; and optimizing the farm machine dynamics model according to an actual track and a predicted track of the vehicle by using a reinforcement learning model. The application can dynamically adjust the farm machine dynamics model by the method of reinforcement learning to provide a more accurate model for MPC, improve the track tracking precision of the farm machine in the paddy field environment, and provide a technical guarantee for efficient paddy field agricultural operation.

Description

Technical Field

[0001] This invention relates to the field of farmland vehicle control technology, specifically to a trajectory tracking control system and method for paddy field vehicles. Background Technology

[0002] In recent years, the comprehensive implementation of the rural revitalization strategy has significantly accelerated the modernization of agriculture and rural areas. The application of modern information technology in agriculture has become increasingly profound, leading to widespread attention on smart agriculture and digital agriculture. Against this backdrop, new agricultural production models have placed higher demands on agricultural machinery, driving its development towards intelligence and digitalization. Autonomous driving technology for agricultural machinery, as a crucial component, is in a phase of rapid development, and trajectory tracking technology is one of the key elements within this technology.

[0003] Currently, the most mainstream trajectory tracking control algorithms include PID control, fuzzy control, linear quadratic regulator (LQR) control, and model predictive control (MPC). PID control algorithms do not rely on precise mathematical models, but their parameter tuning process is cumbersome and requires experienced technicians. Fuzzy control also does not require a mathematical model of the controlled object, but it relies on a rule base established by experts and suffers from shortcomings in data stability.

[0004] In contrast, LQR control and model predictive control (MPC) rely on the mathematical model of the controlled object. LQR control is mainly suitable for linear systems, achieving optimal control by solving for the optimal gain matrix, but its control effect is highly dependent on the accuracy of the model. MPC, on the other hand, can better handle nonlinear systems, dynamically adjusting the control strategy through rolling optimization, and has strong adaptability to system changes.

[0005] Compared to structured roads, paddy field environments are characterized by unevenness, irregularity, and muddy, slippery conditions. In particular, due to the deformability of paddy field soil (i.e., the plasticity and rheological properties of the soil), when vehicles turn, the deformed soil exerts a lateral force on the wheels, causing a phenomenon similar to tire slippage. This phenomenon can easily cause agricultural machinery to deviate from its trajectory, resulting in significant lateral errors. Summary of the Invention

[0006] The purpose of this invention is to provide a trajectory tracking control system and method for paddy field vehicles. This method can dynamically adjust the agricultural machinery dynamics model through reinforcement learning to provide a more accurate model for MPC, thereby improving the accuracy of trajectory tracking of agricultural machinery in paddy field environments and providing technical support for efficient paddy field agricultural operations.

[0007] A trajectory tracking control method for paddy field vehicles includes:

[0008] Obtain vehicle location information and operation route requirements;

[0009] Generate vehicle trajectory based on vehicle location information and operation path requirements;

[0010] The agricultural machinery dynamics model is used to control vehicle operations based on the vehicle trajectory and paddy field parameters.

[0011] A reinforcement learning model is used to optimize the agricultural machinery dynamics model based on the vehicle's actual trajectory and predicted trajectory.

[0012] Preferably, before controlling vehicle operation based on the vehicle trajectory and paddy field parameters using the agricultural machinery dynamics model, the process further includes constructing the agricultural machinery dynamics model, specifically:

[0013] Based on the parameters of the paddy field, a dynamic model of agricultural machinery is constructed, which is expressed as:

[0014]

[0015] In the formula, y represents the displacement along the y-axis in the vehicle coordinate system. yaw angle is , m is the vehicle mass, v x Let be the vehicle's velocity along the x-axis in the vehicle's coordinate system, 'a' be the front wheelbase, 'b' be the rear wheelbase, 'I' be the vehicle's moment of inertia, and 'δ' be the vehicle's rotational inertia. f Indicates the front wheel steering angle. R is the soil response stiffness of the front wheel. ar The soil response stiffness of the rear wheel.

[0016] Preferably, the step of controlling vehicle operation using an agricultural machinery dynamics model based on the vehicle trajectory and paddy field parameters includes:

[0017] The optimization process is performed based on the cost function and constraints, and the optimal front wheel steering angle control value is generated and sent to the vehicle's controller.

[0018] The cost function is expressed as:

[0019]

[0020] In the formula, N p Indicates the prediction time domain, e rr Let N represent the trajectory error sequence predicted at time k from time k to k+1, where Q is the positive definite matrix of the error weights, and N is the number of time points. c For the control time domain, Δδ is the control increment sequence predicted at time k from time k to time k+1, R is the positive definite matrix of control increment weights, F is the positive definite matrix of terminal error weights, ε is the relaxation factor, and ρ is the weight of the relaxation factor.

[0021] Preferably, the step of controlling vehicle operation using an agricultural machinery dynamics model based on the vehicle trajectory and paddy field parameters includes:

[0022] The constraint is expressed as:

[0023]

[0024] In the formula, δ represents the front wheel steering angle, δ min δ represents the left limit value of the front wheel steering angle. max This represents the right limit value of the front wheel steering angle, where Δδ is the increment of the front wheel steering angle. min The minimum value of the increment, Δδ max The maximum value of the increment is determined by the actual vehicle's turning angle limit and angular velocity.

[0025] Preferably, before optimizing the agricultural machinery dynamics model using a reinforcement learning model based on the vehicle's actual trajectory and predicted trajectory, the method further includes constructing a reinforcement learning model, specifically:

[0026] The Actor network is constructed using a 3D state input layer, two fully connected layers as hidden layers, and a 2D action output layer. The first hidden layer consists of 128 neurons, and the second hidden layer consists of 64 neurons. The activation function for the hidden layers of the Actor network is LeakyReLU, and the activation function for the output layer is Tanh.

[0027] A Critic network is constructed, consisting of two input layers, two fully connected hidden layers, and one output layer. The first hidden layer, composed of 128 neurons, is located after the 3D state input layer and is fully connected. The 2D action input layer is connected to the first hidden layer and then to the second hidden layer, which consists of 64 neurons. The output layer outputs the action value function. The activation function of the hidden layers in the Critic network is LeakyReLU, and the activation function of the output layer is Identity.

[0028] Preferably, the step of using a reinforcement learning model to optimize the agricultural machinery dynamics model based on the vehicle's actual trajectory and predicted trajectory includes:

[0029] Initialize the weights and biases of the Actor and Critic networks, and initialize the experience replay buffer;

[0030] Collect vehicle longitudinal velocity, reference trajectory curvature, error between model state estimated position and actual vehicle position, front wheel soil response stiffness, rear wheel soil response stiffness, reward, and next state, and store them in the experience replay buffer.

[0031] A small batch of data is randomly sampled from the experience replay buffer, and the target Q value is calculated, i.e.: y = r w +γQ Target (s t+1 ,πTarget (s t+1 ), where r w It is the reward, γ is the discount factor, and s t+1 The next state is π. target (s t+1 ) is the target Actor network in the next state s t+1 The following action.

[0032] Calculate the loss of the Critic network;

[0033] Calculate the loss of the Actor network;

[0034] Update the target network.

[0035] Preferably, the calculation of the loss of the Critic network includes:

[0036]

[0037] Backpropagation updates the parameters of the Critic network, where N is the number of data points randomly drawn from the experience replay buffer, and y is a hyperparameter. i Let Q(s) be the target Q value. i ,a i The current Critic network has state s i and action a i Q-value prediction.

[0038] Preferably, the calculation of the loss of the Actor network includes:

[0039]

[0040] The Actor network loss is obtained through backpropagation of the Critic network output. The negative sign indicates minimizing the loss function, which actually maximizes the Critic network output. The gradient descent method is used to minimize the loss function and update the weights of the Actor network.

[0041] Preferably, the target network to be updated includes:

[0042] The parameter update of the target network is represented as: θ target ←τθ+(1-τ)θ target ;

[0043] Where θ represents the current network parameters, θ target This represents the target network parameters, where τ is a constant used to control the update speed.

[0044] A trajectory tracking control system for paddy field vehicles includes:

[0045] The data acquisition module is used to acquire vehicle location information and operation route requirements;

[0046] The trajectory prediction module is used to generate a vehicle trajectory based on the vehicle's location information and the work path requirements.

[0047] The trajectory adjustment module is used to control vehicle operation based on the vehicle trajectory and paddy field parameters using an agricultural machinery dynamics model.

[0048] The reinforcement learning module is used to optimize the agricultural machinery dynamics model based on the vehicle's actual trajectory and predicted trajectory using a reinforcement learning model.

[0049] The beneficial effects of this invention are as follows: 1. This invention establishes an agricultural machinery dynamics model that considers the deformation of paddy field soil, providing a more accurate model for model predictive control; 2. Since the soil response stiffness is difficult to calibrate and is significantly affected by the road surface, this invention uses the idea of ​​reinforcement learning to continuously update the soil response characteristic parameter table, making the vehicle more and more accurate as it drives, and can provide better initial values ​​for other similar working conditions. Attached Figure Description

[0050] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a flowchart of the present invention;

[0053] Figure 2 This is a schematic diagram of the Actor neural network of the present invention;

[0054] Figure 3 This is a schematic diagram of the Critic neural network of the present invention. Detailed Implementation

[0055] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0056] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0057] Furthermore, the use of terms such as "first" and "second" in this invention is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" and "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but only on the basis of being achievable by those skilled in the art. When the combination of technical solutions is contradictory or impossible to implement, such a combination of technical solutions should be considered non-existent and not within the scope of protection claimed by this invention.

[0058] Compared to structured roads, paddy field environments are characterized by unevenness, irregularity, and muddy, slippery conditions. In particular, due to the deformability of paddy field soil (i.e., the plasticity and rheological properties of the soil), when vehicles turn, the deformed soil exerts a lateral force on the wheels, causing a phenomenon similar to tire slippage. This phenomenon can easily cause agricultural machinery to deviate from its trajectory, resulting in significant lateral errors.

[0059] This invention establishes a dynamic model for agricultural machinery that considers soil deformation in paddy fields, providing a more accurate model for predictive control. Because soil response stiffness is difficult to calibrate and is significantly affected by the road surface, this invention utilizes reinforcement learning to continuously update the soil response characteristic parameter table, making the vehicle more and more accurate with each drive, and can provide better initial values ​​for other similar operating conditions.

[0060] Example 1

[0061] A trajectory tracking control method for paddy field vehicles includes:

[0062] S100, obtains vehicle location information and operation route requirements;

[0063] S200 generates vehicle trajectories based on vehicle location information and work path requirements;

[0064] The trajectory planning module fits and optimizes local paths that the vehicle hardware performance cannot track by using a fifth-order polynomial, and provides discrete trajectory points and the coordinates of each point and the curvature of the local path composed of nearby trajectory points.

[0065] The S300 uses an agricultural machinery dynamics model to control vehicle operations based on vehicle trajectory and paddy field parameters;

[0066] The acquired vehicle information consists of the vehicle's current longitudinal speed and front wheel angle, while the trajectory information is the curvature of the reference trajectory point at the current moment.

[0067] The S400 uses a reinforcement learning model to optimize the agricultural machinery dynamics model based on the vehicle's actual trajectory and predicted trajectory.

[0068] This invention replaces the tire lateral stiffness in conventional vehicle dynamics models with soil response stiffness. However, since soil response stiffness is difficult to calibrate and is significantly affected by the road surface, this invention uses reinforcement learning technology to dynamically obtain this parameter, thereby establishing a more accurate vehicle mathematical model.

[0069] Preferably, S300, before using the agricultural machinery dynamics model to control vehicle operation based on vehicle trajectory and paddy field parameters, further includes constructing the agricultural machinery dynamics model, specifically:

[0070] Based on the parameters of the paddy field, a dynamic model of agricultural machinery is constructed, which is expressed as:

[0071]

[0072] In the formula, y represents the displacement along the y-axis in the vehicle coordinate system. yaw angle is , m is the vehicle mass, v x Let be the vehicle's velocity along the x-axis in the vehicle's coordinate system, 'a' be the front wheelbase, 'b' be the rear wheelbase, 'I' be the vehicle's moment of inertia, and 'δ' be the vehicle's rotational inertia. f Indicates the front wheel steering angle. R is the soil response stiffness of the front wheel. ar The soil response stiffness of the rear wheel.

[0073] Preferably, S300, using an agricultural machinery dynamics model to control vehicle operations based on vehicle trajectory and paddy field parameters includes:

[0074] The optimization process is performed based on the cost function and constraints, and the optimal front wheel steering angle control value is generated and sent to the vehicle's controller.

[0075] The cost function is expressed as:

[0076]

[0077] In the formula, N p Indicates the prediction time domain, e rr Let N represent the trajectory error sequence predicted at time k from time k to k+1, where Q is the positive definite matrix of the error weights, and N is the number of time points. c For the control time domain, Δδ is the control increment sequence predicted at time k from time k to time k+1, R is the positive definite matrix of control increment weights, F is the positive definite matrix of terminal error weights, ε is the relaxation factor, and ρ is the weight of the relaxation factor.

[0078] Preferably, S300, using an agricultural machinery dynamics model to control vehicle operations based on vehicle trajectory and paddy field parameters includes:

[0079] The constraint is expressed as:

[0080]

[0081] In the formula, δ represents the front wheel steering angle, δ min δ represents the left limit value of the front wheel steering angle. max This represents the right limit of the front wheel steering angle, where Δδ is the increment of the front wheel steering angle. min The minimum value of the increment, Δδ max The maximum value of the increment is determined by the actual vehicle's turning angle limit and angular velocity.

[0082] Preferably, in step S400, before optimizing the agricultural machinery dynamics model using a reinforcement learning model based on the vehicle's actual and predicted trajectories, the process further includes constructing a reinforcement learning model, specifically:

[0083] The Actor network is constructed using a 3D state input layer, two fully connected layers as hidden layers, and a 2D action output layer. The first hidden layer consists of 128 neurons, and the second hidden layer consists of 64 neurons. The activation function for the hidden layers of the Actor network is LeakyReLU, and the activation function for the output layer is Tanh.

[0084] A Critic network is constructed, consisting of two input layers, two fully connected hidden layers, and one output layer. The first hidden layer, composed of 128 neurons, is located after the 3D state input layer and is fully connected. The 2D action input layer is connected to the first hidden layer and then to the second hidden layer, which consists of 64 neurons. The output layer outputs the action value function. The activation function of the hidden layers in the Critic network is LeakyReLU, and the activation function of the output layer is Identity.

[0085] The observed states of the reinforcement learning module are defined as vehicle longitudinal speed, reference trajectory curvature, and the error between the model state estimated position and the actual vehicle position. The action set is the soil response stiffness of the front wheel and the soil response stiffness of the rear wheel. A soil response characteristic parameter table is constructed by saving the soil response stiffness through different vehicle longitudinal speeds and reference trajectory curvatures.

[0086] Using the Actor-Critic method, such as Figure 2As shown, the Actor network consists of a 3D state input layer, two fully connected layers as hidden layers, and a 2D action output layer. The first hidden layer consists of 128 neurons, and the second hidden layer consists of 64 neurons. The activation function for the hidden layers of the Actor network is LeakyReLU, and the activation function for the output layer is Tanh.

[0087] like Figure 3 As shown, a Critic network is constructed. The Critic network consists of two input layers, two fully connected hidden layers, and one output layer. The first hidden layer, composed of 128 neurons, is a fully connected layer following the 3D state input layer. The second hidden layer, composed of 64 neurons, connects to the first hidden layer. The output layer outputs the value function of the action. The hidden layer activation function of the Critic network is LeakyReLU, and the output layer activation function is Identity.

[0088] The reward function is shown in Table 1. The error range between the model state estimated position and the actual vehicle position is divided into less than 0.03m, less than 0.05m, less than 0.08m, less than 0.10m, less than 0.15m, less than 0.20m, greater than 0.20m, greater than 0.30m, greater than 0.40m, and greater than 0.50m.

[0089] Table 1 Reward Distribution Table

[0090]

[0091] Preferably, in step S400, the reinforcement learning model is used to optimize the agricultural machinery dynamics model based on the vehicle's actual trajectory and predicted trajectory, including:

[0092] Initialize the weights and biases of the Actor and Critic networks, and initialize the experience replay buffer;

[0093] Initialize the weights and biases of the Actor and Critic networks. Initialize the target network, which is a copy of the Actor and Critic networks, for stabilizing the training process. Initialize the experience replay buffer to store sampled data.

[0094] Collect vehicle longitudinal velocity, reference trajectory curvature, error between model state estimated position and actual vehicle position, front wheel soil response stiffness, rear wheel soil response stiffness, reward, and next state, and store them in the experience replay buffer.

[0095] A small batch of data is randomly sampled from the experience playback buffer, and the target Q value is calculated, i.e.: y = r w +γQ Target (s t+1 ,πTarget (s t+1 ), where r w It is the reward, γ is the discount factor, and s t+1 The next state is π. target (s t+1 ) is the target Actor network in the next state s t+1 The following action.

[0096] A small batch of data is randomly sampled from the experience replay buffer, including the current state, action, reward, and next state. The target Q-value is calculated. The target Q-value is calculated using the target Critic network, i.e.: y = r w +γQ Target (s t+1 ,π Target (s t+1 ), where r w It is the reward, γ is the discount factor, and s t+1 The next state is π. target (s t+1 ) is the target Actor network in the next state s t+1 The following action.

[0097] Calculate the loss of the Critic network;

[0098] Calculate the loss of the Actor network;

[0099] Update the target network.

[0100] Preferably, calculating the loss of the Critic network includes:

[0101]

[0102] Backpropagation updates the parameters of the Critic network, where N is the number of data points randomly drawn from the experience replay buffer, and y is a hyperparameter. i Let Q(s) be the target Q value. i ,a i The current Critic network has state s i and action a i Q-value prediction.

[0103] Calculate the loss of the Critic network, backpropagate, and update the parameters of the Critic network.

[0104] Preferably, calculating the loss of the Actor network includes:

[0105]

[0106] The Actor network loss is obtained through backpropagation of the Critic network output. The negative sign indicates minimizing the loss function, which actually maximizes the Critic network output. The gradient descent method is used to minimize the loss function and update the weights of the Actor network.

[0107] Preferably, updating the target network includes:

[0108] The parameter update of the target network is represented as: θ target ←τθ+(1-τ)θ target ;

[0109] Where θ represents the current network parameters, θ target This represents the target network parameters, where τ is a constant used to control the update speed.

[0110] To stabilize the training process, a soft update method is needed to update the target network. τ is a very small value (e.g., 0.001) used to control the update speed.

[0111] Example 2

[0112] A trajectory tracking control system for paddy field vehicles includes:

[0113] The data acquisition module is used to acquire vehicle location information and operation route requirements;

[0114] The trajectory prediction module is used to generate vehicle trajectories based on vehicle location information and operation path requirements;

[0115] The trajectory adjustment module is used to control vehicle operation based on the vehicle trajectory and paddy field parameters using an agricultural machinery dynamics model.

[0116] The reinforcement learning module is used to optimize the agricultural machinery dynamics model based on the vehicle's actual trajectory and predicted trajectory using a reinforcement learning model.

[0117] This invention establishes a dynamic model for agricultural machinery that considers soil deformation in paddy fields, providing a more accurate model for predictive control. Because soil response stiffness is difficult to calibrate and is significantly affected by the road surface, this invention utilizes reinforcement learning to continuously update the soil response characteristic parameter table, making the vehicle more and more accurate with each drive, and can provide better initial values ​​for other similar operating conditions.

[0118] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A trajectory tracking control method for paddy field vehicles, characterized in that, include: Obtain vehicle location information and work route requirements; Generate vehicle trajectory based on vehicle location information and operation path requirements; The agricultural machinery dynamics model is used to control vehicle operations based on the vehicle trajectory and paddy field parameters. A reinforcement learning model is used to optimize the agricultural machinery dynamics model based on the vehicle's actual trajectory and predicted trajectory. Before using the agricultural machinery dynamics model to control vehicle operations based on the vehicle trajectory and paddy field parameters, the process also includes constructing the agricultural machinery dynamics model, specifically: Based on the parameters of the paddy field, a dynamic model of agricultural machinery is constructed, which is expressed as: ； In the formula On the vehicle body coordinate system Displacement in the axial direction, For the horizontal swing angle, For the overall vehicle weight, For the vehicle's coordinate system along The speed of the shaft, This refers to the vehicle's front wheelbase. This refers to the rear wheelbase of the vehicle. For the vehicle's moment of inertia, Indicates the front wheel steering angle. For the soil response stiffness of the front wheel, For the soil response stiffness of the rear wheel; The method of controlling vehicle operation based on the vehicle trajectory and paddy field parameters using an agricultural machinery dynamics model includes: The optimal front wheel steering angle control value is then sent to the vehicle controller based on the cost function and constraints. The cost function is expressed as: ； In the formula, Indicates the prediction time domain. express Time prediction arrive Trajectory error sequence at time step The error weights are positive definite matrices. To control the time domain, for Time prediction arrive The increase in front wheel steering angle at any given moment To control the positive definite matrix of the incremental weights, This is a positive definite matrix of terminal error weights. As a relaxation factor, The weights of the relaxation factor.

2. The trajectory tracking control method for paddy field vehicles according to claim 1, characterized in that, The method of controlling vehicle operation based on the vehicle trajectory and paddy field parameters using an agricultural machinery dynamics model includes: The constraint is expressed as: ； In the formula Indicates the front wheel steering angle. This represents the left limit value of the front wheel steering angle. This represents the rightmost limit of the front wheel steering angle. For the front wheel steering angle increment, The minimum value of the increment. The maximum value of the increment is determined by the actual vehicle's turning angle limit and angular velocity.

3. The trajectory tracking control method for paddy field vehicles according to claim 1, characterized in that, Before optimizing the agricultural machinery dynamics model using a reinforcement learning model based on the vehicle's actual and predicted trajectories, the process also includes constructing a reinforcement learning model, specifically: The Actor network is constructed using a three-dimensional state input layer, two fully connected layers as hidden layers, and a two-dimensional action output layer. The first hidden layer consists of 128 neurons, and the second hidden layer consists of 64 neurons. The activation function for the hidden layers of the Actor network is LeakyReLU, and the activation function for the output layer is Tanh. A Critic network is constructed, consisting of two input layers, two fully connected hidden layers, and one output layer. The first hidden layer, composed of 128 neurons, is located after the 3D state input layer and is fully connected. The 2D action input layer is connected to the first hidden layer and is connected to the second hidden layer, which consists of 64 neurons. The output layer outputs the action value function. The activation function of the hidden layers in the Critic network is LeakyReLU, and the activation function of the output layer is Identity.

4. The trajectory tracking control method for paddy field vehicles according to claim 3, characterized in that, The optimization of the agricultural machinery dynamics model using a reinforcement learning model based on the vehicle's actual trajectory and predicted trajectory includes: Initialize the weights and biases of the Actor and Critic networks, and initialize the experience replay buffer; Collect vehicle longitudinal velocity, reference trajectory curvature, error between model state estimated position and actual vehicle position, front wheel soil response stiffness, rear wheel soil response stiffness, reward, and next state, and store them in the experience replay buffer. A small batch of data is randomly sampled from the experience replay buffer, and the target Q value is calculated, i.e.: ,in, It's a reward. It is a discount factor. The next state is... Is the target Actor network in the next state? The following action; Calculate the loss of the Critic network; Calculate the loss of the Actor network; Update the target network.

5. The trajectory tracking control method for paddy field vehicles according to claim 4, characterized in that, The calculation of the loss of the Critic network includes: ； Backpropagation is performed to update the parameters of the Critic network. The number of data points randomly drawn from the experience replay buffer is a hyperparameter. For the target Q value, Current Critic network state and actions Q-value prediction.

6. The trajectory tracking control method for paddy field vehicles according to claim 5, characterized in that, The calculation of the loss of the Actor network includes: ； The Actor network loss is obtained through backpropagation of the Critic network output. The negative sign indicates minimizing the loss function, which actually maximizes the Critic network output. The gradient descent method is used to minimize the loss function and update the weights of the Actor network.

7. The trajectory tracking control method for paddy field vehicles according to claim 4, characterized in that, The target network for updating includes: The parameter update of the target network is represented as: ; in, Indicates the current network parameters. Indicates the target network parameters. It is a constant used to control the update speed.

8. A trajectory tracking control system for paddy field vehicles, characterized in that, include: The data acquisition module is used to acquire vehicle location information and operation route requirements; The trajectory prediction module is used to generate a vehicle trajectory based on the vehicle's location information and the work path requirements. The trajectory adjustment module is used to control vehicle operation based on the vehicle trajectory and paddy field parameters using an agricultural machinery dynamics model. The reinforcement learning module is used to optimize the agricultural machinery dynamics model based on the vehicle's actual trajectory and predicted trajectory using a reinforcement learning model. Before controlling vehicle operations based on the vehicle trajectory and paddy field parameters using the agricultural machinery dynamics model, the module further includes constructing the agricultural machinery dynamics model, specifically: Based on the parameters of the paddy field, a dynamic model of agricultural machinery is constructed, which is expressed as: ； In the formula On the vehicle body coordinate system Displacement in the axial direction, For the horizontal swing angle, For the overall vehicle weight, For the vehicle's coordinate system along The speed of the shaft, This refers to the vehicle's front wheelbase. This refers to the rear wheelbase of the vehicle. For the vehicle's moment of inertia, Indicates the front wheel steering angle. For the soil response stiffness of the front wheel, For the soil response stiffness of the rear wheel; The method of controlling vehicle operation based on the vehicle trajectory and paddy field parameters using an agricultural machinery dynamics model includes: The optimal front wheel steering angle control value is then sent to the vehicle controller based on the cost function and constraints. The cost function is expressed as: ； In the formula, Indicates the prediction time domain. express Time prediction arrive Trajectory error sequence at time step The error weights are positive definite matrices. To control the time domain, for Time prediction arrive The increase in front wheel steering angle at any given moment To control the positive definite matrix of the incremental weights, This is a positive definite matrix of terminal error weights. As a relaxation factor, The weights of the relaxation factor.

Images