Silage corn harvester header height real-time automatic lifting system based on deep reinforcement learning
Through deep reinforcement learning combined with DDPG and PID controllers, the header automatic lifting system is solved, and the existing technology lacks adaptability to complex terrain and poor stability of control systems is achieved, and automatic control of header height and efficient agricultural production are achieved.
Patent Information
- Application Number
- CN202411077667.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-07
- Publication Date
- 2024-12-06
AI Technical Summary
The existing header automatic lifting technology is insufficient to adapt to complex terrain, has poor control system stability, relies on driver experience to operate, and has low lifting accuracy.
Using a real-time automatic lifting system for cutting table height of silage corn harvester based on deep reinforcement learning, combined with DDPG algorithm and PID controller, the automatic control of cutting table height is achieved through angle sensors, profiling devices and hydraulic systems. The DDPG controller uses intelligent modules , observation module, reward module and termination module generate action signals and output them to the PID controller to optimize the target network update process to improve system stability and adaptability.
It realizes automatic control of header height, reduces driver operation errors, improves system stability and ability to adapt to complex terrain, and improves the performance of harvesters and agricultural production efficiency.
Smart Images

Figure CN119073098A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of automated control technology, and in particular to a real-time automatic lifting system for the cutting table of a silage corn harvester based on deep reinforcement learning. Background Art
[0002] In order to adapt to the influence of factors such as crop height, terrain slope, soil moisture and other factors on the cutting height, it is necessary to adjust the height of the header at any time to deal with various complex situations. The automatic lifting technology of the header is an automation technology used in agricultural machinery to control the lifting of the header of the harvester. The development of this technology aims to improve agricultural production efficiency, reduce the workload of farmers, and improve the performance of the harvester. The lifting of the traditional header requires the driver to rely on experience to operate manually, which is not only time-consuming and labor-intensive, but also relies on the driver's experience. At the same time, the lifting accuracy may not be high. There is an automatic lifting technology for the header in the prior art, but there are still many problems in the existing system, such as: insufficient adaptability to complex terrain, poor stability of the control system, etc. Summary of the invention
[0003] The technical problem to be solved by the present invention is to address the above shortcomings and provide a real-time automatic lifting system for the cutting table height of a silage corn harvester based on deep reinforcement learning. The automatic lifting of the cutting table is completely controlled by the DDPG algorithm and the PID controller, which realizes the high automation of the cutting table lifting. It does not need to rely on the driver to operate, reduces errors, saves the driver's time and energy, and the control system is stable and suitable for various complex terrains.
[0004] In order to solve the above technical problems, the present invention adopts the following technical solutions:
[0005] The real-time automatic lifting system of the cutting platform height of the silage corn harvester based on deep reinforcement learning includes an angle sensor, a DDPG controller and a PID controller. The angle sensor can convert the angle data between the cutting platform and the ground into an electrical signal and output it to the DDPG controller. The input of the DDPG controller is the target value, the actual value and the deviation value, and an experience pool is added to the DDPG algorithm. The DDPG controller outputs a signal to the PID controller. The PID controller is provided with two signal input terminals, which respectively input the action signal and the deviation value. The output terminal of the PID controller is connected to the hydraulic system, and the lifting of the cutting platform is controlled by the hydraulic system, wherein the target value is given by the outside, the actual value is fed back by the angle sensor, and the deviation value is the difference between the target value and the actual value.
[0006] Furthermore, it also includes a profiling device, which is installed on both sides of the front of the cutting platform. The height and angle data of the cutting platform and the ground are measured according to the profiling device, including using the profiling roller to touch the ground. The ground height fluctuation can be sensed by the profiling roller and produce angle changes. The angle sensor converts the acquired angle change data into an electrical signal and transmits it to the DDPG controller for data processing.
[0007] Furthermore, the DDPG controller includes an intelligent module, an observation module, a reward module and a termination module. The observation module, the reward module and the termination module are respectively used to generate an observation signal, a reward signal and a termination signal and output them to the intelligent module. The intelligent module is used to generate an action signal and output it to the PID controller.
[0008] Furthermore, the observation module is provided with two input terminals, respectively inputting the actual value and the deviation value, and the observation signal includes three components, namely the actual value, the deviation value and the accumulated value of the deviation value.
[0009] Furthermore, the reward module is provided with three signal input terminals for inputting a deviation value, a target value and a termination signal, and for generating a reward signal. The generation process of the reward signal includes three steps:
[0010] Step 1: Calculate the deviation range based on the deviation value and the target value. When the deviation range is less than 1%, a reward of 10 is given; when the deviation range is greater than or equal to 1%, a penalty of -1 is given.
[0011] Step 2: When the termination signal is "true", a penalty is given, and the penalty value is -100;
[0012] Step 3: Add the results of step 1 and step 2 to get the reward signal.
[0013] Furthermore, the input of the termination module is the actual value, and the termination module outputs a corresponding termination signal according to whether the actual value exceeds the limit, wherein the termination signal is "true" when the actual value exceeds the limit, and the termination signal is "false" when the actual value does not exceed the limit.
[0014] Furthermore, the intelligent module is configured with an Actor network and a Critic network, and an improvement is made to the original experience pool of the DDPG algorithm, and an experience pool B2 is added after the original experience pool B1 to improve the efficiency of the target network update.
[0015] Furthermore, the updating process of the target network after optimization includes the following steps:
[0016] Step 1: Randomly initialize the Critic network Q(s, a|θ Q ) and the Actor network μ(s|θ μ ) parameter θQ and θ μ , s is the current observation state, a is the execution action selected by the current strategy;
[0017] Step 2: Initialize the parameters θ of the target network Q and μ Q ←θ Q ,θ μ ←θ μ ;
[0018] Step 3, initialize experience pools B1 and B2;
[0019] Step 4: Let the initial round be 1, and randomly initialize the distribution N actions to explore in M rounds;
[0020] Step 5: Initialize the noise function N for motion exploration. The purpose of exploring noise is to increase the diversity of data, make the system model better adapt to different data distributions, and improve the stability of the system.
[0021] Step 6, initialize the environment and read the initial observation state S1 from the environment, that is, initialize the header height. The initial observation state is the initial signal fed back by the angle sensor when the header height is in the initialization state;
[0022] Step 7, record the initial action as 1 and perform the action within T times.
[0023] Furthermore, the updating process of the target network after optimization further includes the following steps:
[0024] Step 8, select an action based on the current strategy and exploration noise. The current strategy refers to the action selected in the experience pool data in the current state. According to the current experience output signal, the action refers to the ddpg output to control the parameters of the PID system to control the hydraulic system to adjust the header height:
[0025] a t =μ(s t |θ μ )+N t ;
[0026] In the formula, a t The execution action selected for the current strategy refers to the parameters of the PID controller output by the Actor network, which controls the lifting of the header through the hydraulic system. t is the current observation state, N t To explore noise;
[0027] Step 9: Execute action a in the environment t , read the return value r after the action is executed t , and observe the new state s t+1 ;
[0028] Step 10: The interaction samples (s t , a t , r t ,s t+1 ) is stored in the experience pool B1;
[0029] Step 11, obtaining B2 by arranging B1 in descending order according to the reward value and removing the data with smaller reward value;
[0030] Step 12, set the weight w, and obtain the following batches from the priority sampling set and uniform sampling set divided proportionally by B2: s ize*w and batch_size*(1-w) samples (s i , a i , r i ,s i+1 );
[0031] Step 13, calculate the target Q function target value to estimate the expected benefit of performing an action in a specific state:
[0032] y i =r i +γQ′(s i+1 ,μ′(s i+1 |θ μ′ )|θ Q′ )
[0033] In the formula, γ is the attenuation factor;
[0034] Step 14, update the Critic network by minimizing the TD deviation:
[0035] Where L is the mean square error loss function, and N is the number of minimum batch sample data;
[0036] Step 15, calculate the policy gradient through the minimum batch sample to update the Actor network:
[0037]
[0038] In the formula, is the Actor network gradient, is the Critic network gradient, It enables the Actor to continuously adjust its network parameters θ towards the maximum reward that can be obtained. μ ;
[0039] Step 16, update the two target networks:
[0040] θ Q′ ←τθ Q +(1-τ)θQ′ ;
[0041] θ μ′ ←τθ μ +(1-τ)θ μ ;
[0042] In the formula, θ Q′ is the target network parameter, θ μ′ is the target strategy network parameter, τ is the soft update constant;
[0043] Furthermore, the intelligent module can generate corresponding action signals to the PID controller according to the input observation signal, reward signal and termination signal;
[0044] The PID controller outputs instructions to the hydraulic system according to the received action signal to control the automatic lifting of the cutting platform. The output of the PID controller is calculated and expressed as:
[0045]
[0046] Where: u(k) is the output of the PID controller, K p Proportional coefficient, K i Integration time constant, K d The differential time constants are all parameters of the PID controller, and they are all action signals trained by the intelligent module. k is the sampling sequence number 0, 1, 2,...; e(k) is the deviation value input at the kth sampling time; e(k-1) is the deviation value input at the k-1th sampling time; e(j) is the deviation value input at the jth sampling time, j∈(0,k).
[0047] The present invention adopts the above technical solution, and has the following technical effects compared with the prior art:
[0048] 1. The automatic lifting of the harvesting table is completely controlled by the DDPG algorithm and PID controller, which realizes a high degree of automation of the lifting of the harvesting table. It does not need to rely on the driver to operate, reduces errors, saves the driver's time and energy, and is suitable for various complex terrains. It can greatly improve the performance of the harvester, reduce the waste in the harvesting operation, and achieve a consistent height of the harvesting table under different farm conditions. It is very important for improving the efficiency and quality of agricultural production.
[0049] 2. The signal output by the automatic lifting model of the cutting platform is used to adjust the parameters of the PID controller in real time, including that the output of the PID controller can be adjusted by the DDPG algorithm; the DDPG agent may take over control under certain conditions. Under normal circumstances, the PID controller provides stable basic control with high system accuracy.
[0050] 3. Through the linkage control of dual controllers, the real-time automatic lifting system of the header height is obtained, which has much better control performance of the header height than the traditional header height lifting system. It is controlled by DDPG-PID linkage, and the control time and delay time are significantly shortened. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the specific embodiments or the description of the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual scale.
[0052] Figure 1 This is the structure diagram of the DDPG-PID header control system for this application;
[0053] Figure 2 This is the flowchart for the DDPG algorithm implementation for this application;
[0054] Figure 3 PID controller flow chart for this application. DETAILED DESCRIPTION
[0055] Examples, such as Figure 1 As shown in the figure, a real-time automatic lifting system for the cutting table height of a silage corn harvester based on deep reinforcement learning includes an angle sensor, a DDPG controller and a PID controller. The angle sensor can convert the angle data between the cutting table and the ground into an electrical signal and output it to the DDPG controller. The input of the DDPG controller is the target value, the actual value and the deviation value, and an experience pool is added to the DDPG algorithm. The DDPG controller outputs a signal to the PID controller. The PID controller is provided with two signal input terminals, which respectively input the action signal and the deviation value. The output terminal of the PID controller is connected to the hydraulic system, and the lifting of the cutting table is controlled by the hydraulic system, wherein the target value is given by the outside, the actual value is fed back by the angle sensor, and the deviation value is the difference between the target value and the actual value.
[0056] A profiling device is installed on both sides of the front of the cutting platform. The height and angle data of the cutting platform and the ground are measured by the profiling device, including using the profiling roller to touch the ground. The ground height fluctuation can be sensed by the profiling roller and produce angle changes. The angle sensor converts the acquired angle change data into an electrical signal and transmits it to the DDPG controller for data processing.
[0057] The DDPG controller includes an intelligent module, an observation module, a reward module and a termination module. The observation module, the reward module and the termination module are respectively used to generate an observation signal, a reward signal and a termination signal and output them to the intelligent module. The intelligent module is used to generate an action signal and output it to the PID controller.
[0058] The observation module is provided with two input terminals, respectively inputting the actual value and the deviation value, and the observation signal includes three components, namely the actual value, the deviation value and the accumulated value of the deviation value.
[0059] At the beginning of each round of training of the deep learning neural network of the intelligent module, or when the target value of the DDPG controller input changes, the accumulated value is cleared and accumulation starts again.
[0060] The reward module is provided with three signal input terminals for inputting a deviation value, a target value and a termination signal, and is used to generate a reward signal. The generation process of the reward signal includes three steps:
[0061] Step 1: Calculate the deviation range based on the deviation value and the target value. When the deviation range is <1%, a reward of 10 is given; when the deviation range is ≥1%, a penalty of -1 is given;
[0062] Step 2: When the termination signal is "true", a penalty is given, and the penalty value is -100;
[0063] Step 3: Add the results of step 1 and step 2 to get the reward signal.
[0064] The input of the termination module is the actual value, and the termination module outputs a corresponding termination signal according to whether the actual value exceeds the limit, wherein the termination signal is "true" when the actual value exceeds the limit, and the termination signal is "false" when the actual value does not exceed the limit.
[0065] like Figure 2 As shown, the intelligent module is configured with an Actor network and a Critic network, and an improvement is made to the original experience pool of the DDPG algorithm, and an experience pool B2 is added after the original experience pool B1 to improve the efficiency of the target network update.
[0066] The sample data originally stored in the experience pool B2 is subjected to mixed uniform sampling and priority sampling processing to ensure the quality of the sample data and improve the utilization rate of the sample data.
[0067] The update process of the target network after optimization is the process of training the signal data collected by the angle sensor. The trained data can be placed in the experience pool to control the PID output control signal, thereby controlling the lifting of the header. The update process of the target network after optimization is:
[0068] Step 1: Randomly initialize the Critic network Q(s, a|θ Q ) and the Actor network μ(s|θ μ ) parameter θ Q and θ μ, s is the current observation state, a is the execution action selected by the current strategy;
[0069] Step 2: Initialize the parameters θ of the target network Q′ and μ′ Q′ ←θ Q ,θ μ′ ←θ μ ;
[0070] Step 3, initialize experience pools B1 and B2;
[0071] Step 4: Let the initial round be 1, and randomly initialize the distribution N actions to explore in M rounds;
[0072] Step 5: Initialize the noise function N for motion exploration. The purpose of exploring noise is to increase the diversity of data, make the system model better adapt to different data distributions, and improve the stability of the system.
[0073] Step 6, initialize the environment and read the initial observation state S1 from the environment, that is, initialize the header height. The initial observation state is the initial signal fed back by the angle sensor when the header height is in the initialization state;
[0074] Step 7, record the initial action as 1 and perform the action within T times;
[0075] Step 8, select an action based on the current strategy and exploration noise. The current strategy refers to the action selected in the experience pool data in the current state. According to the current experience output signal, the action refers to the ddpg output to control the parameters of the PID system to control the hydraulic system to adjust the header height:
[0076] a t =μ(s t |θ μ )+N t ;
[0077] In the formula, a t The execution action selected for the current strategy refers to the parameters of the PID controller output by the Actor network, which controls the lifting of the header through the hydraulic system. t is the current observation state, N t To explore noise;
[0078] Step 9: Execute action a in the environment t , read the return value r after the action is executed t , and observe the new state s t+1 ;
[0079] Step 10: The interaction samples (s t , a t , r t ,s t+1) is stored in the experience pool B1;
[0080] Step 11, obtaining B2 by arranging B1 in descending order according to the reward value and removing the data with smaller reward value;
[0081] Step 12, set the weight w, and obtain the following from the priority sampling set and uniform sampling set divided proportionally by B2: s ize*w and batch_size*(1-w) samples (s i , a i , r i ,s i+1 );
[0082] Step 13, calculate the target Q function target value to estimate the expected benefit of performing an action in a specific state:
[0083] y i =r i +γQ′(s i+1 ,μ′(s i+1 |θ μ′ )|θ Q′ )
[0084] In the formula, γ is the attenuation factor;
[0085] Step 14, update the Critic network by minimizing the TD deviation:
[0086] Where L is the mean square error loss function, and N is the number of minimum batch sample data;
[0087] Step 15, calculate the policy gradient through the minimum batch sample to update the Actor network:
[0088]
[0089] In the formula, is the Actor network gradient, is the Critic network gradient, It enables the Actor to continuously adjust its network parameters θ towards the maximum reward that can be obtained. μ ;
[0090] Step 16, update the two target networks:
[0091] θ Q′ ←τθ Q +(1-r)θ Q′ ;
[0092] θ μ′ ←τθ μ +(1-τ)θu′ ;
[0093] In the formula, θ Q′ is the target network parameter, θ μ′ is the target strategy network parameter, τ is the soft update constant;
[0094] like Figure 1 As shown, the intelligent module can generate corresponding action signals to the PID controller according to the input observation signal, reward signal and termination signal.
[0095] like Figure 3 As shown, the PID controller outputs instructions to the hydraulic system according to the received action signal to control the automatic lifting of the cutting platform. The output of the PID controller is calculated and expressed as:
[0096]
[0097] Where: u(k) is the output of the PID controller, K p Proportional coefficient, K i Integration time constant, K d The differential time constants are all parameters of the PID controller, and they are all action signals trained by the intelligent module. k is the sampling sequence number 0, 1, 2,...; e(k) is the deviation value input at the kth sampling time; e(k-1) is the deviation value input at the k-1th sampling time; e(j) is the deviation value input at the jth sampling time, j∈(0,k).
[0098] Compared with the existing PID controller parameter K p , K i , K d The parameters of the PID controller in this application are dynamically adjusted by the PPDG controller and given K p , K i , K d Based on the parameters, the PID controller generates the corresponding hydraulic system control strategy according to the input deviation value, and finally outputs it to the hydraulic system in the form of a proportional signal, thereby achieving precise control of the lifting and lowering of the cutting platform.
[0099] The description of the present invention is given for the purpose of illustration and description, and is not intended to be exhaustive or to limit the invention to the disclosed form. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present invention and to enable those of ordinary skill in the art to understand the present invention and thereby design various embodiments with various modifications suitable for particular uses.
Claims
1. A real-time automatic lifting system for the cutting platform of a silage corn harvester based on deep reinforcement learning, characterized in that: It includes an angle sensor, a DDPG controller and a PID controller. The angle sensor can convert the angle data between the cutting platform and the ground into an electrical signal and output it to the DDPG controller. The input of the DDPG controller is the target value, the actual value and the deviation value, and an experience pool is added to the DDPG algorithm. The DDPG controller outputs a signal to the PID controller. The PID controller is provided with two signal input terminals, which respectively input the action signal and the deviation value. The output terminal of the PID controller is connected to the hydraulic system, and the lifting of the cutting platform is controlled by the hydraulic system. The target value is given by the outside, the actual value is fed back by the angle sensor, and the deviation value is the difference between the target value and the actual value.
2. The real-time automatic lifting system for the cutting platform of a silage corn harvester based on deep reinforcement learning as claimed in claim 1, characterized in that: It also includes a profiling device, which is installed on both sides of the front of the cutting platform. The height and angle data of the cutting platform and the ground are measured according to the profiling device, including using the profiling roller to touch the ground. The ground height fluctuation can be sensed by the profiling roller and produce angle changes. The angle sensor converts the acquired angle change data into an electrical signal and transmits it to the DDPG controller for data processing.
3. The real-time automatic lifting system for the cutting platform of a silage corn harvester based on deep reinforcement learning as claimed in claim 1, characterized in that: The DDPG controller includes an intelligent module, an observation module, a reward module and a termination module. The observation module, the reward module and the termination module are respectively used to generate an observation signal, a reward signal and a termination signal and output them to the intelligent module. The intelligent module is used to generate an action signal and output it to the PID controller.
4. The real-time automatic lifting system for the cutting platform of a silage corn harvester based on deep reinforcement learning as claimed in claim 1, characterized in that: The observation module is provided with two input terminals, respectively inputting the actual value and the deviation value, and the observation signal includes three components, namely the actual value, the deviation value and the accumulated value of the deviation value.
5. The real-time automatic lifting system for the cutting platform of a silage corn harvester based on deep reinforcement learning as claimed in claim 1, characterized in that: The reward module is provided with three signal input terminals for inputting a deviation value, a target value and a termination signal, and is used to generate a reward signal. The generation process of the reward signal includes three steps: Step 1: Calculate the deviation range based on the deviation value and the target value. When the deviation range is less than 1%, a reward of 10 is given; when the deviation range is greater than or equal to 1%, a penalty of -1 is given. Step 2: When the termination signal is "true", a penalty is given, and the penalty value is -100; Step 3: Add the results of step 1 and step 2 to get the reward signal.
6. The real-time automatic lifting system for the cutting platform of a silage corn harvester based on deep reinforcement learning as claimed in claim 1, characterized in that: The input of the termination module is the actual value, and the termination module outputs a corresponding termination signal according to whether the actual value exceeds the limit, wherein the termination signal is "true" when the actual value exceeds the limit, and the termination signal is "false" when the actual value does not exceed the limit.
7. The real-time automatic lifting system for the cutting platform of a silage corn harvester based on deep reinforcement learning as claimed in claim 1, characterized in that: The intelligent module is configured with an Actor network and a Critic network, and improves the original experience pool of the DDPG algorithm by adding an experience pool B2 after the original experience pool B1 to improve the efficiency of updating the target network.
8. The real-time automatic lifting system for the cutting platform of a silage corn harvester based on deep reinforcement learning as claimed in claim 7, characterized in that: The updating process of the target network after optimization comprises the following steps: Step 1: Randomly initialize the Critic network Q(s, a|θ Q ) and the Actor network μ(s|θ μ ) parameter θ Q and θ μ , s is the current observation state, a is the execution action selected by the current strategy; Step 2: Initialize the parameters θ of the target network Q′ and μ′ Q′ ←θ Q ,θ μ′ ←θ μ ; Step 3, initialize experience pools B1 and B2; Step 4: Let the initial round be 1, and randomly initialize the distribution N actions to explore in M rounds; Step 5: Initialize the noise function N for motion exploration. The purpose of exploring noise is to increase the diversity of data, make the system model better adapt to different data distributions, and improve the stability of the system. Step 6, initialize the environment and read the initial observation state S1 from the environment, that is, initialize the header height. The initial observation state is the initial signal fed back by the angle sensor when the header height is in the initialization state; Step 7, record the initial action as 1 and perform the action within T times.
9. The real-time automatic lifting system for the cutting platform of a silage corn harvester based on deep reinforcement learning as claimed in claim 8, characterized in that: The target network after optimization update process also includes the following steps: Step 8, select an action based on the current strategy and exploration noise. The current strategy refers to the action selected in the experience pool data in the current state. According to the current experience output signal, the action refers to the ddpg output to control the parameters of the PID system to control the hydraulic system to adjust the header height: a t =μ(s t |θ μ )+N t ; In the formula, a t The execution action selected for the current strategy refers to the parameters of the PID controller output by the Actor network, which controls the lifting of the header through the hydraulic system. t is the current observation state, N t To explore noise; Step 9: Execute action a in the environment t , read the return value r after the action is executed t , and observe the new state s t+1 ; Step 10: The interaction samples (s t , a t , r t ,s t+1 ) is stored in the experience pool B1; Step 11, obtaining B2 by arranging B1 in descending order according to the reward value and removing the data with smaller reward value; Step 12, set the weight w, and obtain the following batches from the priority sampling set and uniform sampling set divided proportionally by B2: s ize*w and batch_size*(1-w) samples (s i , a i , r i ,s i+1 ); Step 13, calculate the target Q function target value to estimate the expected benefit of performing an action in a specific state: y i =r i +γQ′(s i+1 ,μ′(s i+1 |θ μ′ )|θ Q′ ) In the formula, γ is the attenuation factor; Step 14, update the Critic network by minimizing the TD deviation: Where L is the mean square error loss function, and N is the number of minimum batch sample data; Step 15, calculate the policy gradient through the minimum batch sample to update the Actor network: In the formula, is the Actor network gradient, is the Critic network gradient, It enables the Actor to continuously adjust its network parameters θ towards the maximum reward that can be obtained. μ ; Step 16, update the two target networks: i Q′ ←tth Q +(1-τ)θ Q′ ; i μ′ ←t μ ′+(1-τ) μ′ ; In the formula, θ Q′ is the target network parameter, θ μ′ is the target policy network parameter, and τ is the soft update constant.
10. The real-time automatic lifting system for the cutting platform of a silage corn harvester based on deep reinforcement learning as claimed in claim 1, characterized in that: The intelligent module can generate corresponding action signals to the PID controller according to the input observation signal, reward signal and termination signal; The PID controller outputs instructions to the hydraulic system according to the received action signal to control the automatic lifting of the cutting platform. The output of the PID controller is calculated and expressed as: Where: u(k) is the output of the PID controller, K p Proportional coefficient, K i Integration time constant, K d The differential time constants are all parameters of the PID controller, and they are all action signals trained by the intelligent module. k is the sampling sequence number 0, 1, 2,...; e(k) is the deviation value input at the kth sampling time; e(k-1) is the deviation value input at the k-1th sampling time; e(j) is the deviation value input at the jth sampling time, j∈(0,k).
Citation Information
Patent Citations
Automatic cutting type tea harvester cutting bench automatic levelling method and device
CN104521435A
Chip heating method for pressure sensor calibration
CN107543652A
Intelligent control system for mechanical arm based on reservoir sampling and double experience pools
CN111300390A
Electro-hydraulic regulation and control method and system for height of header of soybean harvester
CN112189443A
PID dynamic setting motor control system based on DDPG reinforcement learning
CN114967426A
Cited By
Harvester header height adjusting method, harvester and electronic equipment
CN120036127A
Corn harvester energy-saving regulation and control method and system based on reinforcement learning
CN120240137A
On-line detection and regulation system and method for peeling residual quantity of corn harvester
CN121866976A
On-line detection and control system and method for shelling residue of corn harvester
CN121866976B
