Agricultural machine automatic driving adaptation method and device

Abstract

The invention provides an agricultural machinery automatic driving adaptation method and device, and relates to the technical field of automatic driving, and the method comprises the steps: training a robot through a reinforcement learning algorithm based on a simulation environment, and obtaining a pre-training control strategy; migrating the pre-training control strategy to an entity agricultural robot through a dual-environment strategy distillation technology to obtain a robot control strategy; and based on a real farmland environment, a dynamic course learning mechanism is utilized to adjust the robot control strategy, and an on-site control adaptation strategy is obtained. According to the invention, the adaptability of automatic driving of the agricultural machinery can be improved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a method and device for autonomous driving adaptation of agricultural machinery. Background Art

[0002] With the advancement of agricultural modernization, autonomous driving technology has also been widely used in agriculture. Traditional autonomous driving is based on conventional road environments, with clear and well-defined traffic rules and simple vehicle-ground interactions. It is generally assumed that the ground and tires are in constant, reliable contact, eliminating complex situations such as slipping. As a result, the relationship between characteristic quantities such as speed and turning angle and operational variables such as steering wheel rotation and throttle pressure is relatively simple.

[0003] However, agricultural fields are primarily unpaved, with complex and variable surface conditions, often characterized by potholes and undulations. This makes agricultural machinery susceptible to slippage and other surface adhesion variations during operation. Existing autonomous driving technologies struggle to achieve the specific turning angles and speeds required by agricultural machinery, making them incapable of meeting their operational requirements. Summary of the Invention

[0004] The problem solved by the present invention is how to improve the adaptability of automatic driving of agricultural machinery.

[0005] In order to solve the above problems, the present invention provides an agricultural machinery automatic driving adaptation method and device.

[0006] In a first aspect, the present invention provides an adaptation method for autonomous driving of agricultural machinery, comprising: training a robot through a reinforcement learning algorithm based on a simulation environment to obtain a pre-trained control strategy;

[0007] The pre-trained manipulation strategy is transferred to the physical agricultural robot through the dual-environment strategy distillation technology to obtain the robot manipulation strategy;

[0008] Based on the real farmland environment, the robot control strategy is adjusted using a dynamic curriculum learning mechanism to obtain a field control adaptation strategy.

[0009] Optionally, the step of training the robot to obtain a pre-trained manipulation strategy by using a reinforcement learning algorithm based on a simulation environment includes:

[0010] Constructing a motion space of the robot, the motion space including pedal force and steering wheel angle of the robot;

[0011] Based on the action space, a multi-dimensional reward function is constructed through safety indicators, energy consumption indicators and trajectory tracking errors;

[0012] The pre-trained control strategy is obtained through the multi-dimensional reward function.

[0013] Optionally, the step of training the robot to obtain a pre-trained control strategy through a reinforcement learning algorithm based on a simulation environment further includes:

[0014] The pre-trained control strategy is obtained by randomly perturbing physical parameters in the simulation environment and using the randomly perturbed physical parameters to train the robot, wherein the physical parameters include transmission efficiency and tire slip rate.

[0015] Optionally, the safety index includes the collision probability between the robot and an obstacle and the emergency braking trigger rate when the robot detects a potential collision risk; the energy consumption index includes the fuel consumption per unit operating area of ​​the robot and the peak power ratio; the trajectory tracking error is determined by the lateral deviation between the actual position of the robot and the preset path and the heading angle deviation between the actual driving direction of the robot and the driving direction of the preset path.

[0016] Optionally, the pre-trained manipulation strategy is transferred to a physical agricultural robot through the dual-environment strategy distillation technology to obtain the robot manipulation strategy, which includes:

[0017] A lightweight neural network model is constructed on the physical agricultural robot, and the pre-trained control strategy is migrated to the lightweight neural network model using importance sampling technology to obtain the robot control strategy, wherein the lightweight neural network model is used to learn the pre-trained control strategy.

[0018] Optionally, before migrating the pre-trained control strategy to the physical agricultural robot through the dual-environment policy distillation technology to obtain the robot control strategy, the agricultural machinery automatic driving adaptation method further includes:

[0019] According to preset mapping rules, the pre-trained control strategy is converted from simulation instructions into electrical signals through a physical signal converter.

[0020] Optionally, after migrating the pre-trained control strategy to the physical agricultural robot through the dual-environment policy distillation technology to obtain the robot control strategy, the agricultural machinery automatic driving adaptation method further includes:

[0021] Obtaining delay characteristics of each component of the physical agricultural robot;

[0022] constructing a delay compensation model based on the delay characteristics and the state predictor;

[0023] When a signal delay of the physical agricultural robot is detected, a compensation signal is generated according to the delay predicted by the delay compensation model;

[0024] The action of the physical agricultural robot is adjusted according to the compensation signal.

[0025] Optionally, the robot manipulation strategy is adjusted based on the real farmland environment using a dynamic curriculum learning mechanism to obtain a field manipulation adaptation strategy including:

[0026] Divide the operation into multiple stages according to the complexity of the operation scenario;

[0027] Obtaining a trajectory tracking error of the physical agricultural robot in the current operation phase;

[0028] When the trajectory tracking error is less than or equal to a preset threshold for multiple consecutive preset cycles, the physical agricultural robot is controlled to adjust to the next operation stage until the operation of the operation scene is completed, thereby obtaining the field control adaptation strategy.

[0029] Optionally, the robot manipulation strategy is adjusted based on a real farmland environment using a dynamic curriculum learning mechanism to obtain a field manipulation adaptation strategy, further comprising:

[0030] When the predicted collision time between the physical agricultural robot and the obstacle is less than the preset collision time and / or the lateral safety margin of the physical agricultural robot is continuously less than the preset safety margin within a preset continuous time, the physical agricultural robot is controlled to trigger emergency braking, and the operation of the physical agricultural robot is controlled by a takeover mechanism.

[0031] In a second aspect, the present invention provides an agricultural machinery automatic driving adaptation device, comprising:

[0032] The acquisition module is used to train the robot to obtain pre-trained control strategies through reinforcement learning algorithms based on a simulation environment;

[0033] A migration module is used to migrate the pre-trained manipulation strategy to the physical agricultural robot through dual-environment strategy distillation technology to obtain the robot manipulation strategy;

[0034] The adjustment module is used to adjust the robot control strategy based on the real farmland environment and utilize the dynamic curriculum learning mechanism to obtain a field control adaptation strategy.

[0035] The beneficial effects of the agricultural machinery automatic driving adaptation method of the present invention are as follows: reinforcement learning training based on a simulation environment can obtain pre-trained control strategies for agricultural robots. In a simulation environment, pre-trained control strategies can be quickly generated, avoiding the time and resource consumption caused by actual operations on a physical robot. Training through a simulation environment can reduce the demand for actual data, thereby reducing data acquisition costs. Through the dual-environment strategy distillation technology, pre-trained control strategies can be migrated to physical agricultural robots. The simulation environment can simulate many actual situations, but there is still a gap in the real environment. The dual-environment strategy distillation technology can effectively narrow the gap and ensure that the strategies learned in the simulation environment can remain efficient and reliable in actual applications. The dynamic course learning mechanism based on the real farmland environment can flexibly adjust the robot control strategy to adapt to operation scenarios of different complexities. By gradually increasing the complexity of the operation, the robot can gradually learn and adapt in actual operation, thereby improving the adaptability of automatic driving technology in agricultural machinery. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a flow chart of a method for adapting automatic driving of agricultural machinery according to an embodiment of the present invention;

[0037] Figure 2 This is a schematic structural diagram of an automatic driving adaptation device for agricultural machinery according to an embodiment of the present invention;

[0038] Figure 3 The figure is a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] To make the above-mentioned objects, features, and advantages of the present invention more clearly understood, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings. Although certain embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0040] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0041] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to"; the term "based on" means "based at least in part on"; the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments"; the term "optionally" means "optional embodiments". The relevant definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc. mentioned in the present invention are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0042] It should be noted that the modifications of "one" and "multiple" mentioned in the present invention are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0043] The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0044] like Figure 1 As shown, an embodiment of the present invention provides an agricultural machinery automatic driving adaptation method, including:

[0045] S100, based on a simulation environment, trains the robot through a reinforcement learning algorithm to obtain pre-trained control strategies.

[0046] Specifically, within a high-precision agricultural scene simulation platform, a virtual scene containing farmland elements such as muddy roads, ramps, and obstacles is constructed. The Proximal Policy Optimization (PPO) reinforcement learning algorithm, combined with the LSTM (Long Short-Term Memory) algorithm, can be used to train robots to acquire pre-trained control strategies. In the application of autonomous agricultural machinery, LSTM can effectively capture the agricultural robot's past state information and its impact on current decision-making. By introducing the LSTM layer, dynamic changes over long time spans can be better understood, leading to more reasonable driving decisions. PPO can converge to the optimal strategy more quickly while reducing the drastic fluctuations that may occur during training, enabling agricultural robots to quickly adapt to and perform tasks under diverse environmental conditions.

[0047] S200, through the dual-environment strategy distillation technology, the pre-trained control strategy is transferred to the physical agricultural robot to obtain the robot control strategy.

[0048] Specifically, to bridge the gap between simulation and physical environments, domain randomization can be used to perturb key physical parameters in the simulation environment, enhancing the model's generalization capabilities. The pre-trained control policy obtained through randomization can be transferred to the physical agricultural robot. A physical signal converter can be used to convert the virtual control signals into real control signals for the robot hardware. Dual-environment policy distillation technology is used to achieve knowledge transfer from simulation policies to physical policies.

[0049] S300, based on the real farmland environment, using a dynamic course learning mechanism, adjust the robot control strategy to obtain a field control adaptation strategy.

[0050] Specifically, a dynamic curriculum learning mechanism based on real-world farmland environments can flexibly adjust the robot's control strategy to adapt to operational scenarios of varying complexity. By gradually increasing the complexity of the task, the robot can gradually learn and adapt during actual operation, avoiding the high risks and low efficiency associated with facing a complex environment all at once.

[0051] In this embodiment, reinforcement learning training based on a simulation environment can obtain pre-trained control strategies for agricultural robots. In a simulation environment, pre-trained control strategies can be quickly generated, avoiding the time and resource consumption caused by actual operations on a physical robot. Training through a simulation environment can reduce the demand for actual data, thereby reducing data acquisition costs. Through dual-environment policy distillation technology, pre-trained control strategies can be transferred to physical agricultural robots. The simulation environment can simulate many actual situations, but there is still a gap in the real environment. The dual-environment policy distillation technology can effectively narrow the gap and ensure that the strategies learned in the simulation environment can remain efficient and reliable in actual applications. The dynamic curriculum learning mechanism based on the real farmland environment can flexibly adjust the robot's control strategy to adapt to operation scenarios of varying complexity. By gradually increasing the complexity of the operation, the robot can gradually learn and adapt in actual operation, improving the adaptability of autonomous driving technology in agricultural machinery.

[0052] Optionally, the step of training the robot to obtain a pre-trained manipulation strategy by using a reinforcement learning algorithm based on a simulation environment includes:

[0053] Constructing a motion space of the robot, the motion space including pedal force and steering wheel angle of the robot;

[0054] Based on the action space, a multi-dimensional reward function is constructed through safety indicators, energy consumption indicators and trajectory tracking errors;

[0055] The pre-trained control strategy is obtained through the multi-dimensional reward function.

[0056] Specifically, an action space encompassing pedal force and steering wheel angle is constructed within a simulation environment. Specifically, by defining the operating ranges of the accelerator and brake pedals, as well as the steering wheel angle range, the robot can learn optimal control strategies for different scenarios. Based on this defined action space, a multidimensional reward function is constructed using safety metrics, energy consumption metrics, and trajectory tracking error. Agricultural machinery operates in complex and changing environments. Setting safety metrics encourages the robot to make rational decisions when faced with emergencies, avoiding accidents and ensuring smooth operation. By incorporating collision probability and emergency braking trigger rate into the reward function, the robot can learn how to effectively avoid collision risks and reduce the frequency of emergency braking, thereby ensuring the safety of both operators and equipment. By incorporating energy consumption metrics into the reward function, the robot can be guided to learn more energy-efficient driving strategies and achieve efficient energy utilization. The robot's trajectory is tracked, and using the trajectory tracking error, the robot learns how to more accurately follow the preset path, reducing deviations and thus improving the accuracy and quality of the operation. A multidimensional reward function is constructed by weightedly integrating safety metrics, energy consumption metrics, and trajectory tracking error. During training, a multidimensional reward function is constructed to balance safety, energy consumption, and trajectory tracking error during learning. The robot continuously adjusts its strategy based on feedback (i.e., reward) after executing an action to maximize cumulative reward. In a simulation environment, the robot randomly selects an action (pedal force and steering wheel angle) from the action space, executes the randomly selected action, and moves to a new position. The reward value for the current action is obtained based on the multidimensional reward function, reflecting the quality of the action. A reinforcement learning algorithm (such as the PPO algorithm) is used to update the parameters of the policy network based on the obtained reward value, resulting in a pre-trained control policy with optimal overall performance.

[0057] In this optional embodiment, by constructing an action space covering pedal force and steering wheel angle, the robot can learn a general control strategy applicable to a variety of scenarios. By covering a variety of possible actions in the simulation, the robot can better cope with various complex situations in reality. By setting a multi-dimensional reward function that integrates safety indicators, energy consumption indicators and trajectory tracking errors, the robot can have stronger robustness and adaptability when facing different environmental conditions and work tasks. According to the reward function, the driving strategy is automatically adjusted to meet the requirements of safety, energy consumption and trajectory tracking, ensuring that the agricultural robot can operate stably and efficiently in various complex situations.

[0058] Optionally, the step of training the robot to obtain a pre-trained control strategy through a reinforcement learning algorithm based on a simulation environment further includes:

[0059] The pre-trained control strategy is obtained by randomly perturbing physical parameters in the simulation environment and using the randomly perturbed physical parameters to train the robot, wherein the physical parameters include transmission efficiency and tire slip rate.

[0060] Specifically, physical parameters that affect the robot's performance, such as transmission efficiency and tire slip, are set in the simulation environment. Transmission efficiency refers to the ratio of the effective power transmitted from the power source (such as the engine or motor) to the input power to the wheel. Tire slip refers to the degree of sliding of the tire relative to the ground during driving, and is used to describe the situation where the tire fails to roll completely as expected under specific load and ground conditions. Slip occurs when the agricultural robot accelerates, brakes or turns, and is more common on low-adhesion roads (such as muddy, wet or soft land). Random perturbations of the transmission efficiency and tire slip in the simulation environment enable the robot to maintain good performance in the face of various unforeseen changes, thereby being more adaptable to the complexity and variability of real farmland, able to handle various complex situations in actual operations, and reducing the gap between simulation and actual application. During the reinforcement learning training process, more complex environmental conditions can be simulated by randomly perturbing the physical parameters. The robot tries different actions based on the current physical parameters (including the perturbed parameters) and evaluates the quality of these actions based on a multi-dimensional reward function. Through continuous trial and adjustment, the optimal control strategy under various physical conditions is obtained, namely the pre-trained manipulation strategy.

[0061] In this optional embodiment, by randomly perturbing the physical parameters in the simulation environment, the pre-trained strategy can perform well under a wider range of conditions. When the robot migrates from the simulation environment to the real farmland, it can travel stably under different transmission efficiencies and tire slip rates, thereby improving the overall reliability. There are certain differences between the simulation environment and the real farmland. Directly applying the simulation strategy that has not been randomized to the physical robot may cause the robot's performance to deteriorate. By introducing the difference between the simulation environment and the real farmland in advance during the simulation stage, the "simulation-reality gap" can be reduced, making the strategy migration smoother and smoother. Using the pre-trained control strategy obtained after random perturbation, the robot can adapt to the new or changing working environment more quickly, thereby enhancing the adaptability and stability of the strategy.

[0062] Optionally, the safety index includes the collision probability between the robot and an obstacle and the emergency braking trigger rate when the robot detects a potential collision risk; the energy consumption index includes the fuel consumption per unit operating area of ​​the robot and the peak power ratio; the trajectory tracking error is determined by the lateral deviation between the actual position of the robot and the preset path and the heading angle deviation between the actual driving direction of the robot and the driving direction of the preset path.

[0063] Specifically, to ensure the safety of the robot during mission execution, two safety metrics are set: the collision probability between the robot and an obstacle and the emergency braking trigger rate when the robot detects a potential collision risk. The collision probability is calculated by using sensors to perceive the surrounding environment, based on the distance between the robot and the obstacle and the relative speed between the robot and the obstacle, to predict the likelihood of a collision between the robot and the obstacle. The collision probability is associated with a negative reward; that is, a higher collision probability results in a lower reward for the robot, thereby encouraging the robot to learn a collision-avoiding driving strategy. The emergency braking trigger rate refers to the frequency with which the robot activates its emergency braking mechanism when a potential collision risk is detected, reflecting the robot's response frequency to emergencies. A negative reward can be assigned to each emergency braking trigger, incentivizing the robot to minimize emergency braking and thus promoting safer driving behavior. Energy consumption metrics include the robot's fuel consumption per unit operating area and peak power ratio. The fuel consumption per unit operating area can be set inversely proportional to the reward value; that is, the lower the fuel consumption, the higher the reward. This helps encourage the robot to adopt an energy-saving driving strategy, properly control its speed, and avoid unnecessary acceleration and deceleration, thereby reducing agricultural production costs. The peak power ratio reflects the ratio of the robot's maximum power achieved during the entire operation to its total power. A lower peak power percentage can be assigned a higher reward, encouraging the agent to adopt a smoother driving style, improving fuel economy or energy efficiency, and further reducing costs. Lateral deviation refers to the degree to which the robot deviates from the centerline of the preset path during travel. Lateral deviation directly affects operational accuracy. Smaller lateral deviation indicates that the agricultural machine's trajectory is closer to the preset path, helping to improve operational accuracy. Heading angular deviation refers to the angular difference between the robot's current direction and the preset path. Smaller heading angular deviation indicates that the agricultural machine's direction is more aligned with the preset path, ensuring that the machine maintains the correct direction during operation, improving operational quality and efficiency. A multidimensional reward function is constructed by weightedly integrating safety indicators, energy consumption indicators, and trajectory tracking error. In agricultural scenarios, if accident avoidance is a priority, a higher weight can be given to safety indicators; if improving energy efficiency or reducing operating costs is desired, the weight of energy consumption indicators can be appropriately increased. By considering multiple dimensions simultaneously, a multidimensional reward function can better balance the relationships between different indicators and promote the development of a more comprehensive optimization strategy.

[0064] In this optional embodiment, agricultural machinery operates in complex and ever-changing environments, presenting numerous potential hazards. By incorporating collision probability and emergency braking trigger rate into the reward function, the robot can learn how to effectively avoid collision risks and reduce the frequency of emergency braking, thereby ensuring the safety of both operators and equipment. Fuel consumption per unit operating area is related to fuel efficiency; reducing this metric can reduce fuel consumption and, consequently, production costs. Peak power ratio reflects the proportion of time the engine or motor operates under high load. By optimizing this metric, agricultural machinery can operate within a more economical power range, improving fuel economy or energy efficiency, and further reducing costs. By integrating positioning data with the lateral deviation and heading angle deviation of a pre-set path to quantify trajectory tracking error, the robot can learn how to more accurately track the pre-set path, reduce deviation, and thus improve the accuracy and quality of operations. The robot continuously tries different action combinations in a simulation environment, evaluating its own performance based on the reward signals it receives, thereby learning the optimal control strategy for various complex situations. When faced with emergencies, such as an obstacle suddenly appearing ahead, the robot can quickly make decisions on whether to perform emergency braking, detours, or other reasonable obstacle avoidance actions based on the weight of the safety indicator in the reward function. While considering the impact of energy consumption and trajectory deviation, it selects the optimal obstacle avoidance path and method to ensure the continuity and safety of operations.

[0065] Optionally, the pre-trained manipulation strategy is transferred to a physical agricultural robot through the dual-environment strategy distillation technology to obtain the robot manipulation strategy, which includes:

[0066] A lightweight neural network model is constructed on the physical agricultural robot, and the pre-trained control strategy is migrated to the lightweight neural network model using importance sampling technology to obtain the robot control strategy, wherein the lightweight neural network model is used to learn the pre-trained control strategy.

[0067] Specifically, a lightweight neural network model is deployed on a physical agricultural robot, and importance sampling technology is used to transfer knowledge from a pre-trained control policy in a simulation environment to the lightweight neural network model on the physical agricultural robot. Based on the data distribution differences between the simulation and physical environments, a weight coefficient is determined. The simulation data is reweighted based on the weight coefficient, allowing importance sampling to focus on key samples and improve the efficiency of transfer learning. In the simulation environment, the virtual robot is controlled by the pre-trained control policy, and state-action pairs (s, a) are collected. S represents the robot state, including position, speed, and direction; a represents the action taken by the robot in that state, namely, pedal force and steering wheel angle. These collected state-action pairs are then used to implement the control policy on the physical robot. Initially, the robot control policy may be randomly initialized or a simple strategy, which is continuously optimized through the transfer process. For each collected state-action pair (s, a), the probability of the robot taking action a when in state s in the simulation environment is used as the pre-trained control strategy probability density. The probability of the robot taking action a when in state s in the real environment is used as the robot control strategy probability density. Based on the pre-trained control strategy probability density and the robot control strategy probability density, the importance weight of each state-action pair (i.e., the ratio of the robot control strategy probability density to the pre-trained control strategy probability density) is calculated. The robot control strategy is updated based on the importance weight using weighted KL divergence. The updated robot control strategy is executed on the physical robot, and its performance is evaluated to check whether it meets the preset performance standards. If the strategy performance meets the standards, the robot control strategy is deployed on the physical robot for actual operation.

[0068] In this optional embodiment, building a lightweight neural network model helps ensure that it can run efficiently on actual devices. Importance sampling technology can bridge the distribution gap between the simulation and physical environments, reweighting the effectiveness of simulated actions in the physical environment to resolve the state-action space mismatch between the simulation and physical environments. By appropriately weighting the actions generated in the simulation environment, it can ensure that those strategies that are also effective in the physical environment are prioritized, thereby improving the success rate of the entire migration process.

[0069] Optionally, before migrating the pre-trained control strategy to the physical agricultural robot through the dual-environment policy distillation technology to obtain the robot control strategy, the agricultural machinery automatic driving adaptation method further includes:

[0070] According to preset mapping rules, the pre-trained control strategy is converted from simulation instructions into electrical signals through a physical signal converter.

[0071] Specifically, due to the discrepancy between the control commands used in the simulation environment and the actual electrical signals received by the physical mechanical system, a physical signal converter is used to convert the pre-trained control strategy from simulation commands to electrical signals. This allows for a smooth transition from the simulation environment to the actual hardware platform (i.e., the physical agricultural robot). This ensures that the strategy learned in simulation can be correctly executed in the real farmland environment, allowing the agricultural robot to perform precise operations based on the learned strategy. Different pedal forces can be set to correspond to different voltage output values, and the steering wheel torque angle can be represented by a pulse-width modulation (PWM) wave. A physical signal converter is used to convert the simulation commands generated by reinforcement learning (such as steering wheel angle and pedal force value) into physical signals (such as PWM waves and voltage / current signals) that can be executed by the physical agricultural robot hardware, ensuring that the pre-trained strategy can be effectively executed in the real farmland environment. The preset mapping rules include the correspondence between simulation commands and electrical signals.

[0072] In this optional embodiment, the pre-trained control strategy is converted from simulation instructions to electrical signals via a physical signal converter according to preset mapping rules. This ensures that control instructions in the simulation environment are accurately converted into specific electrical signals that can be recognized and executed by the physical agricultural robot, effectively resolving the "simulation-reality gap" between simulation and reality.

[0073] Optionally, after migrating the pre-trained control strategy to the physical agricultural robot through the dual-environment policy distillation technology to obtain the robot control strategy, the agricultural machinery automatic driving adaptation method further includes:

[0074] Obtaining delay characteristics of each component of the physical agricultural robot;

[0075] constructing a delay compensation model based on the delay characteristics and the state predictor;

[0076] When a signal delay of the physical agricultural robot is detected, a compensation signal is generated according to the delay predicted by the delay compensation model;

[0077] The action of the physical agricultural robot is adjusted according to the compensation signal.

[0078] Specifically, due to the response delays of mechanical and electrical components in actual agricultural robots, this delay can cause a time lag between receiving a control signal and executing the control action, which in turn affects the robot's operational accuracy and efficiency. Based on the delay characteristics of each component in the agricultural robot, the response delay of each component under different input signals is determined (for example, the time difference between an accelerator pedal command and an actual speed change, and the time difference between a steering wheel angle command and an actual direction change). A state predictor predicts the robot's state after this response delay based on the response delay. Based on the delay characteristics of each component and the prediction results of the state predictor, a delay compensation model is constructed. The state predictor can be constructed based on an LSTM neural network. The actual state lag is calculated as the time it takes for the actual state to reach the expected state minus the theoretical time it takes to reach the expected state. This time difference is then used to update the state predictor. For example, in angle control, if the current angle is 0 and the control command changes the angle to 15, the expected change time is 1 second, but the actual execution time is 2 seconds. The state predictor is then updated based on this difference. After receiving a control command at the current moment, the delay compensation model predicts the delay and generates a compensation signal to adjust the control command. For example, during a turning maneuver, if an 80ms delay is detected in the overall control link, the state predictor predicts the actual position and posture of the agricultural machine 80ms later and adjusts the current steering angle accordingly to offset the trajectory deviation caused by the delay. The delay compensation model predicts the delay between the actual response and the ideal response and generates a compensation signal to adjust the control command and thus the movement of the physical agricultural robot. During actual operation, by comparing the difference between the predicted state and the actual response, the parameters of the delay compensation model are dynamically adjusted to improve compensation accuracy. Based on the difference between the actual response and the predicted state, the delay parameters in the state predictor are updated, thereby improving the accuracy of future state predictions and further reducing the impact of delay.

[0079] In this optional embodiment, a delay compensation model is used to eliminate policy delays caused by hardware response lag, ensuring that the agricultural robot can make timely and accurate control decisions based on its current state when performing tasks. By predicting the agricultural robot's state and adjusting control commands in advance, action delays or deviations caused by hardware response lag can be effectively reduced. Furthermore, by eliminating the impact of delays, the agricultural robot can execute control commands more quickly and accurately, improving operational efficiency. In the face of emergencies, it can promptly and accurately perform emergency braking, obstacle avoidance, and other operations, improving the safety and reliability of the agricultural robot and reducing the risk of accidents.

[0080] Optionally, the robot manipulation strategy is adjusted based on the real farmland environment using a dynamic curriculum learning mechanism to obtain a field manipulation adaptation strategy including:

[0081] Divide the operation into multiple stages according to the complexity of the operation scenario;

[0082] Obtaining a trajectory tracking error of the physical agricultural robot in the current operation phase;

[0083] When the trajectory tracking error is less than or equal to a preset threshold for multiple consecutive preset cycles, the physical agricultural robot is controlled to adjust to the next operation stage until the operation of the operation scene is completed, thereby obtaining the field control adaptation strategy.

[0084] Specifically, the entire operation process is divided into multiple stages based on the actual needs and difficulty of farmland operations. The operation scenarios can be divided into three stages: the primary stage involves straight-line driving, including straight-line driving and low-speed operation, testing the robot's basic speed and direction control capabilities; the intermediate stage involves curved driving, introducing curved driving and simple obstacle avoidance tasks to test the robot's path planning capabilities; and the advanced stage involves complex scenarios, including more complex tasks such as complex terrain (such as slopes), dynamic obstacle avoidance, and variable operating speed adjustments. During each operation stage, the agricultural robot's actual driving trajectory is continuously monitored and recorded, and compared with the preset path. Lateral deviation and heading angle deviation are used as error indicators. During the execution of the operation stage, the trajectory tracking error is continuously monitored. If the error is less than or equal to a threshold for multiple consecutive preset cycles, it indicates that the robot is performing well in the current stage and can be controlled to adjust to the next operation stage. Conversely, if the error is greater than the threshold for multiple consecutive preset cycles, it indicates that the robot is encountering difficulties in the current stage, and human operators can take over the robot's control to adjust and optimize the operation strategy in real time. By analyzing the actions taken during a worker's takeover, the robot learns the correct actions to take in similar situations, continuously improving its performance. Retraining and adjustments are performed based on high-quality training samples obtained during a worker's takeover, until the robot completes the entire operation scenario. This results in a field-adapted control strategy that is tailored to the actual farmland environment.

[0085] In this optional embodiment, by dividing the work scenario into multiple operational phases with gradually increasing complexity, the robot can adapt to different environments and task requirements in stages. This phased operation reduces the overall difficulty of the task, making it easier for the robot to complete each stage, ultimately increasing the success rate of the entire work scenario. This allows the robot to gradually adapt to environments of varying complexity, improving its adaptability in complex farmland environments.

[0086] Optionally, the robot manipulation strategy is adjusted based on a real farmland environment using a dynamic curriculum learning mechanism to obtain a field manipulation adaptation strategy, further comprising:

[0087] When the predicted collision time between the physical agricultural robot and the obstacle is less than the preset collision time and / or the lateral safety margin of the physical agricultural robot is continuously less than the preset safety margin within a preset continuous time, the physical agricultural robot is controlled to trigger emergency braking, and the operation of the physical agricultural robot is controlled by a takeover mechanism.

[0088] Specifically, to ensure the safe operation of agricultural robots in complex and dynamic working environments, a human takeover mechanism is implemented to take over control of the robot in emergency situations. Radar sensors are deployed in a real-world farmland environment to provide real-time environmental information surrounding the agricultural robot. Radar data continuously monitors the presence of static or dynamic obstacles ahead and, based on current speed, relative distance, and motion trends, predicts the collision time with the obstacle. When the predicted collision time falls below a preset value, emergency braking is immediately triggered to rapidly slow the robot and avoid a collision with the obstacle. The lateral safety margin between the agricultural robot and the obstacle is determined during travel—that is, the minimum lateral distance between the robot's centerline and the obstacle's edge. If the obtained lateral safety margin of the physical agricultural robot remains below the preset safety margin for a continuous period of time, the physical agricultural robot is controlled to trigger emergency braking. Simultaneously with emergency braking, the takeover mechanism controls the robot's operation, further adjusting the robot's direction to avoid collisions and protect the robot, the obstacle, and the surrounding environment. Timely braking and operational adjustments reduce downtime and repair costs caused by collisions, ensuring operational continuity.

[0089] In this optional embodiment, by monitoring the distance and relative speed between the physical agricultural robot and obstacles, predicting the collision time, and monitoring the lateral safety margin, potential collision risks can be promptly identified. If the predicted collision time is less than a preset value or the lateral safety margin is insufficient, emergency braking is immediately triggered and a human takeover mechanism is initiated to prevent a collision. By promptly triggering emergency braking and human takeover, the robot's limited capabilities in complex scenarios can be compensated, improving the success rate of operations.

[0090] like Figure 2 As shown, an embodiment of the present invention provides an automatic driving adaptation device 200, including:

[0091] An acquisition module 210 is configured to train the robot to obtain a pre-trained control strategy using a reinforcement learning algorithm based on a simulation environment;

[0092] A migration module 220 is configured to migrate the pre-trained manipulation strategy to a physical agricultural robot through a dual-environment strategy distillation technique to obtain a robot manipulation strategy;

[0093] The adjustment module 230 is used to adjust the robot control strategy based on the real farmland environment and utilize a dynamic curriculum learning mechanism to obtain a field control adaptation strategy.

[0094] like Figure 3 As shown, an electronic device 300 provided by an embodiment of the present invention includes a processor 310 and a memory 320; the processor 310 is used to store a computer program; the memory 320 is used to implement the above-mentioned agricultural machinery automatic driving adaptation method when executing the computer program.

[0095] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned agricultural machinery automatic driving adaptation method is implemented.

[0096] An electronic device 300 that can serve as a server or client of the present invention will now be described, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device 300 is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 300 can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0097] The electronic device 300 includes a computing unit that can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) or a computer program loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for device operation can also be stored. The computing unit, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0098] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM). In this application, the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments of the present invention. In addition, the functional units in the various embodiments of the present invention can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The above-mentioned integrated units can be implemented in the form of hardware or software functional units.

[0099] Although the present invention is disclosed as above, the protection scope of the present invention is not limited thereto. Those skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention, and these changes and modifications will fall within the protection scope of the present invention.

Claims

1. A method for adapting automatic driving of agricultural machinery, characterized in that: include: Based on the simulation environment, the robot is trained through reinforcement learning algorithms to obtain pre-trained control strategies; The pre-trained manipulation strategy is transferred to the physical agricultural robot through the dual-environment strategy distillation technology to obtain the robot manipulation strategy; Based on the real farmland environment, the robot control strategy is adjusted using a dynamic curriculum learning mechanism to obtain a field control adaptation strategy.

2. The agricultural machinery automatic driving adaptation method according to claim 1, characterized in that: The method of training the robot to obtain a pre-trained control strategy by using a reinforcement learning algorithm based on a simulation environment includes: Constructing a motion space of the robot, the motion space including pedal force and steering wheel angle of the robot; Based on the action space, a multi-dimensional reward function is constructed through safety indicators, energy consumption indicators and trajectory tracking errors; The pre-trained control strategy is obtained through the multi-dimensional reward function.

3. The agricultural machinery automatic driving adaptation method according to claim 1, characterized in that: The method of training the robot to obtain a pre-trained control strategy through a reinforcement learning algorithm based on a simulation environment further includes: The pre-trained control strategy is obtained by randomly perturbing physical parameters in the simulation environment and using the randomly perturbed physical parameters to train the robot, wherein the physical parameters include transmission efficiency and tire slip rate.

4. The agricultural machinery automatic driving adaptation method according to claim 2, characterized in that: The safety indicators include the collision probability between the robot and an obstacle and the emergency braking trigger rate when the robot detects a potential collision risk; the energy consumption indicators include the fuel consumption per unit operating area of ​​the robot and the peak power ratio; the trajectory tracking error is determined by the lateral deviation between the actual position of the robot and the preset path and the heading angle deviation between the actual driving direction of the robot and the driving direction of the preset path.

5. The agricultural machinery automatic driving adaptation method according to claim 1, characterized in that: The dual-environment strategy distillation technology is used to transfer the pre-trained control strategy to the physical agricultural robot to obtain the robot control strategy, including: A lightweight neural network model is constructed on the physical agricultural robot, and the pre-trained control strategy is migrated to the lightweight neural network model using importance sampling technology to obtain the robot control strategy, wherein the lightweight neural network model is used to learn the pre-trained control strategy.

6. The agricultural machinery automatic driving adaptation method according to claim 1, characterized in that: Before migrating the pre-trained control strategy to the physical agricultural robot through the dual-environment strategy distillation technology to obtain the robot control strategy, the agricultural machine automatic driving adaptation method further includes: According to preset mapping rules, the pre-trained control strategy is converted from simulation instructions into electrical signals through a physical signal converter.

7. The method for adapting automatic driving of agricultural machinery according to claim 1, characterized in that: After migrating the pre-trained control strategy to the physical agricultural robot through the dual-environment policy distillation technology to obtain the robot control strategy, the agricultural machine automatic driving adaptation method further includes: Obtaining delay characteristics of each component of the physical agricultural robot; constructing a delay compensation model based on the delay characteristics and the state predictor; When a signal delay of the physical agricultural robot is detected, a compensation signal is generated according to the delay predicted by the delay compensation model; The action of the physical agricultural robot is adjusted according to the compensation signal.

8. The agricultural machinery automatic driving adaptation method according to claim 1, characterized in that: Based on the real farmland environment, the robot control strategy is adjusted by using a dynamic course learning mechanism to obtain a field control adaptation strategy including: Divide the operation into multiple stages according to the complexity of the operation scenario; Obtaining a trajectory tracking error of the physical agricultural robot in the current operation phase; When the trajectory tracking error is less than or equal to a preset threshold for multiple consecutive preset cycles, the physical agricultural robot is controlled to adjust to the next operation stage until the operation of the operation scene is completed, thereby obtaining the field control adaptation strategy.

9. The agricultural machinery automatic driving adaptation method according to claim 7, characterized in that: The robot control strategy is adjusted based on the real farmland environment using a dynamic course learning mechanism to obtain a field control adaptation strategy, which also includes: When the predicted collision time between the physical agricultural robot and the obstacle is less than the preset collision time and / or the lateral safety margin of the physical agricultural robot is continuously less than the preset safety margin within a preset continuous time, the physical agricultural robot is controlled to trigger emergency braking, and the operation of the physical agricultural robot is controlled by a takeover mechanism.

10. An automatic driving adaptation device for agricultural machinery, characterized in that: include: The acquisition module is used to train the robot to obtain pre-trained control strategies through reinforcement learning algorithms based on a simulation environment; A migration module is used to migrate the pre-trained manipulation strategy to the physical agricultural robot through dual-environment strategy distillation technology to obtain the robot manipulation strategy; The adjustment module is used to adjust the robot control strategy based on the real farmland environment and utilize the dynamic curriculum learning mechanism to obtain a field control adaptation strategy.

Images