A control method and system for an autonomous harvester based on digital twins

Abstract

The present invention relates to the field of program control technology, and discloses a control method and system for an autonomous driving harvester based on digital twins. The method collects image information of a plot of land to be harvested by installing a monitoring device on the autonomous driving harvester, and pre-processes the collected image information of the plot of land to be harvested, and sets a coordinate system and a landmark in the pre-processed image information of the plot of land to be harvested, and accurately positions the autonomous driving harvester in real time based on the set coordinate system and landmark, and establishes a real-time prediction digital twin model based on the accurate positioning of the autonomous driving harvester, the setting of the coordinate system and the landmark through a Markov decision process, and controls the autonomous driving harvester based on the established real-time prediction digital twin model, thereby ensuring the accuracy, efficiency and real-time performance of the control of the autonomous driving harvester.

Description

Technical Field

[0001] The present invention relates to the field of program control technology, and specifically to a control method and system for an autonomous driving harvester based on digital twins. Background Art

[0002] As advanced driver assistance system technology continues to upgrade and improve and gradually develop towards intelligence, how to efficiently test and verify the rationality of autonomous driving algorithms and the stability of the system has become a major challenge in the industry. However, the current driver assistance system technology still has problems such as inaccurate positioning and insufficient driving control precision in complex environments.

[0003] The existing Chinese patent application CN115494772A obtains the path planning information of the vehicle's on-board high-precision map in real time by obtaining the vehicle's own information and the vehicle's surrounding environment information in real time; obtains relative information based on the vehicle's own information, the vehicle's surrounding environment information and the path planning information; constructs an information fusion space based on the vehicle's own information, the vehicle's surrounding environment information, the path planning information and the relative information; extracts key information from the information fusion space to construct a state space, and inputs it into a deep reinforcement learning model; the deep reinforcement learning model outputs control variables for autonomous driving control; converts the control variables into action command values ​​for the vehicle; wherein the path planning information is represented by continuous path points, and the continuous path points are a preset number of continuous path points following the current position of the vehicle. However, since it only obtains the path planning information of the vehicle's on-board high-precision map, it fails to consider the problem of inaccurate positioning in complex environments, and has certain limitations. Summary of the invention

[0004] Technical issues solved

[0005] In view of the shortcomings of the existing technology, the present invention provides an autonomous driving harvester control method and system based on digital twins, which has the advantages of accuracy, efficiency, and real-time, and solves the problems of inaccurate positioning and insufficient driving control precision in the current assisted driving system technology in complex environments.

[0006] (II) Technical solution

[0007] In order to solve the above-mentioned technical problems that the current assisted driving system technology still has inaccurate positioning and insufficient driving control precision in complex environments, the present invention provides the following technical solutions:

[0008] The present invention discloses a control method for an autonomous driving harvester based on digital twin, which specifically comprises the following steps:

[0009] S1. Collect image information of the plot to be harvested by installing a monitoring device on the automatic driving harvester, and pre-process the collected image information of the plot to be harvested to obtain the pre-processed image information of the plot to be harvested;

[0010] S2, setting a coordinate system and landmarks based on the pre-processed image information of the plot to be harvested;

[0011] S3, based on the set coordinate system and landmarks, accurately locate the autonomous harvester in real time during operation;

[0012] S4, based on the precise positioning of the autonomous harvester, setting coordinate system and landmarks, a real-time prediction digital twin model is established through the Markov decision process;

[0013] S5. Control the autonomous driving harvester based on the established real-time prediction digital twin model.

[0014] The present invention collects image information of the plots to be harvested by installing monitoring equipment on the autonomous driving harvester, and pre-processes the collected image information of the plots to be harvested, and sets coordinate systems and landmarks in the pre-processed image information of the plots to be harvested, and accurately positions the autonomous driving harvester in real time based on the set coordinate systems and landmarks during operation, and at the same time, based on the accurate positioning of the autonomous driving harvester, the setting of the coordinate systems and landmarks, a real-time prediction digital twin model is established through a Markov decision process, and the autonomous driving harvester is controlled based on the established real-time prediction digital twin model, thereby ensuring the accuracy, efficiency and real-time control of the autonomous driving harvester.

[0015] Preferably, the collecting of image information of the plots to be harvested by installing a monitoring device on the self-driving harvester and preprocessing the collected image information of the plots to be harvested comprises the following steps:

[0016] S11, installing a monitoring device on the self-driving harvester to capture multiple frames of image information of the plot to be harvested in real time;

[0017] S12, processing the image information of the plot to be harvested based on the multiple frames shot using a RANSAC algorithm;

[0018] The RANSAC algorithm processes the image information of the multiple frames of land to be harvested, including the following steps:

[0019] S121, randomly selecting m ground points in the image information of the land plot to be harvested in the kth frame;

[0020] S122, fitting the selected m ground points to construct a fitting model;

[0021] S123, calculating the distances from other ground points in the image information of the plot to be harvested in the k-th frame to the fitting model, setting a distance threshold, and when the distances from other ground points in the image information of the plot to be harvested in the k-th frame to the fitting model are less than the set distance threshold, setting the ground point as an inlier; when the distances from other ground points in the image information of the plot to be harvested in the k-th frame to the fitting model are greater than or equal to the set distance threshold, setting the ground point as a noise point, and counting the number of inliers;

[0022] S124, repeat steps S121-S123 to select the model with the largest number of inliers;

[0023] S125. Set the model with the largest number of internal points as the pre-processed image information of the plot to be harvested.

[0024] The present invention ensures the accuracy of recognition and measurement of the automatic driving harvester by installing a monitoring device on the automatic driving harvester to capture multiple frames of image information of the plots to be harvested in real time and preprocessing the captured multiple frames of image information of the plots to be harvested.

[0025] Preferably, the setting of the coordinate system and the landmark based on the pre-processed image information of the plot to be harvested comprises the following steps:

[0026] The user manually inputs the coordinate origin, landmark interval distance and number of landmarks in the pre-processed image information of the plot to be harvested.

[0027] The present invention sets the coordinate origin, landmark spacing and number of punctuation marks in the preprocessed image information of the plot to be harvested by manual input, thereby improving the degree of freedom of the automatic driving harvester and ensuring the high efficiency of the automatic driving harvester.

[0028] Preferably, the real-time accurate positioning of the autonomous driving harvester during operation based on the set coordinate system and landmarks includes the following steps:

[0029] S31, establishing a motion model of the autonomous driving harvester and an observation model of the autonomous driving harvester based on real-time measurement results of monitoring equipment installed on the autonomous driving harvester;

[0030] S32, setting error terms of the automatic driving harvester motion model and the automatic driving harvester observation model;

[0031] S33. The nonlinear least squares method is used iteratively to accurately position the autonomous driving harvester during operation.

[0032] Preferably, the step of establishing the autonomous driving harvester motion model and the autonomous driving harvester observation model based on the real-time measurement results of the monitoring equipment installed on the autonomous driving harvester comprises the following steps:

[0033] The motion model of the autonomous harvester is as follows:

[0034]

[0035] in, represents the position of the autonomous harvester in the set coordinate system at time t, Indicates the position of the autonomous harvester in the set coordinate system at time t-1, represents the measurement data of the monitoring equipment of the autonomous driving harvester at time t, represents the measurement noise data during the measurement process of the autonomous driving harvester monitoring equipment at time t, represents the motion equation of the autonomous harvester, Indicates about and The equations of motion for the autonomous harvester;

[0036] The observation model of the autonomous harvester is as follows:

[0037]

[0038] in, Indicates that the monitoring equipment of the autonomous harvester is at time t Punctuation Points The measurement data, Indicates the i-th punctuation point in the set coordinate system, represents the observation equation of the autonomous harvester, Indicates about and The observation equation of the autonomous harvester, Indicates the calibration point during the measurement process of the self-driving harvester monitoring equipment at time t The measurement noise data.

[0039] Preferably, the setting of the error terms of the automatic driving harvester motion model and the automatic driving harvester observation model comprises the following steps:

[0040] The error terms of the autonomous harvester motion model and the autonomous harvester observation model are as follows:

[0041]

[0042] in, represents the error term of the autonomous harvester motion model, Represents the error term of the observation model of the autonomous harvester.

[0043] Preferably, the method of iteratively using the nonlinear least squares method to accurately locate the automatic driving harvester during operation includes the following steps:

[0044] The nonlinear least squares method is shown below:

[0045]

[0046] in, is the nonlinear least squares function of the position x of the autonomous harvester, indicating the precise positioning of the current autonomous harvester, T represents the transposition, represents the data matrix of the position of the autonomous harvester at time t, represents the observation data matrix of the autonomous harvester at time t and punctuation, represents the error matrix of the autonomous harvester motion model, represents the error matrix of the observation model of the autonomous harvester, represents the error matrix of the autonomous harvester motion model, Represents the error matrix of the observation model of the autonomous harvester.

[0047] The present invention establishes an autonomous driving harvester motion model and an autonomous driving harvester observation model based on real-time measurement results of monitoring equipment installed on the autonomous driving harvester, and through iterative use of the nonlinear least squares method, completes the precise positioning of the autonomous driving harvester during operation, thereby ensuring the accuracy of the positioning of the autonomous driving harvester.

[0048] Preferably, the process of establishing a real-time prediction digital twin model based on the precise positioning of the autonomous driving harvester, setting the coordinate system and landmarks through a Markov decision process includes the following steps:

[0049] S41. Set the state set S of the autonomous driving harvester = { , , ..., },in represents the first state of the autonomous harvester, where represents the g-th state of the autonomous harvester; the action set A={ , , ..., }in represents the first action of the autonomous harvester in the first state, where represents the jth state of the gth state of the autonomous harvester, where each state of the autonomous harvester includes the precise positioning, set coordinate system and landmarks of the current autonomous harvester;

[0050] S42, calculating the initial probability based on the set state set and action set of the autonomous driving harvester, and calculating the one-step state transition probability;

[0051] The calculation of one-step state transition probability is as follows:

[0052] One-step state transition probability distribution:

[0053] in, is the probability distribution of one-step state transition, Indicates the state of the autonomous harvester After being processed by the reward function, it reaches the state of the autonomous driving harvester The probability of represents the reward function, , Indicates status Execute an action The reward value at time

[0054] S43, calculating the multi-step state transition probability based on the calculated one-step state transition probability;

[0055] S44, establishing a real-time prediction digital twin model based on the output estimated state set of the autonomous driving harvester, the action set corresponding to each state, and the probability of execution of the corresponding action;

[0056] S45. Perform real-time prediction of the autonomous driving harvester based on the established real-time prediction digital twin model, and make real-time adjustments to the established real-time prediction digital twin model through a probability estimation algorithm.

[0057] Preferably, the step of calculating the multi-step state transition probability based on the calculated one-step state transition probability comprises the following steps:

[0058] set up Indicates the number of times the action is executed. Represents the set of all actions of the autonomous harvester in the first state;

[0059]

[0060] in, Indicates the set of actions that the autonomous harvester has gone through in the first state Arrival state Self-driving harvester state The number of times; The set of actions executed by the autonomous harvester in the first state The number of times Indicates the set of actions that the autonomous harvester has gone through in the first state Arrival state Self-driving harvester state probability;

[0061] The process from initial positioning to destination positioning of the autonomous harvester is decomposed through the coordinate system and landmarks set by the autonomous harvester, and the process from initial positioning to destination positioning of the autonomous harvester is divided into separate small states.

[0062] The probability estimation algorithm is used to estimate each individual state of the process from initial positioning to destination positioning of the autonomous driving harvester, and the estimated autonomous driving harvester state set, the action set corresponding to each state and the probability of executing the corresponding action are output.

[0063] The present invention establishes a real-time prediction digital twin model based on the precise positioning of the autonomous driving harvester, setting the coordinate system and landmarks through the Markov decision process, and controls the autonomous driving harvester in real time through the established real-time prediction digital twin model, thereby ensuring the accuracy and efficiency of the operation of the autonomous driving harvester.

[0064] The present invention also discloses a digital twin-based autonomous driving harvester control system, comprising a data acquisition module, installed monitoring equipment, a database, a real-time positioning module, a digital twin module, and a control module;

[0065] The data acquisition module is used to collect real-time monitoring data of the monitoring equipment installed on the automatic driving harvester, and transmit the real-time collected monitoring data to the real-time positioning module;

[0066] The installed monitoring equipment is used to collect image information of the plots to be harvested in real time;

[0067] The database is used to store data collected in real time by the data acquisition module;

[0068] The real-time positioning module is used to perform real-time positioning according to the received real-time collected monitoring data, and transmit the real-time positioning data to the digital twin module;

[0069] The digital twin module is used to establish a real-time prediction digital twin model based on the received real-time positioning data;

[0070] The control module is used to control the autonomous driving harvester according to the established real-time prediction digital twin model.

[0071] (III) Beneficial effects

[0072] Compared with the prior art, the present invention provides a control method and system for an autonomous driving harvester based on digital twins, which has the following beneficial effects:

[0073] 1. The invention collects image information of the plots to be harvested by installing monitoring equipment on the autonomous driving harvester, and pre-processes the collected image information of the plots to be harvested, and sets coordinate systems and landmarks in the pre-processed image information of the plots to be harvested, and accurately positions the autonomous driving harvester in real time based on the set coordinate systems and landmarks during operation, and establishes a real-time prediction digital twin model based on the precise positioning of the autonomous driving harvester, the setting of the coordinate system and landmarks through the Markov decision process, and controls the autonomous driving harvester based on the established real-time prediction digital twin model, thereby ensuring the accuracy, efficiency and real-time control of the autonomous driving harvester.

[0074] 2. The invention ensures the accuracy of recognition and measurement of the autonomous driving harvester by installing a monitoring device on the autonomous driving harvester to capture multiple frames of image information of the land to be harvested in real time and pre-processing the multiple frames of image information of the land to be harvested.

[0075] 3. The invention, by means of manual input, sets the coordinate origin, landmark spacing distance and number of punctuation marks in the pre-processed image information of the plot to be harvested, thereby improving the freedom of the autonomous driving harvester and ensuring the high efficiency of the autonomous driving harvester.

[0076] 4. This invention establishes an autonomous driving harvester motion model and an autonomous driving harvester observation model based on real-time measurement results of monitoring equipment installed on the autonomous driving harvester and iteratively uses the nonlinear least squares method to accurately position the autonomous driving harvester during operation, thereby ensuring the accuracy of the positioning of the autonomous driving harvester.

[0077] 5. This invention establishes a real-time prediction digital twin model based on the precise positioning of the autonomous driving harvester, setting the coordinate system and landmarks through the Markov decision process, and controls the autonomous driving harvester in real time through the established real-time prediction digital twin model, thereby ensuring the accuracy and efficiency of the operation of the autonomous driving harvester. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 It is a schematic diagram of the flow structure of the automatic driving harvester control method based on digital twin of the present invention. DETAILED DESCRIPTION

[0079] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0080] The present invention discloses a control method for an autonomous driving harvester based on digital twin, which specifically comprises the following steps:

[0081] S1. Collect image information of the plot to be harvested by installing a monitoring device on the automatic driving harvester, and pre-process the collected image information of the plot to be harvested to obtain the pre-processed image information of the plot to be harvested;

[0082] The image information of the plot to be harvested is collected by installing a monitoring device on the self-driving harvester, and the image information of the plot to be harvested is preprocessed, including the following steps:

[0083] S11, installing a monitoring device on the self-driving harvester to capture multiple frames of image information of the plot to be harvested in real time;

[0084] S12, processing the image information of the plot to be harvested based on the multiple frames shot using a RANSAC algorithm;

[0085] The RANSAC algorithm processes the image information of the multiple frames of land to be harvested, including the following steps:

[0086] S121, randomly selecting m ground points in the image information of the land plot to be harvested in the kth frame;

[0087] S122, fitting the selected m ground points to construct a fitting model;

[0088] S123, calculating the distances from other ground points in the image information of the plot to be harvested in the k-th frame to the fitting model, setting a distance threshold, and when the distances from other ground points in the image information of the plot to be harvested in the k-th frame to the fitting model are less than the set distance threshold, setting the ground point as an inlier; when the distances from other ground points in the image information of the plot to be harvested in the k-th frame to the fitting model are greater than or equal to the set distance threshold, setting the ground point as a noise point, and counting the number of inliers;

[0089] S124, repeat steps S121-S123 to select the model with the largest number of inliers;

[0090] S125, setting the model with the largest number of internal points as the pre-processed image information of the plot to be harvested;

[0091] S2, setting a coordinate system and landmarks based on the pre-processed image information of the plot to be harvested;

[0092] The steps of setting the coordinate system and landmarks based on the pre-processed image information of the plot to be harvested include:

[0093] The user manually inputs the coordinate origin, landmark interval distance and number of landmarks in the pre-processed image information of the plot to be harvested;

[0094] S3, based on the set coordinate system and landmarks, accurately locate the autonomous harvester in real time during operation;

[0095] The following steps are used to accurately locate the autonomous harvester in real time based on the set coordinate system and landmarks:

[0096] S31, establishing a motion model of the autonomous driving harvester and an observation model of the autonomous driving harvester based on real-time measurement results of monitoring equipment installed on the autonomous driving harvester;

[0097] The motion model of the autonomous harvester is as follows:

[0098]

[0099] in, represents the position of the autonomous harvester in the set coordinate system at time t, Indicates the position of the autonomous harvester in the set coordinate system at time t-1, represents the measurement data of the monitoring equipment of the autonomous driving harvester at time t, represents the measurement noise data during the measurement process of the autonomous driving harvester monitoring equipment at time t, represents the motion equation of the autonomous harvester, Indicates about and The equations of motion for the autonomous harvester;

[0100] The observation model of the autonomous harvester is as follows:

[0101]

[0102] in, Indicates that the monitoring equipment of the autonomous harvester is at time t Punctuation Points The measurement data, Indicates the i-th punctuation point in the set coordinate system, represents the observation equation of the autonomous harvester, Indicates about and The observation equation of the autonomous harvester, Indicates the calibration point during the measurement process of the self-driving harvester monitoring equipment at time t The measurement noise data of

[0103] S32, setting error terms of the automatic driving harvester motion model and the automatic driving harvester observation model;

[0104] The error terms of the autonomous harvester motion model and the autonomous harvester observation model are as follows:

[0105]

[0106] in, represents the error term of the autonomous harvester motion model, represents the error term of the observation model of the autonomous harvester;

[0107] S33, accurately positioning the autonomous driving harvester during operation by iteratively using a nonlinear least squares method;

[0108] The nonlinear least squares method is shown below:

[0109]

[0110] in, is the nonlinear least squares function of the position x of the autonomous harvester, indicating the precise positioning of the current autonomous harvester, T represents the transposition, represents the data matrix of the position of the autonomous harvester at time t, represents the observation data matrix of the autonomous harvester at time t and punctuation, represents the error matrix of the autonomous harvester motion model, represents the error matrix of the observation model of the autonomous harvester, represents the error matrix of the autonomous harvester motion model, represents the error matrix of the observation model of the autonomous harvester;

[0111] S4, based on the precise positioning of the autonomous harvester, setting coordinate system and landmarks, a real-time prediction digital twin model is established through the Markov decision process;

[0112] The establishment of a real-time prediction digital twin model based on the precise positioning of the autonomous harvester, setting the coordinate system and landmarks through the Markov decision process includes the following steps:

[0113] S41. Set the state set S of the autonomous driving harvester = { , , ..., },in represents the first state of the autonomous harvester, where represents the g-th state of the autonomous harvester; the action set A={ , , ..., }in represents the first action of the autonomous harvester in the first state, where represents the jth state of the gth state of the autonomous harvester, where each state of the autonomous harvester includes the precise positioning, set coordinate system and landmarks of the current autonomous harvester;

[0114] S42, calculating the initial probability based on the set state set and action set of the autonomous driving harvester, and calculating the one-step state transition probability;

[0115] The calculation of one-step state transition probability is as follows:

[0116] One-step state transition probability distribution:

[0117] in, is the probability distribution of one-step state transition, Indicates the state of the autonomous harvester After being processed by the reward function, it reaches the state of the autonomous driving harvester The probability of represents the reward function, , Indicates status Execute an action The reward value at time

[0118] S43, calculating the multi-step state transition probability based on the calculated one-step state transition probability;

[0119] Estimate the possible states at each step through the probability estimation algorithm;

[0120] set up Indicates the number of times the action is executed. Represents the set of all actions of the autonomous harvester in the first state;

[0121]

[0122] in, Indicates the set of actions that the autonomous harvester has gone through in the first state Arrival state Self-driving harvester state The number of times; The set of actions executed by the autonomous harvester in the first state The number of times Indicates the set of actions that the autonomous harvester has gone through in the first state Arrival state Self-driving harvester state probability;

[0123] Furthermore, the process of the autonomous harvester from initial positioning to destination positioning is decomposed through the coordinate system and landmarks set by the autonomous harvester, and the process of the autonomous harvester from initial positioning to destination positioning is divided into separate small states;

[0124] Furthermore, each individual state of the process from initial positioning to destination positioning of the autonomous driving harvester is estimated by a probability estimation algorithm, and the estimated autonomous driving harvester state set, the action set corresponding to each state, and the probability of executing the corresponding action are output;

[0125] S44, establishing a real-time prediction digital twin model based on the output estimated state set of the autonomous driving harvester, the action set corresponding to each state, and the probability of execution of the corresponding action;

[0126] S45. Perform real-time prediction on the autonomous driving harvester based on the established real-time prediction digital twin model, and adjust the established real-time prediction digital twin model in real time through a probability estimation algorithm;

[0127] S5. Control the autonomous harvester based on the established real-time prediction digital twin model;

[0128] The present invention also discloses a digital twin-based autonomous driving harvester control system, comprising a data acquisition module, installed monitoring equipment, a database, a real-time positioning module, a digital twin module, and a control module;

[0129] The data acquisition module is used to collect real-time monitoring data of the monitoring equipment installed on the automatic driving harvester, and transmit the real-time collected monitoring data to the real-time positioning module;

[0130] The installed monitoring equipment is used to collect image information of the plots to be harvested in real time;

[0131] The database is used to store data collected in real time by the data acquisition module;

[0132] The real-time positioning module is used to perform real-time positioning according to the received real-time collected monitoring data, and transmit the real-time positioning data to the digital twin module;

[0133] The digital twin module is used to establish a real-time prediction digital twin model based on the received real-time positioning data;

[0134] The control module is used to control the autonomous driving harvester according to the established real-time prediction digital twin model.

[0135] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A control method for an autonomous driving harvester based on digital twins, characterized in that: The following steps are involved: S1. Collect image information of the plot to be harvested by installing a monitoring device on the automatic driving harvester, and pre-process the collected image information of the plot to be harvested to obtain the pre-processed image information of the plot to be harvested; S2, setting a coordinate system and landmarks based on the pre-processed image information of the plot to be harvested; S3, based on the set coordinate system and landmarks, accurately locate the autonomous harvester in real time during operation; S4, based on the precise positioning of the autonomous harvester, setting coordinate system and landmarks, a real-time prediction digital twin model is established through the Markov decision process; S5. Control the autonomous harvester based on the established real-time prediction digital twin model; The real-time accurate positioning of the autonomous driving harvester during operation based on the set coordinate system and landmarks includes the following steps: S31, establishing a motion model of the autonomous driving harvester and an observation model of the autonomous driving harvester based on real-time measurement results of monitoring equipment installed on the autonomous driving harvester; S32, setting error terms of the automatic driving harvester motion model and the automatic driving harvester observation model; S33, accurately positioning the autonomous driving harvester during operation by iteratively using a nonlinear least squares method; The method of establishing the autonomous driving harvester motion model and the autonomous driving harvester observation model based on the real-time measurement results of the monitoring equipment installed on the autonomous driving harvester comprises the following steps: The motion model of the autonomous harvester is as follows: x t =f(x t-1 ,p t )+q t ; Among them, x t represents the position of the autonomous harvester in the set coordinate system at time t, x t-1 represents the position of the autonomous harvester in the set coordinate system at time t-1, p t represents the measurement data of the monitoring equipment of the autonomous harvester at time t, q t represents the measurement noise data of the self-driving harvester monitoring device during the measurement process at time t, f represents the motion equation of the self-driving harvester, and f(x t-1 ,p t ) represents the value of x t-1 and q t The equations of motion for the autonomous harvester; The observation model of the autonomous harvester is as follows: z t,i =h(x t ,y i )+q t,i ; Among them, z t,i Indicates that the monitoring equipment of the autonomous harvester is at x at time t t Point y i The measurement data, y i represents the i-th punctuation point in the set coordinate system, h represents the observation equation of the autonomous driving harvester, h(x t ,y i ) represents the value of x t and i The observation equation of the autonomous harvester, q t,i Indicates the reference point y during the measurement process of the self-driving harvester monitoring equipment at time t i The measurement noise data of The error terms of setting the motion model of the autonomous driving harvester and the observation model of the autonomous driving harvester include the following steps: The error terms of the autonomous harvester motion model and the autonomous harvester observation model are as follows: Among them, e t represents the error term of the autonomous harvester motion model, e t,i represents the error term of the observation model of the autonomous harvester, x t represents the position of the autonomous harvester in the set coordinate system at time t, f(x t-1 ,p t ) represents the value of x t-1 and q t The motion equation of the autonomous harvester, x t-1 represents the position of the autonomous harvester in the set coordinate system at time t-1, p t represents the measurement data of the monitoring equipment of the autonomous harvester at time t, z t,i Indicates that the monitoring equipment of the autonomous harvester is at x at time t t Point y i The measurement data, y i represents the i-th punctuation point in the set coordinate system, h represents the observation equation of the autonomous driving harvester, h(x t ,y i ) represents the value of x t and i The observation equation of the autonomous harvester, q t,i Indicates the reference point y during the measurement process of the self-driving harvester monitoring equipment at time t i The measurement noise data of The method of iteratively using the nonlinear least squares method to accurately locate the autonomous driving harvester during operation includes the following steps: The nonlinear least squares method is shown below: Among them, J(x) is the nonlinear least squares function of the position x of the autonomous harvester, which represents the precise positioning of the current autonomous harvester, T represents transposition, represents the data matrix of the position of the autonomous harvester at time t, represents the observation data matrix of the autonomous harvester at time t and punctuation, represents the error matrix of the autonomous harvester motion model, represents the error matrix of the observation model of the autonomous harvester, e t (x) represents the error matrix of the motion model of the autonomous harvester, e t,i (x) represents the error matrix of the observation model of the autonomous harvester; The method of establishing a real-time prediction digital twin model based on the precise positioning of the autonomous driving harvester, setting the coordinate system and landmarks through the Markov decision process includes the following steps: S41. Set the state set of the autonomous harvester S = {s1, s2, ..., s g }, where s g represents the g-th state of the autonomous harvester; the action set A = {a 11 , a 12 , ..., a gj }where a gj represents the jth state of the gth state of the autonomous harvester, where each state of the autonomous harvester includes the precise positioning, set coordinate system and landmarks of the current autonomous harvester; S42, calculating the initial probability based on the set state set and action set of the autonomous driving harvester, and calculating the one-step state transition probability; The calculation of one-step state transition probability is as follows: One-step state transition probability distribution: P:S×R,P(s1,a 11 ,s2):s1×R(s1,a 11 )→s2 Among them, P is the probability distribution of one-step state transition, P(s1,a 11 ,s2) represents the probability that the autonomous driving harvester state s1 reaches the autonomous driving harvester state s2 after being processed by the reward function, R represents the reward function, R=S×A,R(s1,a 11 )=s1×a 11 , R(s1,a 11 ) indicates that state s1 executes action a 11 The reward value at time S43, calculating the multi-step state transition probability based on the calculated one-step state transition probability; S44, establishing a real-time prediction digital twin model based on the output estimated state set of the autonomous driving harvester, the action set corresponding to each state, and the probability of execution of the corresponding action; S45. Perform real-time prediction on the autonomous driving harvester based on the established real-time prediction digital twin model, and adjust the established real-time prediction digital twin model in real time through a probability estimation algorithm; The step of calculating the multi-step state transition probability based on the calculated one-step state transition probability comprises the following steps: Estimate the possible states at each step through the probability estimation algorithm; Set N to represent the number of times the action is executed, a 1* Represents the set of all actions of the autonomous harvester in the first state; Among them, N(s1,a 1* , s2) represents the action set a of the autonomous harvester in the first state 1* The number of times the state of the automatic driving harvester is reached is s2; N(s1,a 1* ) The autonomous harvester performs action set a in the first state 1* The number of times, P(s1,a 1* , s2) represents the action set a of the autonomous harvester in the first state 1* The probability of reaching the state s2 of the autonomous harvester; The process from initial positioning to destination positioning of the autonomous harvester is decomposed through the coordinate system and landmarks set by the autonomous harvester, and the process from initial positioning to destination positioning of the autonomous harvester is divided into separate small states. Each individual state after division is estimated through the probability estimation algorithm, and the estimated state set of the autonomous driving harvester, the action set corresponding to each state and the probability of executing the corresponding action are output.

2. The method for controlling an autonomous driving harvester based on digital twin according to claim 1, characterized in that: The step of collecting image information of the plot to be harvested by installing a monitoring device on the self-driving harvester and preprocessing the collected image information of the plot to be harvested includes the following steps: S11, installing a monitoring device on the self-driving harvester to capture multiple frames of image information of the plot to be harvested in real time; S12, processing the image information of the plot to be harvested based on the multiple frames shot using a RANSAC algorithm; The RANSAC algorithm processes the image information of the multiple frames of land to be harvested, including the following steps: S121, randomly selecting m ground points in the image information of the land plot to be harvested in the kth frame; S122, fitting the selected m ground points to construct a fitting model; S123, calculating the distances from other ground points in the image information of the plot to be harvested in the k-th frame to the fitting model, setting a distance threshold, and when the distances from other ground points in the image information of the plot to be harvested in the k-th frame to the fitting model are less than the set distance threshold, setting the ground point as an inlier; when the distances from other ground points in the image information of the plot to be harvested in the k-th frame to the fitting model are greater than or equal to the set distance threshold, setting the ground point as a noise point, and counting the number of inliers; S124, repeat steps S121-S123 to select the model with the largest number of inliers; S125. Set the model with the largest number of internal points as the pre-processed image information of the plot to be harvested.

3. The method for controlling an autonomous driving harvester based on digital twin according to claim 1, characterized in that: The method of setting the coordinate system and landmarks based on the pre-processed image information of the plot to be harvested comprises the following steps: The user manually inputs the coordinate origin, landmark interval distance and number of landmarks in the pre-processed image information of the plot to be harvested.

4. A system for implementing the digital twin-based autonomous driving harvester control method according to any one of claims 1 to 3, characterized in that: It includes data acquisition module, installed monitoring equipment, database, real-time positioning module, digital twin module and control module; The data acquisition module is used to collect real-time monitoring data of the monitoring equipment installed on the automatic driving harvester, and transmit the real-time collected monitoring data to the real-time positioning module; The installed monitoring equipment is used to collect image information of the plots to be harvested in real time; The database is used to store data collected in real time by the data acquisition module; The real-time positioning module is used to perform real-time positioning according to the received real-time collected monitoring data, and transmit the real-time positioning data to the digital twin module; The digital twin module is used to establish a real-time prediction digital twin model based on the received real-time positioning data; The control module is used to control the autonomous driving harvester according to the established real-time prediction digital twin model.

Images