Digital-twin-based gesture interaction robot seeding monitoring and compensation method

Abstract

The application provides a digital twin gesture interaction robot seeding monitoring and compensation method, relates to the field of intelligent seeding, and comprises the following steps: initializing a digital twin system, triggering a virtual robot to simulate, monitoring and trajectory prediction of operation driven by the digital twin, compensation control, and leak seeding fault detection; wherein the leak seeding fault detection specifically comprises the following steps: in the seeding process, the real robot's sensing system is used to monitor the leak seeding fault in real time, including the failure of seed to be normally planted, the lack of seeds in a seed box, or mechanical abnormalities, and the improved D-S evidence theory is used to accurately identify the fault category; then the virtual robot generates corresponding decision instructions according to the fault category. The method can improve the seeding quality, reduce labor input, enhance operation safety, and reduce seeding cost, and has good seeding stability, high safety factor, and effectively deals with the sudden conditions in the seeding process.

Description

Technical Field

[0001] This invention relates to the field of intelligent seeding technology, specifically to a digital twin gesture-interactive robot seeding monitoring and compensation method. Background Technology

[0002] Food security is a major strategic issue of overall importance to my country's national economic development, social stability, and national self-reliance. Currently, my country's sown area has reached 1.79 billion mu, of which corn, wheat, and other grains account for about 82%.

[0003] Grain seeders are planting machines that sow grain seeds. They have a significant impact on improving planting efficiency and yield, reducing planting costs, saving planting time, and expanding planting area. Currently, most grain seeders are tractor-drawn seeders. These structures result in heavy soil compaction, large turning radii at the field edge, and a rearward-shifted center of gravity, leading to poor sowing stability and large errors in experience-based adjustments. This not only makes it difficult to meet the agronomical requirements for precision sowing of grains such as corn and wheat, but also easily causes soil compaction or damages the sowing effect. Furthermore, existing grain seeders rely on manual operation, where the operator sits on the seeder to move and sow. However, manual operation makes it impossible to visually monitor the sowing process and identify missed sowing points in real time. This necessitates subsequent manual inspection and reseeding, affecting sowing efficiency and increasing the risk of missed or duplicate sowing, ultimately impacting sowing quality and yield. In addition, the high labor intensity of driving and replanting has led to increasingly prominent problems such as agricultural labor shortages and a significant increase in labor costs. Furthermore, when people are driving, sowing, and replanting in the field, they inhale a large amount of soil dust, which affects their health. Summary of the Invention

[0004] To address the problems existing in the prior art, the present invention aims to provide a digital twin gesture-interactive robot seeding monitoring and compensation method. This method transmits and communicates data on the real robot's dynamic seeding behavior and the virtual robot's characteristic parameters in real time. It uses data to drive the movement of the twin virtual robot, enabling visualized monitoring of the real robot's operation. Simultaneously, it uses gesture interaction to remotely control the real robot to perform operations, thereby improving seeding quality, reducing labor input, enhancing operational safety, and lowering seeding costs. The method offers good seeding stability, a high safety factor, and can effectively cope with unexpected situations that occur during the seeding process.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] A method for monitoring and compensating for seeding in a digital twin gesture-interactive robot, comprising: Step S1: Initialize the digital twin system: Start the real robot located in the field, collect its initial state parameters, and synchronize the initial state parameters of the real robot to the virtual robot; Step S2: Trigger the virtual robot to perform simulation: Trigger the virtual robot to perform simulation through gesture interaction. The virtual robot start command is transmitted to the industrial control computer of the real robot in real time, and the field sowing begins. Step S3, Digital Twin-Driven Operation Monitoring and Trajectory Prediction: The operation data of the real robot is transmitted to the virtual robot of the twin in real time. After receiving the operation data, the virtual robot replicates all the operation actions of the real robot and predicts the movement trajectory of the real robot in the field, the sowing action and the trajectory of the seeds. Step S4, Compensation Control: The virtual robot uses the predicted trajectory as a benchmark and obtains the actual deviation through dual-dimensional error detection; then it predicts the error trend through an improved PID control algorithm, and finally performs multi-dimensional compensation through heading angle, position, and visual positioning to achieve precise seeding; Step S5, Missed Seeding Fault Detection: During the seeding process, the real robot's sensing system monitors for missed seeding faults in real time, including seeds not falling properly, missing seeds in the seed box, or mechanical abnormalities. The improved DS evidence theory is used to accurately identify the fault category. Then, the virtual robot generates corresponding decision instructions based on the fault category.

[0007] Based on further optimization of the above scheme, the real robot includes a mobile chassis, frame, seeder, navigation vision camera, navigation and positioning module, seed box assembly, seed drop detection photoelectric sensor, replanting mechanism, industrial control computer, and display. The navigation and positioning module includes a lidar, a BeiDou satellite module, and an inertial measurement unit (IMU). The seed box assembly includes a seed box and a seed box detection camera and laser ranging sensor installed inside the seed box. The seed drop detection photoelectric sensor is installed between the seed box and the seeder. The replanting mechanism includes a replanting tube, a spring valve, and a push rod. The replanting tube is connected to the seed box and located behind the seeder. A spring valve is installed at the bottom of the replanting tube's inner cavity, and a push rod is installed above the spring valve to control its opening and closing. The chassis, seeder, navigation vision camera, navigation and positioning module, seed box assembly, replanting mechanism, industrial control computer, and display are all mounted on the frame.

[0008] Based on further optimization of the above scheme, in step S1, the acquired initial state parameters of the real robot include the position coordinates of the moving chassis ( x 0, y 0) Initial heading angle Initial height of seeds in the seed box h 0. Initial rotation speed of the seeding tray w 0. Initial speed of the moving chassis v 0 and a three-dimensional field map.

[0009] Based on further optimization of the above scheme, step S3 specifically includes: The real robot uses a navigation vision camera to identify the position and height of field ridges, field edges / obstacles in real time, while a lidar simultaneously collects distance data. A BeiDou satellite module and a chassis detection module acquire the coordinates of the moving chassis in real time. x t ,y t ), driving speed v t Heading angle The seed-dropping photoelectric sensor acquires real-time information on the seed-dropping status and simultaneously measures the real-time seed height within the seed box. h t All data is initially processed by the industrial control computer and then uploaded to the virtual robot every 0.1 to 0.2 seconds. The virtual robot generates a theoretical motion trajectory using a two-round differential kinematics model: ; In the formula: L lj Indicates the chassis wheel track; Indicates the steering wheel deflection angle; Meanwhile, the virtual robot uses an Extensive Kalman Filter (EKF) to filter and predict the output of the kinematic model, eliminating noise and predicting the chassis trajectory within the next 0.5 seconds. State prediction: ; Covariance prediction: ; In the formula: Represents the state vector; Indicates control input; F k Represents the state transition matrix; B k Represents the control input matrix; Q k Represents the process noise covariance; The virtual robot ultimately outputs a set of predicted trajectory points {( x 1 ,y 1), ( x 2 ,y 2),…, ( x n ,y n This enables visualization and early prediction of the movement trajectory of the mobile chassis. Meanwhile, the virtual robot rotates according to the seeding tray speed of the real robot. w bz Length of duckbill robotic arm lyz Establish a circular motion trajectory model for the tip of the duckbill: ; In the formula: ( x xz0 ,y xz0 () indicates the coordinates of the center of rotation of the seeding tray; Indicates the initial angle of the duckbill; t bz Indicates the running time of the seeding tray; The virtual robot predicts the opening or closing time and position of the duckbill using a circular motion trajectory model, and compares it with the photoelectric sensor data of the real robot to determine whether the sowing action is normal. In addition, the virtual robot incorporates field gravity acceleration. g t air drag coefficient c air Soil friction coefficient f mc Establish a three-dimensional motion trajectory model of the seed: ; In the formula: v zz0 Indicates the initial velocity of the seed falling; Indicates the angle at which the seed falls; t zz This indicates the time it takes for a seed to move after it falls, that is, the cumulative time from when the seed leaves the duck's beak and begins to fall. Indicates the height at which the duckbill lands; The virtual robot predicts the location where seeds will fall and compares it with the planned location on the field ridges to determine the risk of missed or off-center sowing.

[0010] Based on further optimization of the above scheme, step S4, "obtaining the true deviation through dual-dimensional error detection," specifically involves: Based on the field ridge line recognition results from the navigation vision camera, the virtual robot extracts the x-axis pixel deviation between the center of the real robot's mobile chassis and the planned center line of the field ridge. e px Thus, the actual x-axis deviation is obtained: ; In the formula: k ex Indicates the pixel-to-distance conversion factor; Field ridge direction obtained through navigation vision camera With the chassis planning heading angle The angle between the two points is used to obtain the heading deviation: ; The cumulative excitation error between the theoretical and actual travel distances of the mobile chassis is calculated to obtain the motion model deviation. ; In the formula: v act Indicates the actual driving speed; v plan Indicates the theoretical driving speed; The virtual robot has preset x-axis deviation threshold, heading deviation threshold, and motion model deviation threshold. If the detected deviation is less than the corresponding threshold, the current state is maintained; if it is greater than the threshold, the error correction process is triggered.

[0011] Based on further optimization of the above scheme, step S4, "predicting the error trend by improving the PID control algorithm," specifically involves: Establish an incremental PID control algorithm: ; ; In the formula: K p , K i , K d These represent the proportional, integral, and differential coefficients, respectively. Indicates the sampling period; Indicates control increment; in, e k To unify the error variables, the following results were obtained by weighted fusion of the actual x-axis deviation, heading deviation, and motion model deviation: ; In the formula: This represents the corresponding weighting coefficient; To address sudden errors caused by soil slippage and undulating field ridges, the virtual robot incorporates fuzzy control to manage error variables. e k and error change rate ec k As fuzzy input, ,Will K p , K i , K d Correction amount As a fuzzy output, a 7-level fuzzy set (including negative large, negative medium, negative small, zero, positive large, positive medium, and positive small) is established. The fuzzy rule base is trained using sample data from field operations to achieve real-time self-tuning of PID parameters. ; In the formula: These represent the PID parameters after self-tuning.

[0012] Based on further optimization of the above scheme, step S4, "multi-dimensional compensation through heading angle, position, and visual positioning," specifically includes: Based on the error prediction results, the virtual robot sends supplementary control commands to the real robot, achieving real-time error correction through three dimensions: heading angle compensation, position compensation, and visual positioning compensation. Heading angle compensation: based on heading angle deviation Adjusting the steering wheel angle of a real robot : ; In the formula: Indicates the heading angle compensation coefficient; Position compensation: for x-axis deviation in the width direction e x Lateral position compensation of the moving chassis is achieved by adjusting the speed difference between the left and right wheels: ; In the formula: This indicates the average speed of the mobile chassis during the current sampling period; k v Indicates the speed compensation coefficient; This indicates the compensated rotational speeds of the left and right wheels, and the difference in rotational speed causes the moving chassis to shift towards the center line of the field ridge. Visual positioning compensation: The virtual robot corrects its visual positioning coordinates by establishing a visual positioning deviation compensation matrix. ; In the formula: , Represents the x and y axis offsets for visual positioning; Indicates the actual coordinates after compensation; Motion model parameter update: If a deviation in the chassis motion model is detected e s If the deviation exceeds the motion model deviation threshold, then the virtual robot's wheelbase... L lj Wheel diameter D cl Update: ; This ensures that the kinematic model matches the actual driving state of the real robot, reducing model errors.

[0013] Based on further optimization of the above scheme, step S5, "accurately identifying fault categories using improved DS evidence theory," specifically involves: To address the uncertainty of missed seeding failures, the virtual robot, based on an improved DS evidence theory, achieves accurate identification of missed seeding failures through hierarchical extraction of failure features, basic probability allocation, and fusion of conflicting evidence, thus solving the problem of distorted results in conflicting evidence in traditional DS evidence theory. Hierarchical extraction of fault features: Establishing an identification framework for the three core fault types of missed broadcasts. ;in, A 1 indicates a problem with the duckbill not opening. A 2 indicates a blockage at the seed outlet. A 3 indicates a seed box missing seed fault. A x This indicates an unknown fault (i.e., a fault other than the three types of faults mentioned above). The virtual robot extracts three layers of fault characteristics from the real robot's photoelectric sensors, the seed box monitoring component consisting of a seed box detection camera and a laser rangefinder, and the seed tray rotation speed sensor, as the original evidence for evidence-based decision-making: The first layer of features (direct features): the seed detection signal from the photoelectric sensor, 0 for the presence of seeds and 1 for the absence of seeds; the second layer of features (mechanical features): the rotational speed fluctuation rate of the seeding tray, less than 5% is considered normal fluctuation, otherwise it is considered abnormal fluctuation; the third layer of features (material features): the seed box monitoring component identifies the seed height scale; the virtual robot uses these three layers of features as three independent sources of evidence. E 1. E 2. E 3; Basic probability allocation: Calculate the basic probability of failure for each source of evidence: ; In the formula: Indicate the source of evidence E i For the fault X Feature matching degree; Indicates the weighting coefficient of the evidence source; Simultaneously satisfy , ; Fusion of conflicting evidence: First, the degree of conflict among the three sources of evidence is measured to obtain the inter-evidence conflict coefficient. ; In the formula: Representing the sources of evidence respectively E 1. E 2. E 3. In the recognition framework Any possible subset of fault propositions in; , Indicates no conflict This indicates a complete conflict; Preset conflict threshold ,when Then, the evidence source with the lowest credibility is removed and the evidence is re-fused. Subsequently, based on the fault identification accuracy of the evidence source... With conflict coefficient To obtain the credibility of each source of evidence: ; Fault identification accuracy rate Obtained through offline calibration and online evaluation, it represents the performance metrics of each evidence source on historical fault data; it meets the following requirements. ; Finally, the basic probability allocations of each source of evidence are first weighted and adjusted, and then fused using the DS synthesis rule to obtain a comprehensive basic probability allocation: Weighted adjustment: ;in, Represents the prior probability of an unknown fault; Binary synthesis: for two sources of evidence E i , E j The basic probability distribution after their fusion is as follows: ; Triadic fusion: The result of binary fusion is fused again with a third source of evidence to obtain a comprehensive basic probability allocation. m ijz (X) .

[0014] Based on further optimization of the above scheme, in step S5, "the virtual robot generates corresponding decision instructions according to the fault category" specifically means: First, virtual robots have a more comprehensive basic probability allocation. m ijz (X) Develop dual-threshold fault decision rules to achieve accurate identification and decision-making regarding missed broadcast faults: Maximum probability rule: ;in, X Indicates a candidate fault; Probability difference threshold rule: ;in, This represents the fault type with the second largest value in the comprehensive basic probability distribution. Indicates the probability difference threshold; Unknown fault threshold rules: ;in, Indicates the threshold for the probability of unknown faults; If a detected fault meets all three of the above rules, then the fault is determined to be... X If the maximum probability rule m (X ) The corresponding fault type did not meet the double threshold rule, i.e. and Meanwhile, if no seeds fall from the photoelectric sensor, check the previous N consecutive sowing cycles. If the seed drop detection is normal, the fluctuation rate of the sowing tray speed is normal, and there are enough seeds in the seed box, then it is determined to be a single missed sowing (i.e. the duckbill did not open once, and it is not a continuous failure). Trigger the corresponding fault handling process; Single missed broadcast: based on the real robot's real-time driving speed v t With replanting response time t bbz To obtain the plant spacing between the replanting tube and the missed sowing location: L z = v t · t bbz The virtual robot immediately transmits the replanting command to the real robot, which pushes the spring valve downward via a push rod to achieve precise replanting. After replanting is completed, the replanting mechanism resets, the real robot continues sowing, and the virtual robot synchronously simulates the replanting action and records the replanting information. The problem was determined to be a duckbill not opening malfunction. A 1. Clogged or blocked seed outlet A 2 (Mechanical Failure): The virtual robot generates an immediate stop command through gesture interaction, the real robot stops and ensures that all mechanisms are reset, and the operator handles the fault on-site according to the location and type of the fault; after the fault is resolved, the sowing process is resumed by gesture interaction. The problem was determined to be a seed box missing seeds. A 3. The virtual robot generates a stop command. After the operator re-sows the seed box on site, the operator can start sowing through gesture interaction to resume the sowing process. At the same time, the virtual robot updates the seed box status parameters through the seed box detection camera and laser rangefinder in the seed box cavity. Determined as an unknown fault A xThe virtual robot initiates behavior simulation, simulates the operational characteristics of different faults, and compares them with the abnormal data of the real robot; the operator analyzes the simulation results and manually determines the fault type based on the sensor data, and then takes corresponding actions according to the fault type.

[0015] The following are the technical effects of the present invention: This application integrates multiple technologies such as digital twins, machine vision, satellite navigation, gesture interaction, and sensor monitoring to address issues such as trajectory deviation, low sowing accuracy, inaccurate identification of missed sowing faults, and untimely fault handling in field robot sowing. It achieves simultaneous virtual and real control of sowing operations, dynamic and precise compensation, accurate fault identification, and closed-loop intelligent processing.

[0016] By initializing the digital twin system, the initial state dimensions of the real and virtual robots are aligned, ensuring high-precision digital modeling of the field environment and equipment status, laying a deviation-free benchmark for subsequent synchronous virtual-real operations. Using gesture-triggered simulation not only improves the convenience and anti-interference capability of starting field operations and adapts to complex field environments, ensuring synchronized triggering of virtual and real robot operation commands and avoiding asynchrony caused by command transmission delays, but also avoids operators being exposed to prolonged field conditions during sowing and protects their health from soil dust. Through digital twin-driven operation monitoring and trajectory prediction, multi-dimensional real-time prediction and visualization of sowing operation movement trajectories, sowing actions, and seed drop trajectories are achieved, transforming the sowing operation process from a mere "event" to a comprehensive and efficient one. The system upgrades from "post-correction" to "pre-judgment," significantly reducing the probability of deviations and malfunctions. It also enables remote monitoring of operational status and reduces manual supervision costs. Through compensatory control, it dynamically, adaptively, and multi-dimensionally compensates for sowing deviations caused by complex field environments, greatly improving the robot's sowing position accuracy and trajectory tracking precision. This solves problems such as off-center sowing and uneven plant spacing caused by environmental interference. Furthermore, through missed sowing fault detection, it achieves accurate classification and identification of missed sowing faults, exhibiting good anti-interference capabilities and strong conflict information processing ability. Simultaneously, by utilizing differentiated fault handling strategies, it achieves "automatic handling of small faults and precise early warning of large faults," ensuring the continuity of sowing operations while promptly addressing persistent faults, significantly reducing the missed sowing rate and improving the stability and efficiency of sowing operations.

[0017] This application can effectively solve problems such as poor environmental adaptability, large trajectory deviation, inaccurate fault identification, and untimely handling during field robot sowing operations, and realize precise, intelligent, visualized, and closed-loop control of sowing operations, thus making it suitable for large-scale sowing operations in complex field environments. Attached Figure Description

[0018] Figure 1This is a schematic diagram of the structure of a real robot in an embodiment of the present invention; wherein, Figure 1 (a) is a schematic diagram of the overall framework structure. Figure 1 (b) is a schematic diagram of the structure of the seeder, seed box assembly and replanting mechanism.

[0019] Figure 2 This is a flowchart illustrating the seeding process of a digital twin robot in an embodiment of the present invention.

[0020] Figure 3 This is a comparison image before and after image processing during gesture feature extraction in an embodiment of the present invention.

[0021] Figure 4 This is a schematic diagram of an improved lightweight network structure in the gesture recognition method of this invention.

[0022] Figure 5 This is a schematic diagram of the network structure of the gesture recognition method in the embodiment of the present invention.

[0023] The components include: 1. mobile chassis; 2. frame; 3. seeder; 4. navigation vision camera; 5. navigation positioning module; 6. seed box assembly; 61. seed box; 62. seed box detection camera; 7. seed drop detection photoelectric sensor; 81. replanting tube; 82. spring valve; and 83. push rod. Detailed Implementation

[0024] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0025] Example 1: A method for monitoring and compensating for seeding in a digital twin gesture-interactive robot, comprising: Step S1: Initialize the digital twin system: Start the real robot located in the field, collect its initial state parameters, and synchronize the initial state parameters of the real robot to the virtual robot; like Figure 1As shown: The real robot includes a mobile chassis, frame, seeder, navigation vision camera, navigation and positioning module, seed box assembly, seed drop detection photoelectric sensor, replanting mechanism, industrial control computer, and display. The navigation and positioning module includes a lidar, a BeiDou satellite module, and an inertial measurement unit (IMU). The seed box assembly includes a seed box and a seed box detection camera and laser ranging sensor installed inside the seed box. The seed drop detection photoelectric sensor is installed between the seed box and the seeder. The replanting mechanism includes a replanting tube, a spring valve, and a push rod. The replanting tube is connected to the seed box and located behind the seeder. A spring valve is located at the bottom of the replanting tube's inner cavity, and a push rod is located above the spring valve to control its opening and closing. The chassis, seeder, navigation vision camera, navigation and positioning module, seed box assembly, replanting mechanism, industrial control computer, and display are all mounted on the frame.

[0026] The collected initial state parameters of the real robot include the position coordinates of the mobile chassis. x 0, y 0) Initial heading angle Initial height of seeds in the seed box h 0. Initial rotation speed of the seeding tray w 0. Initial speed of the moving chassis v 0 and field 3D maps; the specific method for obtaining them is as follows: The real robot performs a power-on self-test, the Beidou satellite module receives satellite signals, and outputs WGS-84 geodetic coordinates. B , L , H (i.e., latitude, longitude, and elevation); using the geodetic coordinate to northeast-north-sky coordinate system (ENU) algorithm, the WGS-84 geodetic coordinates are converted to northeast-north-sky local coordinates, thus obtaining the location coordinates. x 0, y 0): ; In the formula: ( B 0 ,L 0 ,H 0) represents the coordinates of the reference point in the work area; r earth Indicates the Earth's radius; The inertial measurement unit outputs triaxial angular velocity and acceleration, and the initial heading angle is obtained through complementary filtering. : ; In the formula: This represents the heading angle calculated by the inertial measurement unit using triaxial angular velocity and acceleration. This indicates the heading angle output by the BeiDou satellite module; This represents the fusion coefficient (obtained through experimental calibration, typically 0.8–0.9). A seed box detection camera captures images of the inside of the seed box and identifies the seed region and preset scale lines using a semantic segmentation network (such as U-Net). A laser rangefinder measures the distance between the seed surface and the sensor. d z-c At the same time, combined with the sensor installation height h cg Obtain the initial height of the seed. h 0: ; The encoder of the seeding tray motor outputs a pulse signal to obtain the initial speed of the seeding tray. w 0; The motor driver of the mobile chassis outputs the initial speed of the mobile chassis through current-speed closed-loop control. v 0; 3D field map: The navigation vision camera acquires RGB images of the field in real time and detects targets such as field ridges, field edges, and obstacles using an improved YOLO V11 algorithm. The improved YOLO V11 algorithm adds a SimAM attention mechanism to each detection head of the conventional YOLO V11 network (the SimAM attention mechanism enhances target features and outputs the target category, 2D bounding box, and confidence score), thereby improving the target detection accuracy and enhancing the model's feature extraction capabilities.

[0027] The lidar acquires 360° point cloud data at a frequency of 10Hz and outputs three-dimensional coordinates. x s ,y s ,z s The data is compared with the reflection intensity, and preprocessed by denoising (statistical filtering), downsampling (voxel grid), and ground point segmentation (RANSAC). First, a checkerboard calibration board is used to obtain the rotation matrix from the camera to the LiDAR. R xj With translation vector T xj The visually detected 2D target is then projected onto the laser point cloud. ; In the formula: ( x xs ,y xs () represents the pixel coordinates of the image; K nc Represents the camera intrinsic parameter matrix; Z c This represents the depth value in the camera coordinate system. The fused point cloud and target detection results are mapped to a 2.5D grid map using "Occupancy Grid Mapping"; each grid cell stores height information and occupancy probability to form a three-dimensional topographic map of the field. While obtaining the 3D topographic map, the least squares fitting is performed on the visually detected field ridge edge points to obtain the field ridge centerline, and combined with the 3D map height information to generate the 3D centerline; non-ground points in the laser point cloud are clustered (DBSCAN) to extract obstacle contours; the center coordinates and bounding boxes of the obstacles are calculated and stored in the map database.

[0028] Step S2: Trigger the virtual robot to perform simulation: Trigger the virtual robot to perform simulation through gesture interaction. The virtual robot start command is transmitted to the industrial control computer of the real robot in real time, and the field sowing begins. Step S3, Digital Twin-Driven Job Monitoring and Trajectory Prediction: The real robot's job data is transmitted to the virtual robot in real time. After receiving the job data, the virtual robot replicates all the real robot's job actions and predicts the real robot's movement trajectory, sowing actions, and seed movement trajectory in the field; specifically: The real robot uses a navigation vision camera to identify the position and height of field ridges, field edges / obstacles in real time, while a lidar simultaneously collects distance data. A BeiDou satellite module and a chassis detection module acquire the coordinates of the moving chassis in real time. x t ,y t ), driving speed v t Heading angle The seed-dropping detection photoelectric sensor acquires the seed-dropping status in real time (1 for seeds dropped, 0 for others), and simultaneously acquires the real-time seed height within the seed box. h t All data is initially processed by the industrial control computer and then uploaded to the virtual robot every 0.1 to 0.2 seconds. The virtual robot generates a theoretical motion trajectory using a two-round differential kinematics model: ; In the formula: L lj Indicates the chassis wheel track; This indicates the steering wheel deflection angle (obtained from field deviation feedback from the navigation vision camera). Meanwhile, the virtual robot uses an Extensive Kalman Filter (EKF) to filter and predict the output of the kinematic model, eliminating noise and predicting the chassis trajectory within the next 0.5 seconds. State prediction: ; Covariance prediction: ; In the formula: Represents the state vector; Indicates control input; F k Represents the state transition matrix; B k Represents the control input matrix; Q k This represents the process noise covariance (calibrated according to the field soil type, generally 10). -4 ~10 -2 ); The virtual robot ultimately outputs a set of predicted trajectory points {( x 1 ,y 1), ( x 2 ,y 2),…, ( x n ,y n This enables visualization and early prediction of the movement trajectory of the mobile chassis. Meanwhile, the virtual robot rotates according to the seeding tray speed of the real robot. w bz Length of duckbill robotic arm l yz Establish a circular motion trajectory model for the tip of the duckbill: ; In the formula: ( x xz0 ,y xz0 () indicates the coordinates of the center of rotation of the seeding tray; Indicates the initial angle of the duckbill; t bz Indicates the running time of the seeding tray; The virtual robot predicts the opening or closing time and position of the duckbill using a circular motion trajectory model, and compares it with the photoelectric sensor data of the real robot to determine whether the sowing action is normal. In addition, the virtual robot incorporates field gravity acceleration. g t air drag coefficient c air Soil friction coefficient f mc Establish a three-dimensional motion trajectory model of the seed: ; In the formula: v zz0 The initial velocity of the seed drop is represented by the vector synthesis of the circumferential tangential velocity caused by the rotation of the seed tray and the real-time driving speed of the moving chassis. Indicates the angle at which the seed falls; t zz This indicates the time it takes for a seed to move after it falls, that is, the cumulative time from when the seed leaves the duck's beak and begins to fall. Indicates the height at which the duckbill lands; For example: the circumferential tangential velocity caused by the rotation of the seed tray's duckbill. v bz = w bz × l yz The real-time driving speed of the mobile chassis is v t ,but ; ; In the formula: Represents circumferential tangential velocity v bz Real-time driving speed of the chassis v t The included angle between them (determined by the installation angle of the seeding mechanism, and is a fixed value; such as 90° or 45°, etc.); The virtual robot predicts the location where seeds will fall and compares it with the planned location on the field ridges to determine the risk of missed or off-center sowing.

[0029] Step S4, Compensation Control: The virtual robot uses the predicted trajectory as a benchmark and first obtains the actual deviation through two-dimensional error detection, specifically: Based on the field ridge line recognition results from the navigation vision camera, the virtual robot extracts the x-axis pixel deviation between the center of the real robot's mobile chassis and the planned center line of the field ridge. e px Thus, the actual x-axis deviation is obtained: ; In the formula: k ex This represents the pixel-to-distance conversion factor (obtained through in-camera height calibration). k ex =Actual distance / number of pixels); Field ridge direction obtained through navigation vision camera With the chassis planning heading angle The angle between the two points is used to obtain the heading deviation: ; The cumulative excitation error between the theoretical and actual travel distances of the mobile chassis is calculated to obtain the motion model deviation. ; In the formula: vact Indicates the actual driving speed (obtained in real time by the chassis motor encoder or Beidou positioning system). v plan Indicates the theoretical driving speed (the target driving speed of the chassis preset before operation); The virtual robot has preset x-axis deviation threshold, heading deviation threshold, and motion model deviation threshold (e.g., x-axis deviation threshold is 2cm, heading deviation threshold is 1°, and motion model deviation threshold is 0.05m). If the detected deviation is less than the corresponding threshold, the current state is maintained; if it is greater, the error correction process is triggered.

[0030] Then, the error trend is predicted by improving the PID control algorithm, specifically as follows: Establish an incremental PID control algorithm: ; ; In the formula: K p , K i , K d These represent the proportional, integral, and differential coefficients, respectively. Indicates the sampling period (usually 0.1s); Indicates control increment; in, e k To unify the error variables, the following results were obtained by weighted fusion of the actual x-axis deviation, heading deviation, and motion model deviation: ; In the formula: This represents the corresponding weighting coefficient (generally) ); To address sudden errors caused by soil slippage and undulating field ridges, the virtual robot incorporates fuzzy control to manage error variables. e k and error change rate ec k As fuzzy input, ,Will K p , K i , K d Correction amount As a fuzzy output, a 7-level fuzzy set (including negative large, negative medium, negative small, zero, positive large, positive medium, and positive small) is established. The fuzzy rule base is trained using sample data from field operations to achieve real-time self-tuning of PID parameters. ; In the formula: These represent the PID parameters after self-tuning.

[0031] Ultimately, precise seeding is achieved through multi-dimensional compensation using heading angle, position, and visual positioning, specifically including: Based on the error prediction results, the virtual robot sends supplementary control commands to the real robot, achieving real-time error correction through three dimensions: heading angle compensation, position compensation, and visual positioning compensation. Heading angle compensation: based on heading angle deviation Adjusting the steering wheel angle of a real robot : ; In the formula: This represents the heading angle compensation coefficient (obtained from chassis steering characteristic calibration, typically 0.5–1.2). Position compensation: for x-axis deviation in the width direction e x Lateral position compensation of the moving chassis is achieved by adjusting the speed difference between the left and right wheels: ; In the formula: This indicates the average speed of the mobile chassis during the current sampling period; k v This represents the speed compensation coefficient (typically 0.02 to 0.05). This indicates the compensated rotational speeds of the left and right wheels, and the difference in rotational speed causes the moving chassis to shift towards the center line of the field ridge. Visual positioning compensation: The virtual robot corrects its visual positioning coordinates by establishing a visual positioning deviation compensation matrix. ; In the formula: , The x and y axis offsets represent visual positioning (obtained from camera calibration experiments and are fixed values). Indicates the actual coordinates after compensation; Motion model parameter update: If a deviation in the chassis motion model is detected e s If the deviation exceeds the motion model deviation threshold, then the virtual robot's wheelbase... L lj Wheel diameter D cl Update: ; This ensures that the kinematic model matches the actual driving state of the real robot, reducing model errors.

[0032] Step S5, Missed Seeding Fault Detection: During the sowing process, the sensor system of the real robot monitors for missed seeding faults in real time, including abnormal seed placement, missing seeds in the seed box, or mechanical abnormalities. The improved DS evidence theory is used to accurately identify the fault category, specifically: To address the uncertainties in sowing omission failures (such as sensor noise, field environmental interference, and fuzzy failure features), the virtual robot, based on an improved DS evidence theory, achieves accurate identification of omission failures through hierarchical extraction of failure features, basic probability allocation, and fusion of conflicting evidence, thus solving the problem of distorted results in conflicting evidence under traditional DS evidence theory. Hierarchical extraction of fault features: Establishing an identification framework for the three core fault types of missed broadcasts. ;in, A 1 indicates a problem with the duckbill not opening. A 2 indicates a blockage at the seed outlet. A 3 indicates a seed box missing seed fault. A x This indicates an unknown fault (i.e., a fault other than the three types of faults mentioned above). The virtual robot extracts three layers of fault characteristics from the real robot's photoelectric sensors, the seed box monitoring component consisting of a seed box detection camera and a laser rangefinder, and the seed tray rotation speed sensor, as the original evidence for evidence-based decision-making: The first layer of features (direct features): the seed detection signal from the photoelectric sensor, 0 for the presence of seeds and 1 for the absence of seeds; the second layer of features (mechanical features): the rotational speed fluctuation rate of the seeding tray, less than 5% is considered normal fluctuation, otherwise it is considered abnormal fluctuation; the third layer of features (material features): the seed box monitoring component identifies the seed height scale; the virtual robot uses these three layers of features as three independent sources of evidence. E 1. E 2. E 3; Basic probability allocation: Calculate the basic probability of failure for each source of evidence: ; In the formula: Indicate the source of evidence E i For the fault X The feature matching degree (which is a function obtained by training field fault samples); The weight coefficient of the evidence source is indicated (obtained based on the reliability calibration of the corresponding evidence source; for example, the reliability of the third layer seed box monitoring is high, so it is 0.4; the reliability of the second layer seed tray rotation speed monitoring is low, so it is 0.25; then the reliability of the first layer seed dropping signal monitoring is 0.35). Simultaneously satisfy , ; Fusion of conflicting evidence: First, the degree of conflict among the three sources of evidence is measured to obtain the inter-evidence conflict coefficient. ; In the formula: Representing the sources of evidence respectively E 1. E 2. E 3. In the recognition framework Any possible subset of fault propositions in; , Indicates no conflict This indicates a complete conflict; Preset conflict threshold (Usually 0.8), when Then, the evidence source with the lowest credibility is removed and the evidence is re-fused. Subsequently, based on the fault identification accuracy of the evidence source... With conflict coefficient To obtain the credibility of each source of evidence: ; Fault identification accuracy rate Obtained through offline calibration and online evaluation, it represents the performance metrics of each evidence source on historical fault data (e.g.: The accuracy of photoelectric seed drop detection E1; The accuracy of E2, the detection rate of seed tray rotation speed; (Accuracy rate of visual inspection of seed boxes E3); meets the requirements. ; Finally, the basic probability allocations of each source of evidence are first weighted and adjusted, and then fused using the DS synthesis rule to obtain a comprehensive basic probability allocation: Weighted adjustment: ;in, This represents the prior probability of an unknown fault (typically 0.05). Binary synthesis: for two sources of evidence E i , E j The basic probability distribution after their fusion is as follows: ; Triadic fusion: The result of binary fusion is fused again with a third source of evidence to obtain a comprehensive basic probability allocation. m ijz (X) .

[0033] Then, the virtual robot generates corresponding decision instructions based on the fault category, specifically: First, virtual robots have a more comprehensive basic probability allocation.m ijz (X) Develop dual-threshold fault decision rules to achieve accurate identification and decision-making regarding missed broadcast faults: Maximum probability rule: ;in, X Indicates a candidate fault; Probability difference threshold rule: ;in, This represents the fault type with the second largest value in the comprehensive basic probability distribution. This represents the probability difference threshold (typically 0.2). Unknown fault threshold rules: ;in, This represents the threshold for the probability of unknown faults (typically 0.1). If a detected fault meets all three of the above rules, then the fault is determined to be... X If the maximum probability rule m (X ) The corresponding fault type did not meet the double threshold rule, i.e. and Meanwhile, if no seeds fall from the photoelectric sensor (indicating a failure to fall seeds, but without sufficient evidence to support that it was caused by a specific persistent fault), check the previous N consecutive sowing cycles (e.g., N=4). If the seed falling detection is normal, the fluctuation rate of the sowing tray speed is normal, and there are enough seeds in the seed box, then it is determined to be a single missed sowing (i.e., the duckbill did not open for a single time, and it is not a persistent fault). Trigger the corresponding fault handling process; Single missed broadcast: based on the real robot's real-time driving speed v t With replanting response time t bbz (Including image processing time and signal transmission time), obtain the plant spacing between the replanting tube and the missed sowing location: L z = v t · t bbz The virtual robot immediately transmits the replanting command to the real robot (the immediate replanting command can be transmitted to the virtual robot through gesture interaction). The real robot pushes the spring valve downwards via a push rod to achieve precise replanting. After the replanting is completed, the replanting mechanism resets, the real robot continues to sow, and the virtual robot synchronously simulates the replanting action and records the replanting information. The problem was determined to be a duckbill not opening malfunction. A 1. Clogged or blocked seed outletA 2 (Mechanical Failure): The virtual robot generates an immediate stop command through gesture interaction, the real robot stops and ensures that all mechanisms are reset, and the operator handles the fault on-site according to the location and type of the fault; after the fault is resolved, the sowing process is resumed by gesture interaction. The problem was determined to be a seed box missing seeds. A 3. The virtual robot generates a stop command. After the operator re-sows the seed box on site, the operator can start sowing through gesture interaction to resume the sowing process. At the same time, the virtual robot updates the seed box status parameters through the seed box detection camera and laser rangefinder in the seed box cavity. Determined as an unknown fault A x The virtual robot initiates behavior simulation, simulates the operational characteristics of different faults, and compares them with the abnormal data of the real robot; the operator analyzes the simulation results and manually determines the fault type based on the sensor data, and then takes corresponding actions according to the fault type.

[0034] Example 2: As another preferred embodiment of the present invention, based on the scheme of embodiment 1, gesture interaction is realized through a vision system for tracking the hand posture of a virtual robot. The vision system for tracking the hand posture is developed using the open-source virtual reality Unity3D software and its SDK package. The LeapMotion function of the SDK package and its vision system are used as the gesture data acquisition device. Sensors capture hand information in the real world and sample the gesture data frame by frame. After the sampling is completed, a processing signal is sent to the server. Gesture recognition methods include gesture image acquisition, gesture feature extraction, and image recognition; Gesture image acquisition: Five commonly used gestures are selected as virtual gesture models (gestures can be selected according to actual conditions, for example: fist is defined as fist, palm as palm, index and middle fingers as two fingers, thumb and little finger as six fingers, and four fingers together as four fingers). Each gesture includes training set, validation set, and test set images (in this embodiment, each gesture includes 1000 training images, 200 validation images, and 100 test images; the original image size is 300 pixels). 300, binary image size 128 128), construct the training model dataset; during training, the total number of images in each round: Z total = n c × t p Time taken per round: t total = f h × nc + t p × k st (In the formula: n c Indicates the number of sheets in each iteration. t p Indicates the number of times. f h Indicates the sampling frequency. k st This indicates the time interval between each acquisition. In this embodiment, it is set that two photos are acquired within 1 second, and the time interval between each photo acquisition is 0.4 seconds.

[0035] Gesture feature extraction: Using the OpenCV library, the R component of the original image is extracted to generate a grayscale image. Histogram equalization and thresholding are then applied to the grayscale image to obtain a binary image with stronger contrast and clearer gesture contours. Specifically: ; In the formula: This represents the pixel value of the current pixel in the original convex shape. This indicates the threshold selected based on actual testing; Indicates the optimization coefficient; Image processing comparison before and after Figure 3 As shown, the processing method adopts feature scaling. The types of features used include Pos, Rot, and Nor. Different types of features have different dimensions, which will lead to differences during training. Therefore, feature scaling is required. The method used is Min-Max standardization and Min-Max standardization to rescale the values ​​to [0,1].

[0036] Image Recognition: An improved lightweight VGG16 network structure is adopted as the target network for the static gesture recognition model. Addressing the issues of high memory consumption, slow loading, and low training efficiency of the VGG16 network in small sample scenarios, the network undergoes its first lightweight modification: First, the number of convolutional layers in the original VGG16 network is reduced (lowering the overall model complexity and adapting to small sample training scenarios); then, a BN (BathNormalization) layer is added after the convolutional layers (accelerating training curve convergence and improving model training efficiency); finally, based on the feature extraction requirements of gesture images, parameters such as the number and size of convolutional kernels are adjusted to make feature extraction more suitable for 128 images. The original VGG16 network's three fully connected layers were replaced with a single layer, and the number of units in the fully connected layers was reduced (reducing the number of parameters and computational cost). This resulted in multiple combinations of Conv2D+BN+MaxPool2D structures (such as Conv2D-32 / 64 / 128 / 256 paired with corresponding pooling layers) for the VGG16-S network (whose input is a 128x128-RGB image adapted for gesture recognition). Figure 4 As shown, a transformation of the VGG16 network has been completed; Based on VGG16-S, a second simplification and structural reconstruction were carried out. First, the convolutional layers were further reduced, and only two convolutional layers were retained in the end to minimize the computational cost of feature extraction. Then, the BN (Batch Normalization) layer was removed to further simplify the network structure (reduce inference time). After that, only one pooling layer was retained (simplifies the downsampling process and adapts to the fast extraction of gesture features). Then, a temporal memory network (LSTM layer) was introduced to improve the accuracy of dynamic gesture recognition by combining the temporal characteristics of gesture operations and adapt to real-time interaction scenarios. Based on the "three layers to one layer" of the fully connected layer of the VGG16-S network, the number of units of the fully connected layer was further reduced (extremely reduce the number of parameters). At the same time, a CONV1D-128 double-layer convolution + MaxPool1D (2) pooling structure was added to replace part of the 2D convolution (adapt to the one-dimensional feature extraction of gestures and improve the recognition speed). An FC-512 fully connected layer + Softmax output layer was set to directly meet the classification requirements of five gestures. Finally, the second-improved VGG16-S-5 network was obtained, such as Figure 5 As shown.

[0037] Running in real time based on the TCP / IP protocol, the computing end provides real-time gesture feedback based on the user's virtual robot's operational needs, achieving stable and intelligent human-computer interaction. This allows remote operators to use hand gestures to simulate the virtual robot's seeding operations and also enables manual handling of emergency events.

Claims

1. A method for monitoring and compensating seeding in a digital twin gesture-interactive robot, characterized in that: include: Step S1: Initialize the digital twin system: Start the real robot located in the field, collect its initial state parameters, and synchronize the initial state parameters of the real robot to the virtual robot; Step S2: Trigger the virtual robot to perform simulation: Trigger the virtual robot to perform simulation through gesture interaction. The virtual robot start command is transmitted to the industrial control computer of the real robot in real time, and the field sowing begins. Step S3, Digital Twin-Driven Operation Monitoring and Trajectory Prediction: The operation data of the real robot is transmitted to the virtual robot of the twin in real time. After receiving the operation data, the virtual robot replicates all the operation actions of the real robot and predicts the movement trajectory of the real robot in the field, the sowing action and the trajectory of the seeds. Step S4, Compensation Control: The virtual robot uses the predicted trajectory as a benchmark and obtains the actual deviation through two-dimensional error detection; Then, by improving the PID control algorithm to predict the error trend, and finally by using multi-dimensional compensation through heading angle, position and visual positioning, precise seeding can be achieved. Step S5, Missed Seeding Fault Detection: During the seeding process, the real robot's sensing system monitors for missed seeding faults in real time, including seeds not falling properly, missing seeds in the seed box, or mechanical abnormalities. The improved DS evidence theory is used to accurately identify the fault category. Then, the virtual robot generates corresponding decision instructions based on the fault category.

2. The method for monitoring and compensating seeding in a digital twin gesture-interactive robot according to claim 1, characterized in that: The real robot includes a mobile chassis, a frame, a seeder, a navigation vision camera, a navigation and positioning module, a seed box assembly, a seed drop detection photoelectric sensor, a replanting mechanism, an industrial control computer, and a display. The navigation and positioning module includes a lidar, a BeiDou satellite module, and an inertial measurement unit. The seed box assembly includes a seed box and a seed box detection camera and a laser ranging sensor installed inside the seed box. The seed drop detection photoelectric sensor is installed between the seed box and the seeder. The replanting mechanism includes a replanting tube, a spring valve, and a push rod. The replanting tube is connected to the seed box and located behind the seeder. A spring valve is located at the bottom of the replanting tube's inner cavity, and a push rod is located above the spring valve. The chassis, seeder, navigation vision camera, navigation and positioning module, seed box assembly, replanting mechanism, industrial control computer, and display are all mounted on the frame.

3. A method for monitoring and compensating seeding in a digital twin gesture-interactive robot according to claim 1 or 2, characterized in that: In step S1, the acquired initial state parameters of the real robot include the position coordinates of the mobile chassis. x 0, y 0) Initial heading angle Initial seed height in the seed box h 0. Initial rotation speed of the seeding tray w 0. Initial speed of the moving chassis v 0 and a three-dimensional field map.

4. A method for monitoring and compensating seeding in a digital twin gesture-interactive robot according to claim 2 or 3, characterized in that: Step S3 specifically involves: The real robot uses a navigation vision camera to identify the position and height of field ridges, field edges / obstacles in real time, while a lidar simultaneously collects distance data. A BeiDou satellite module and a chassis detection module acquire the coordinates of the moving chassis in real time. x t ,y t ), driving speed v t Heading angle The seed-dropping photoelectric sensor acquires real-time information on the seed-dropping status and simultaneously measures the real-time seed height within the seed box. h t All data is initially processed by the industrial control computer and then uploaded to the virtual robot every 0.1 to 0.2 seconds. The virtual robot generates a theoretical motion trajectory using a two-round differential kinematics model: ； In the formula: L lj Indicates the chassis wheel track; Indicates the steering wheel deflection angle; Meanwhile, the virtual robot uses Kalman filtering to filter and predict the output of the kinematic model, eliminating noise and predicting the chassis trajectory within the next 0.5 seconds. State prediction: ; Covariance prediction: ; In the formula: Represents the state vector; Indicates control input; F k Represents the state transition matrix; B k Represents the control input matrix; Q k Represents the process noise covariance; The virtual robot ultimately outputs a set of predicted trajectory points {( x 1 ,y 1), ( x 2 ,y 2),…, ( x n ,y n This enables visualization and early prediction of the movement trajectory of the mobile chassis. Meanwhile, the virtual robot rotates according to the seeding tray speed of the real robot. w bz Length of duckbill robotic arm l yz Establish a circular motion trajectory model for the tip of the duckbill: ； In the formula: ( x xz0 ,y xz0 () indicates the coordinates of the center of rotation of the seeding tray; Indicates the initial angle of the duckbill; t bz Indicates the running time of the seeding tray; The virtual robot predicts the opening or closing time and position of the duckbill using a circular motion trajectory model, and compares it with the photoelectric sensor data of the real robot to determine whether the sowing action is normal. In addition, the virtual robot incorporates field gravity acceleration. g t air drag coefficient c air Soil friction coefficient f mc Establish a three-dimensional motion trajectory model of the seed: ； In the formula: v zz0 Indicates the initial velocity of the seed falling; Indicates the angle at which the seed falls; t zz This indicates the time it takes for a seed to move after it falls, that is, the cumulative time from when the seed leaves the duck's beak and begins to fall. Indicates the height at which the duckbill lands; The virtual robot predicts the location where seeds will fall and compares it with the planned location on the field ridges to determine the risk of missed or off-center sowing.

5. The method for monitoring and compensating seeding in a digital twin gesture-interactive robot according to claim 4, characterized in that: In step S4, "obtaining the true deviation through dual-dimensional error detection" specifically means: Based on the field ridge line recognition results from the navigation vision camera, the virtual robot extracts the x-axis pixel deviation between the center of the real robot's mobile chassis and the planned center line of the field ridge. e px Thus, the actual x-axis deviation is obtained: ； In the formula: k ex Indicates the pixel-to-distance conversion factor; Field ridge direction obtained through navigation vision camera With the chassis planning heading angle The angle between the two points is used to obtain the heading deviation: ； The cumulative excitation error between the theoretical and actual travel distances of the mobile chassis is calculated to obtain the motion model deviation. ； In the formula: v act Indicates the actual driving speed; v plan Indicates the theoretical driving speed; The virtual robot has preset x-axis deviation threshold, heading deviation threshold, and motion model deviation threshold. If the detected deviation is less than the corresponding threshold, the current state is maintained; if it is greater than the threshold, the error correction process is triggered.

6. The method for monitoring and compensating seeding in a digital twin gesture-interactive robot according to claim 5, characterized in that: In step S4, "predicting the error trend by improving the PID control algorithm" specifically means: Establish an incremental PID control algorithm: ； ； In the formula: K p , K i , K d These represent the proportional, integral, and differential coefficients, respectively. Indicates the sampling period; Indicates control increment; in, e k To unify the error variables, the following results were obtained by weighted fusion of the actual x-axis deviation, heading deviation, and motion model deviation: ； In the formula: This represents the corresponding weighting coefficient; To address sudden errors caused by soil slippage and undulating field ridges, the virtual robot incorporates fuzzy control to manage error variables. e k and error change rate ec k As fuzzy input, ,Will K p , K i , K d Correction amount As a fuzzy output, a 7-level fuzzy set is established, and a fuzzy rule base is trained using sample data from field operations to achieve real-time self-tuning of PID parameters. ； In the formula: These represent the PID parameters after self-tuning.

7. The method for monitoring and compensating seeding in a digital twin gesture-interactive robot according to claim 6, characterized in that: In step S4, "multi-dimensional compensation through heading angle, position, and visual positioning" specifically includes: Based on the error prediction results, the virtual robot sends supplementary control commands to the real robot, achieving real-time error correction through three dimensions: heading angle compensation, position compensation, and visual positioning compensation. Heading angle compensation: based on heading angle deviation Adjusting the steering wheel angle of a real robot : ； In the formula: Indicates the heading angle compensation coefficient; Position compensation: for x-axis deviation in the width direction e x Lateral position compensation of the moving chassis is achieved by adjusting the speed difference between the left and right wheels: ； In the formula: This indicates the average speed of the mobile chassis during the current sampling period; k v Indicates the speed compensation coefficient; This indicates the compensated rotational speeds of the left and right wheels, and the difference in rotational speed causes the moving chassis to shift towards the center line of the field ridge. Visual positioning compensation: The virtual robot corrects its visual positioning coordinates by establishing a visual positioning deviation compensation matrix. ； In the formula: , Indicates the x and y axis offsets for visual positioning; Indicates the actual coordinates after compensation; Motion model parameter update: If a deviation in the chassis motion model is detected e s If the deviation exceeds the motion model deviation threshold, then the virtual robot's wheelbase... L lj Wheel diameter D cl Update: ； This ensures that the kinematic model matches the actual driving state of the real robot, reducing model errors.

8. The method for monitoring and compensating seeding in a digital twin gesture-interactive robot according to claim 7, characterized in that: In step S5, "accurately identifying fault categories using improved DS evidence theory" specifically means: To address the uncertainty of missed seeding failures, the virtual robot, based on an improved DS evidence theory, achieves accurate identification of missed seeding failures through hierarchical extraction of failure features, basic probability allocation, and fusion of conflicting evidence, thus solving the problem of distorted results in conflicting evidence in traditional DS evidence theory. Hierarchical extraction of fault features: Establishing an identification framework for the three core fault types of missed broadcasts. ;in, A 1 indicates a problem with the duckbill not opening. A 2 indicates a blockage at the seed outlet. A 3 indicates a seed box missing seed fault. A x Indicates an unknown fault; The virtual robot extracts three layers of fault characteristics from the real robot's photoelectric sensors, the seed box monitoring component consisting of a seed box detection camera and a laser rangefinder, and the seed tray rotation speed sensor, as the original evidence for evidence-based decision-making: The first layer of features: the seed detection signal from the photoelectric sensor, 0 for the presence of seeds and 1 for the absence of seeds; the second layer of features: the rotational speed fluctuation rate of the seeding tray, with a fluctuation rate less than 5% considered normal and otherwise abnormal; the third layer of features: the seed box monitoring component identifying the seed height scale; the virtual robot uses these three layers of features as three independent sources of evidence. E 1. E 2. E 3; Basic probability allocation: Calculate the basic probability of failure for each source of evidence: ； In the formula: Indicate the source of evidence E i For the fault X Feature matching degree; Indicates the weighting coefficient of the evidence source; Simultaneously satisfy , ; Fusion of conflicting evidence: First, the degree of conflict among the three sources of evidence is measured to obtain the inter-evidence conflict coefficient. ； In the formula: Representing the sources of evidence respectively E 1. E 2. E 3. In the recognition framework Any possible subset of fault propositions in; , Indicates no conflict This indicates a complete conflict; Preset conflict threshold ,when Then, the evidence source with the lowest credibility is removed and the evidence is re-fused. Subsequently, based on the fault identification accuracy of the evidence source... With conflict coefficient To obtain the credibility of each source of evidence: ； Fault identification accuracy rate Obtained through offline calibration and online evaluation, it represents the performance metrics of each evidence source on historical fault data; it meets the following requirements. ; Finally, the basic probability allocations of each source of evidence are first weighted and adjusted, and then fused using the DS synthesis rule to obtain a comprehensive basic probability allocation: Weighted adjustment: ;in, Represents the prior probability of an unknown fault; Binary synthesis: for two sources of evidence E i , E j The basic probability distribution after their fusion is as follows: ； Triadic fusion: The result of binary fusion is fused again with a third source of evidence to obtain a comprehensive basic probability allocation. m ijz (X) .

Images