Intelligent agricultural machine remote control system

Abstract

The invention provides an intelligent agricultural machine remote control system, relates to the technical field of remote control, and adopts a cloud-side-end three-level architecture to realize remote accurate control of agricultural machines through fusion of multi-source sensing, AI intelligent decision and 5G / satellite communication technologies. The system is composed of a remote control center, an AI intelligent decision module, an agricultural machine vehicle-mounted control unit, a multi-source sensing module, an execution module and a communication transmission module. The AI intelligent decision module performs fusion analysis on the farmland environment data and the agricultural machine state data based on a deep learning and reinforcement learning algorithm to generate an optimal control strategy; the communication transmission module adopts a 5G and satellite communication dual-mode redundancy design to ensure stable data transmission; the multi-source sensing module realizes omnibearing sensing of a farmland environment and an agricultural machine state; the remote control center constructs a three-dimensional digital twinborn model, remote visual monitoring and instruction issuing are achieved, the labor cost is reduced, and the system is suitable for diversified agricultural scenes such as large-scale farmlands, hilly and mountainous areas and the like.

Description

Technical Field

[0001] This invention belongs to the field of remote control technology, and more specifically, relates to an intelligent agricultural machinery remote control system. Background Technology

[0002] With the acceleration of agricultural modernization, large-scale, precision, and unmanned operations have become the core trends in modern agricultural development. However, traditional agricultural machinery operation modes and existing related technologies still have many problems that urgently need to be solved:

[0003] Traditional agricultural machinery operations heavily rely on manual on-site operation, facing a labor shortage during peak farming seasons. Furthermore, the daily working time for a single person is limited (only 8-10 hours), resulting in high labor intensity. Path planning under manual control depends on experience, easily leading to overlapping or omissions, resulting in low efficiency; the area covered per unit time is only 60%-70% of that in intelligent operation modes. Simultaneously, manual operation struggles to achieve centimeter-level precision, leading to uneven sowing depth and inaccurate fertilization, directly impacting crop yield and quality. In addition, traditional agricultural machinery has poor adaptability to complex terrains such as hilly areas and irregular plots, as well as adverse weather conditions like rain, fog, and sandstorms, making it prone to accidents such as getting stuck and collisions. Equipment malfunctions require on-site troubleshooting by professionals, resulting in long downtimes, high maintenance costs, and irrecoverable losses during critical farming seasons.

[0004] While existing technologies for remote control of agricultural machinery have made some progress, significant limitations remain: some 4G-based remote monitoring systems can only monitor status and cannot meet the low-latency requirements of real-time control (transmission latency > 200ms), and signal coverage is insufficient in remote farmland; driver assistance systems only reduce the driver's workload, not completely freeing up human labor, and have limited perception and decision-making capabilities in unstructured farmland environments; simple remote control systems are limited by control distance (<1km) and terrain obstruction, resulting in insufficient safety and practicality; single-machine intelligent systems lack a global perspective, cannot achieve multi-machine collaborative operation, have weak model generalization capabilities, rely on manual intervention for updates, and are difficult to adapt to diverse farmland scenarios. These technological bottlenecks mean that existing solutions cannot meet the demands of large-scale agricultural production for efficient, precise, and reliable remote control, necessitating an integrated solution that combines AI intelligent decision-making, multi-source perception, and highly reliable communication. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides an intelligent agricultural machinery remote control system, which solves the technical problems of traditional agricultural machinery relying on manual on-site operation, low operating efficiency and accuracy, poor adaptability to complex farmland environments, as well as the existing remote control technologies having high transmission delays, unstable signals, weak AI decision-making capabilities, and a lack of multi-machine collaboration and model adaptive update capabilities.

[0006] A remote control system for intelligent agricultural machinery includes a remote control center, an AI intelligent decision-making module, an on-board control unit for agricultural machinery, a multi-source sensing module, an execution module, and a communication transmission module.

[0007] The remote control center is used to receive user control commands and send them to the AI ​​intelligent decision-making module, while also receiving agricultural machinery operation status data and environmental data fed back by the AI ​​intelligent decision-making module;

[0008] The multi-source sensing module is used to collect farmland environmental data, agricultural machinery operating parameter data, and work object data, and transmits the collected multi-dimensional data to the AI ​​intelligent decision-making module.

[0009] The AI ​​intelligent decision-making module, based on a deep learning model, fuses and analyzes multi-dimensional data collected by the multi-source perception module. Combined with user control commands issued by the remote control center, it generates the optimal control strategy and sends it to the agricultural machinery's onboard control unit. The deep learning model, trained on massive amounts of farmland operation sample data, possesses the ability to adaptively adjust to the environment and optimize operational parameters. The multi-dimensional data fusion employs a weighted fusion algorithm, and the fusion formula for the fused data feature vector X is as follows:

[0010] X-fusion = ω1Xenvironment + ω2XAgricultural Machinery + ω3XOperation, where ω1, ω2, and ω3 are the weight coefficients of farmland environment data, agricultural machinery operation parameter data, and operation object data, respectively, and satisfy ω1+ω2+ω3=1. The weight coefficients are obtained by optimization through gradient descent algorithm.

[0011] The agricultural machinery vehicle control unit is used to analyze the optimal control strategy issued by the AI ​​intelligent decision-making module, generate drive signals, and drive the execution module to perform corresponding operation actions;

[0012] The communication transmission module adopts a dual-mode redundancy design of 5G and satellite communication to achieve low-latency and high-reliability data transmission between the remote control center, the AI ​​intelligent decision-making module and the agricultural machinery vehicle control unit;

[0013] The execution module includes the agricultural machinery's walking drive mechanism, work execution mechanism, and safety protection mechanism, which are used to respond to the drive signals of the on-board control unit to complete the specified work tasks and safety protection actions.

[0014] Preferably, the AI ​​intelligent decision-making module includes a data preprocessing submodule, an environmental situation assessment submodule, an operation strategy optimization submodule, and a fault early warning submodule;

[0015] The data preprocessing submodule is used to perform noise reduction, standardization and data alignment on the raw data collected by the multi-source sensing module, remove abnormal data and generate standardized data samples.

[0016] The environmental situation assessment submodule performs image semantic segmentation on the standardized farmland environmental data based on convolutional neural networks (CNN), identifies obstacles, crop types, crop growth status and terrain features in the farmland, and generates a farmland environmental situation assessment report.

[0017] The semantic segmentation uses an improved U-Net network, and the loss function is a weighted sum of the cross-entropy loss function and the Dice loss function, with the specific expression as follows:

[0018] Loss = α × CrossEntropyLoss + (1-α) × DiceLoss, where α is the weighting coefficient, and its value ranges from 0.3 to 0.7;

[0019] The operation strategy optimization submodule is based on reinforcement learning algorithm, combined with environmental situation assessment report and user control command, to dynamically optimize the operation path, operation speed and operation parameters of agricultural machinery, and generate the optimal operation strategy.

[0020] The reinforcement learning algorithm used is DQN, and the action value function update formula is:

[0021] Q(s,a)=Q(s,a)+γ×[r+maxQ(s',a')-Q(s,a)], where s is the current state, a is the current action, r is the immediate reward, s' is the next state, and γ is the discount factor, with a value range of 0.8~0.95;

[0022] The fault early warning submodule performs time-series analysis on the agricultural machinery's own operating parameter data based on recurrent neural networks (RNN), predicts potential faults of the agricultural machinery, generates fault early warning information, and pushes it to the remote control center and the agricultural machinery's on-board control unit.

[0023] The fault prediction uses an LSTM network, which calculates anomaly scores in the time series of operating parameters to achieve fault early warning. The anomaly score calculation formula is as follows:

[0024] Score = ||x^-x||2 / ||x||2, where x is the actual running parameter vector, x^ is the model predicted parameter vector, and ||·||2 is the L2 norm.

[0025] Preferably, the multi-source sensing module includes a visual sensor, a lidar, a millimeter-wave radar, a GPS / BeiDou positioning module, an inertial measurement unit, and agricultural machinery operating parameter sensors;

[0026] The visual sensor is used to acquire images of the farmland environment and the objects being worked on;

[0027] The lidar and millimeter-wave radar are used to detect obstacles and terrain undulations in farmland.

[0028] The GPS / BeiDou positioning module and inertial measurement unit are used to acquire real-time position and attitude data of the agricultural machinery;

[0029] The agricultural machinery operating parameter sensor is used to collect data on engine speed, oil pressure, hydraulic system pressure, and battery charge.

[0030] Preferably, the AI ​​intelligent decision-making module also has a model adaptive update function, which can receive new farmland operation data in real time through edge computing nodes, incrementally train the deep learning model, optimize the model's environmental recognition accuracy and operation strategy generation efficiency, and the model update process does not affect the normal operation of agricultural machinery; the incremental training adopts the stochastic gradient descent (SGD) optimizer, and the parameter update formula is:

[0031] θt+1=θt-η×∇L(θt), where θt is the model parameter at time t, θt+1 is the model parameter at time t+1, η is the learning rate, which ranges from 1e-5 to 1e-3, and ∇L(θt) is the gradient of the loss function at time t.

[0032] Preferably, the remote control center includes an AI visualization monitoring submodule. Based on the environmental data and operation status data transmitted by the AI ​​intelligent decision module, the AI ​​visualization monitoring submodule constructs a three-dimensional digital twin model of farmland operations, renders the operation trajectory of agricultural machinery, crop distribution and environmental changes in real time, and supports users to remotely control agricultural machinery through the three-dimensional model.

[0033] Preferably, a local backup decision unit is set between the AI ​​intelligent decision module and the agricultural machinery vehicle control unit. When the communication transmission module fails and remote data transmission is interrupted, the local backup decision unit calls the emergency operation strategy pre-stored in the AI ​​intelligent decision module to drive the agricultural machinery to complete emergency avoidance or operation completion actions. After the communication is restored, the data is synchronized to the remote control center.

[0034] Preferably, the operation strategy optimization submodule also has a multi-machine collaborative operation optimization function. When the system simultaneously controls multiple machines, it achieves data sharing and collaborative decision-making among the machines through a federated learning algorithm, optimizing the division of operation areas and path planning for the machines, avoiding overlapping and omissions, and improving overall operation efficiency. The federated learning uses a federated averaging algorithm, and the global model parameter update formula is:

[0035] Wglobal=1 / N×ΣNi=1Wi, where Wglobal is the global model parameter, N is the number of agricultural machines, and Wi is the local model parameter of the i-th agricultural machine.

[0036] Preferably, the AI ​​intelligent decision-making module further includes an operation effect evaluation submodule. The operation effect evaluation submodule is based on the farmland data and crop data collected by the multi-source sensing module after the operation, combined with the preset operation standards, and uses AI algorithms to quantitatively evaluate the operation quality, generate an operation effect evaluation report and feed it back to the remote control center, so as to provide data support for the user to adjust subsequent operation instructions.

[0037] Preferably, the communication transmission module is further provided with a data encryption submodule, which uses an AI-driven dynamic encryption algorithm to encrypt the transmitted data, identify and intercept abnormal access behavior in real time during data transmission, and ensure the security of remote control data transmission.

[0038] Preferably, the AI ​​visualization monitoring submodule also has an AI-assisted decision-making prompt function. When the system identifies abnormalities in the farmland environment or unreasonable agricultural machinery operation parameters, it automatically generates optimized operation suggestions and highlights them in the three-dimensional digital twin model to assist users in making more reasonable remote control decisions.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] The AI ​​intelligent decision-making module, based on semantic segmentation technology using an improved U-Net network, achieves accurate identification of crop types, obstacles, and terrain features. Combined with the DQN reinforcement learning algorithm, it dynamically optimizes the operation path and parameters, effectively avoiding overlapping and omissions in operations. The efficiency per unit time is increased by more than 40% compared to traditional manual operation. The multi-source perception module integrates data from vision, radar, GPS / BeiDou positioning, etc., and, with centimeter-level positioning technology, controls the sowing depth deviation within ±0.2cm and achieves fertilization uniformity of over 95%, significantly improving operation quality and crop yield. At the same time, variable operation technology reduces agricultural input consumption by 20%-30%, realizing green agricultural production.

[0041] The remote control center's 3D digital twin and visual operation functions support a "one person, multiple machines" management mode (a single user can monitor 10-20 agricultural machines simultaneously), completely freeing up the need for manual on-site operation, reducing labor costs for large-scale farms by more than 60%, and effectively solving the "labor shortage" problem during the busy farming season; the fault early warning submodule, based on LSTM network time series analysis technology, predicts potential faults such as abnormal engine speed and hydraulic system leakage in advance, shortening maintenance response time from "days" to "minutes", reducing equipment downtime by 30%, and reducing operation and maintenance costs by 30%; the multi-agricultural machine collaborative operation function optimizes regional division through federated learning algorithms, further improving the overall efficiency of large-scale operations.

[0042] The communication transmission module adopts a dual-mode redundancy design of 5G and satellite communication, achieving seamless switching (switching time <200ms) in complex environments such as remote mountainous areas and areas with signal obstruction, with data transmission latency ≤50ms, ensuring the stability and real-time performance of remote control; the local backup decision unit automatically takes over control when communication is interrupted, performing emergency avoidance or work completion actions to ensure the safety of equipment and personnel; the multi-source sensing module's lidar, millimeter-wave radar, and visual sensors work together, enabling the system to adapt to diverse terrains such as plains, hills, and mountains, as well as harsh weather conditions such as rain, fog, and sandstorms, possessing all-scenario operation capabilities.

[0043] The AI ​​intelligent decision-making module's model adaptive update function optimizes model parameters in real time by incrementally training (SGD optimizer) on edge computing nodes and fusing new job data. This improves environmental recognition accuracy and decision-making efficiency without manual intervention, making the system "smarter with use." The AI-assisted decision-making prompt function in the remote control center automatically generates optimization suggestions when the environment is abnormal or the parameters are unreasonable, reducing the operational threshold. The data encryption submodule, based on AI dynamic encryption algorithms and anomaly detection technology, ensures the security of control data transmission and prevents unauthorized access and data tampering.

[0044] The execution module's safety protection mechanisms (audible and visual alarms, infrared anti-collision, and electromagnetic braking) and fault classification and handling mechanisms enable rapid response to collision risks or equipment failures, preventing safety accidents. The operation effect evaluation submodule uses AI algorithms to quantify operation quality, generate evaluation reports, and feed them back to the remote control center, providing data support for subsequent operation instruction adjustments. This forms a closed-loop control of "instruction-execution-evaluation-optimization," ensuring that operation quality is controllable throughout the entire process. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the system flow of the present invention. Detailed Implementation

[0046] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and should not be construed as limiting the scope of the invention.

[0047] Please see Figure 1 This invention provides an intelligent agricultural machinery remote control system, with the following overall system architecture:

[0048] The intelligent agricultural machinery remote control system disclosed in this embodiment aims to achieve remote and precise control and AI-powered intelligent decision-making of agricultural machinery in farmland operation scenarios, solving problems such as reliance on manual on-site operation, low work efficiency, and poor environmental adaptability in traditional agricultural machinery control. The system as a whole adopts a three-tier architecture of "cloud-edge-device".

[0049] The remote control center is deployed in the cloud and is responsible for user interaction, global monitoring, and command issuance.

[0050] The AI ​​intelligent decision-making module is deployed on edge computing nodes to balance data processing efficiency and real-time decision-making requirements.

[0051] The agricultural machinery vehicle-mounted control unit, multi-source sensing module, execution module and communication transmission module constitute the terminal execution layer, which is installed on the agricultural machinery body to realize data acquisition and action execution.

[0052] Data interaction is achieved at each level through a dual-mode redundant network of 5G and satellite communication, ensuring the stability of data transmission in complex environments such as remote farmland and areas with signal obstruction.

[0053] Multi-source sensing module:

[0054] In this embodiment, the multi-source sensing module adopts a multi-dimensional sensing scheme of "vision + radar + positioning + inertial + operating condition". The selection and parameter settings of each sensor are as follows:

[0055] Visual sensor: An industrial-grade high-definition camera with a resolution of 2592×1944, a frame rate of 14fps, and a lens focal length of 8mm is selected. It is installed on the top of the agricultural machinery cab and the side of the operating mechanism to collect global images of the farmland environment and close-up detail images of crops. The image format is RAW and is transmitted to the edge computing node via USB 3.0 interface.

[0056] LiDAR: Solid-state LiDAR is selected, with a detection range of 0.1~100m, angular resolution of 0.1°×0.1°, and point cloud density of 16 lines. It is installed at the front of agricultural machinery to detect obstacles (such as rocks and tree trunks) and terrain undulations (such as ditches and slopes) in farmland. The output point cloud data format is PCD, and the data transmission rate is 100Mbps.

[0057] Millimeter-wave radar: 77GHz millimeter-wave radar is selected, with a detection range of 0.5~200m and a speed measurement range of -150~150km / h. It is installed at the front and rear of agricultural machinery to assist lidar in achieving accurate obstacle positioning and improve the reliability of perception in adverse weather conditions (rain, fog, dust).

[0058] GPS / BeiDou positioning module: A dual-frequency multi-system positioning module is selected, which supports GPS, BeiDou, GLONASS and Galileo multi-system fusion positioning, with a positioning accuracy of ±1cm (RTK mode) and an update frequency of 10Hz. It outputs location data such as latitude, longitude and altitude of agricultural machinery through the UART interface.

[0059] Inertial Measurement Unit (IMU): A six-axis IMU with an acceleration range of ±16g, an angular velocity range of ±2000° / s, and a sampling frequency of 100Hz is selected to collect agricultural machinery attitude data (pitch angle, roll angle, and heading angle), and to achieve high-precision collaborative positioning of agricultural machinery position and attitude by fusing with GPS / BeiDou data.

[0060] Agricultural machinery operating parameter sensors: A customized chemical condition sensor group is adopted, including engine speed sensor (range 0~8000rpm, accuracy ±1rpm), oil pressure sensor (range 0~10bar, accuracy ±0.1bar), hydraulic system pressure sensor (range 0~300bar, accuracy ±1bar), and battery power sensor (range 0~100%, accuracy ±1%). The sensor output signal is a 4~20mA analog signal, which is converted into a digital signal by an AD converter and then transmitted to the vehicle control unit.

[0061] AI Intelligent Decision-Making Module:

[0062] The AI ​​intelligent decision-making module is deployed on an edge computing node (hardware platform: NVIDIA Jetson AGXXavier, CPU: 8-core ARM Cortex-A78, GPU: Volta architecture 512-core CUDA, memory: 32GB LPDDR4X), and adopts a modular design. The specific implementation of each sub-module is as follows:

[0063] Data preprocessing submodule:

[0064] This submodule preprocesses the raw data collected by the multi-source sensing module. The specific process is as follows:

[0065] Noise reduction: Gaussian filtering (kernel size = 3×3, σ = 1.2) is used to remove image noise from visual images; pass-through filtering (Z-axis range 0.5~5m) is used to remove invalid point clouds on the ground and at long distances from lidar point cloud data; Kalman filtering (state equation: X(k) = AX(k-1) + BU(k) + W(k), observation equation: Z(k) = HX(k) + V(k)) is used to suppress high-frequency noise from IMU and operational data.

[0066] Standardization processing: Data from different dimensions are normalized. Image data is normalized to the [0,1] interval (formula: I_norm=(I-I_min) / (I_max-I_min), where I is the original pixel value, and I_min and I_max are the minimum and maximum pixel values ​​of the image, respectively); working condition data are standardized using Z-score (formula: X_norm=(X-μ) / σ, where μ is the data mean and σ is the data standard deviation).

[0067] Data alignment: Based on timestamps, data from each sensor is synchronized, and linear interpolation is used to fill in missing data to ensure consistency of multi-dimensional data in the time dimension. Finally, standardized data samples are generated (sample format: JSON, including timestamps, environmental data, agricultural machinery status data, and operation object data).

[0068] Environmental situation assessment submodule:

[0069] This submodule implements semantic segmentation of farmland environment images based on an improved U-Net network, as detailed below:

[0070] Network structure improvements: An attention mechanism (CBAM attention module) is added to the encoding end of the traditional U-Net network to enhance the feature extraction capability of key targets such as crops and obstacles; the decoding end adopts a combination of transposed convolution and upsampling to improve segmentation accuracy;

[0071] Loss function settings: A weighted sum of the cross-entropy loss function and the Dice loss function is used, with a value of α=0.5 (balancing target pixel ratio balance and boundary segmentation accuracy). The cross-entropy loss function is calculated as follows:

[0072] CrossEntropyLoss=-Σ(y_ilog(p_i)), where y_i is the true label and p_i is the predicted probability. The Dice loss function is calculated as: DiceLoss=1-2×|Y∩P| / (|Y|+|P|), where Y is the set of true labels and P is the set of predicted labels.

[0073] Model training and inference:

[0074] The training dataset uses a self-built farmland operation dataset (containing 5 common crops such as wheat, corn, and rice, and 8 common obstacles such as stones, tree trunks, and ditches, totaling 100,000 images, labeled in VOC format). Training parameters are set as follows:

[0075] Batch size = 8, number of iterations = 10000, initial learning rate = 1e-4, and cosine annealing learning rate scheduling strategy is adopted. During inference, the preprocessed farmland environment image is input, the semantic segmentation result is output, the segmentation boundary is optimized through image morphological processing (dilation, erosion), and finally the crop type, growth status (seedling stage, growth stage, maturity stage), obstacle type and location, terrain features (flat land, slope, depression) are identified, and an environmental situation assessment report is generated.

[0076] Job strategy optimization submodule:

[0077] This submodule optimizes job strategies based on the DQN algorithm, and the specific implementation is as follows:

[0078] State space definition: State s includes environmental state (crop type, growth status, obstacle location, terrain features), agricultural machinery state (position, attitude, speed, remaining battery power), and work task state (already worked area, remaining work area, work quality).

[0079] Action space definition: Action a includes walking actions (forward, backward, left turn, right turn, acceleration, deceleration) and operation actions (adjustment of seeding rate, adjustment of fertilizer application rate, adjustment of tillage depth, and adjustment of harvesting speed).

[0080] Reward function design: The immediate reward r is calculated based on a combination of work efficiency, work quality, and safety factor, using the following formula:

[0081] r = ω_a × r_a + ω_q × r_q + ω_s × r_s, where ω_a, ω_q, and ω_s are the efficiency weight, quality weight, and safety weight, respectively, and their sum is 1. The work efficiency reward r_a is positively correlated with the work area per unit time, the work quality reward r_q is negatively correlated with work deviations (such as sowing depth deviation and fertilization uniformity deviation), and the safety reward r_s is positive when there is no collision risk and negative when there is a collision risk.

[0082] DQN algorithm execution: It adopts a dual-network structure of target network and evaluation network. The target network parameters are updated every 1000 steps. The experience replay pool capacity is 100,000. The discount factor γ=0.9. The action value function update formula is as described in claim 13. The algorithm dynamically optimizes the agricultural machinery operation path (using A* algorithm combined with DQN to optimize path nodes), operation speed and operation parameters to generate the optimal control strategy.

[0083] Fault early warning submodule:

[0084] This submodule uses an LSTM network to predict agricultural machinery faults, and the specific implementation is as follows:

[0085] Network structure design: The LSTM network consists of 1 input layer (6 neurons, corresponding to 6 types of agricultural machinery operating parameters), 2 hidden layers (64 neurons per layer, with ReLU activation function), and 1 output layer (6 neurons, outputting the predicted operating parameter values).

[0086] Anomaly score calculation: The anomaly score between the actual running parameter vector x and the model predicted parameter vector x^ is calculated using the formula Score=||x^-x||2 / ||x||2, where ||·||2 is the L2 norm (formula: ||x||2=√(x1²+x2²+...+xn²)). An anomaly threshold Score_th=0.15 is set. When Score>Score_th, a potential fault is identified.

[0087] Fault type identification: Abnormal data is classified by a trained fault classification model (using SVM algorithm) to identify fault types (such as abnormal engine speed, low oil pressure, hydraulic system leakage, etc.), generate fault warning information (including fault type, fault level, and suggested handling measures), and push it to the remote control center and vehicle control unit.

[0088] Model adaptive update submodule:

[0089] This submodule implements incremental training of deep learning models, specifically as follows: edge computing nodes receive new farmland operation data in real time (incremental training is triggered every 1000 new samples), and use stochastic gradient descent (SGD) optimizer to update model parameters. The parameter update formula is as described in claim 17, with a learning rate of η=5e-4. By freezing the bottom feature extraction layer of the model and updating only the top classification layer, the computational power consumption for training is reduced, ensuring that the normal operation of agricultural machinery is not affected during the model update process. The updated model parameters are synchronized to the edge computing nodes of all networked agricultural machinery through encrypted transmission to achieve global model optimization.

[0090] Remote control center implementation:

[0091] The remote control center is deployed on a cloud server (hardware configuration: CPU: Intel Xeon Gold 6330, memory: 128GB DDR4, hard drive: 2TB SSD, GPU: NVIDIA A100), adopting a B / S architecture design, mainly including a user interface, an AI visualization monitoring submodule, and a data storage submodule.

[0092] User interface: Developed based on the Vue.js framework, it supports users to issue operation instructions (such as sowing, fertilizing, harvesting, and path planning), view the operation status of agricultural machinery, and receive fault warning information. The interface includes a map monitoring area, an agricultural machinery status display area, an instruction issuance area, and a report statistics area.

[0093] The AI ​​visualization monitoring submodule: Based on the Unity3D engine, it constructs a 3D digital twin model of farmland operations. It receives environmental and operational status data transmitted by the AI ​​intelligent decision-making module in real time via the WebSocket protocol, enabling real-time rendering of agricultural machinery operation trajectories, crop distribution, and environmental changes. It supports users to perform visualized remote control through the 3D model (such as drag-and-drop path planning and clicking to adjust operational parameters). It also has an AI-assisted decision-making prompt function. When the system identifies abnormal farmland environments (such as sudden obstacles) or unreasonable operational parameters (such as sowing depth exceeding the suitable range for crops), it automatically generates optimized control suggestions and highlights them in the 3D model (abnormal areas are marked with red flashing).

[0094] Data storage submodule: It adopts a hybrid storage architecture of MySQL + MongoDB. MySQL is used to store structured data (such as basic information of agricultural machinery, user instructions, and fault records), while MongoDB is used to store unstructured data (such as farmland environment images, point cloud data, and operation videos). The data retention period is 3 years, and it supports historical data query and operation report generation.

[0095] Implementation of the communication transmission module:

[0096] The communication transmission module adopts a dual-mode redundancy design for 5G and satellite communication, as detailed below:

[0097] 5G communication module: An industrial-grade 5G module (model: Huawei ME909s-821) is selected, which supports SA / NSA dual mode, with a peak downlink rate of 1Gbps and a peak uplink rate of 100Mbps. It is installed in the cab of agricultural machinery and connects to the edge computing node through the PCIe interface to realize short-range, high-bandwidth data transmission (such as high-definition images and video streams).

[0098] Satellite communication module: A low-orbit satellite communication module (model: StarlinkMaritime) is selected, which supports global coverage, downlink speed of 100~200Mbps, uplink speed of 20~40Mbps, and serves as a backup for 5G communication. When there is no 5G signal in the agricultural machinery operation area, it automatically switches to satellite communication mode to ensure uninterrupted communication between the remote control center and the agricultural machinery.

[0099] Data encryption submodule: Employs an AI-driven dynamic encryption algorithm, which analyzes data transmission characteristics (such as transmission frequency, data type, and transmission address) in real time based on the LSTM network, dynamically adjusts the encryption key (key length 128~256 bits, updated every 10 minutes), and uses the AES-256 encryption algorithm to encrypt the transmitted data. At the same time, it uses an AI anomaly detection model (based on the Isolation Forest algorithm) to identify abnormal access behavior (such as unauthorized IP access and data tampering) in real time during data transmission and immediately triggers an interception mechanism to ensure data transmission security.

[0100] Agricultural machinery vehicle-mounted control unit and execution module:

[0101] The agricultural machinery vehicle-mounted control unit uses an embedded controller (model: STM32H743, CPU: ARM Cortex-M7, main frequency: 480MHz, memory: 1MB SRAM) to analyze the optimal control strategy issued by the AI ​​intelligent decision-making module and generate drive signals.

[0102] Walking drive mechanism: The walking of the agricultural machinery is controlled by an electro-hydraulic proportional valve. The vehicle control unit outputs a PWM signal (frequency 100Hz, duty cycle 0~100%) to drive the electro-hydraulic proportional valve to realize the adjustment of the speed and direction of the agricultural machinery; it is equipped with an electromagnetic braking device, which will immediately trigger the brake when receiving fault warning information or emergency stop command.

[0103] Work execution mechanism: The corresponding execution components (seeder, fertilizer applicator, harvester) are configured according to different work types. The vehicle control unit communicates with the controller of the work execution mechanism through the CAN bus and sends work parameter commands (such as seeding amount, fertilizer application amount, tillage depth). The execution mechanism achieves precise control through stepper motors or servo motors.

[0104] Safety protection mechanisms include an audible and visual alarm device, an infrared anti-collision sensor, and an emergency stop button. When a collision risk or malfunction is detected, the audible and visual alarm device (buzzer + LED warning light) is activated immediately, and at the same time, the vehicle control unit drives the actuator to stop working, ensuring the safety of personnel and equipment.

[0105] Local backup decision unit: Implemented using an FPGA chip (model: XilinxArtix-7), it pre-stores emergency operation strategies for common operation scenarios (such as obstacle avoidance paths and operation completion procedures). When the communication transmission module fails and remote data transmission is interrupted, the local backup decision unit takes over the control of the vehicle control unit through the CAN bus, drives the agricultural machinery to complete emergency actions, and synchronizes the emergency operation data to the remote control center after communication is restored.

[0106] System workflow:

[0107] The complete workflow of the intelligent agricultural machinery remote control system of this invention is as follows:

[0108] System initialization: Users log in to the system through the interactive interface of the remote control center, select the job type (such as wheat sowing), set the job parameters (such as sowing depth of 2-3cm, sowing density of 300 seeds / m²), and issue job instructions; the remote control center transmits the instructions to the AI ​​intelligent decision-making module through 5G / satellite communication;

[0109] Data Acquisition and Preprocessing: The multi-source sensing module is activated to collect farmland environmental data (images, point clouds, terrain), agricultural machinery operating parameter data (speed, pressure, power), and work object data (crop type, growth status) in real time. The raw data is transmitted to the data preprocessing submodule of the AI ​​intelligent decision-making module for noise reduction, standardization, and data alignment.

[0110] AI-powered intelligent decision-making: The environmental situation assessment submodule performs semantic segmentation on the preprocessed environmental data and generates an environmental situation assessment report; the operation strategy optimization submodule combines the assessment report with user operation instructions and generates the optimal control strategy (including travel path, operation speed, and operation parameters) through the DQN algorithm; the fault early warning submodule performs time-series analysis on agricultural machinery operation parameters and monitors fault risks in real time.

[0111] Command transmission and execution: The optimal control strategy is transmitted to the agricultural machinery vehicle control unit through the communication transmission module. The vehicle control unit analyzes the strategy and generates drive signals to drive the execution module to complete operations such as sowing and fertilizing. At the same time, the vehicle control unit feeds back the agricultural machinery operation status data (completed operation area, operation quality, equipment status) to the AI ​​intelligent decision-making module.

[0112] Visualized monitoring and feedback: The AI ​​intelligent decision-making module transmits environmental data and operation status data to the AI ​​visualized monitoring sub-module in the remote control center, constructs a 3D digital twin model, and renders the operation scene in real time; when environmental anomalies or unreasonable operation parameters are detected, AI-assisted decision-making suggestions are generated and pushed to the user; the user can adjust the operation instructions according to the monitoring situation to achieve closed-loop control;

[0113] Model Adaptive Update and Fault Handling: Edge computing nodes receive new operational data in real time, triggering the model adaptive update function of the AI ​​intelligent decision-making module to optimize the model through incremental training; if the fault warning submodule detects a potential fault, it immediately generates a fault warning message and pushes it to the remote control center and the vehicle control unit. The vehicle control unit performs corresponding processing (alarm, shutdown, emergency avoidance) according to the fault level.

[0114] Work completion: When the system detects that the work area has reached the user-set value, it automatically generates a work completion report (including work area, work quality, and equipment operating status) and pushes it to the remote control center; after the user confirms that the work is completed, a stop command is issued, the agricultural machinery performs finishing actions (such as cleaning the working mechanism and returning to the docking point), and the system shuts down.

[0115] This embodiment uses wheat sowing as an example to illustrate the system's application effect:

[0116] Work scenario: A large-scale wheat planting base, covering an area of ​​1,000 mu (approximately 67 hectares), with mainly flat terrain, some ditches and rock obstacles, and soil moisture of 20-30%.

[0117] System configuration: The agricultural machinery model is Dongfanghong LX904 tractor, equipped with the remote control system of this invention, and equipped with a seeder operation execution mechanism;

[0118] Operation Process: The user issues wheat sowing instructions through the remote control center, setting the sowing depth to 2.5cm and the sowing density to 320 seeds / m². After the system starts, the multi-source sensing module collects farmland environmental images and point cloud data, and the AI ​​intelligent decision-making module identifies wheat planting areas, ditches, and rocks through an improved U-Net network, generating an environmental situation assessment report. The operation strategy optimization submodule plans the optimal sowing path (avoiding ditches and rocks, reducing path overlap) using the DQN algorithm, dynamically adjusting the seeder's operating parameters. The communication transmission module adopts 5G communication mode to achieve low-latency transmission of instructions and data (transmission latency ≤50ms). During operation, the system monitors the agricultural machinery engine speed (stable at 2000rpm) and oil pressure (3.5~4.5bar) in real time, and no fault warnings are issued.

[0119] Operational results: The sowing of 1,000 mu of wheat took 25 hours, which is 40% more efficient than traditional manual operation of agricultural machinery; the sowing depth deviation was ≤ ±0.2cm, the sowing uniformity was ≥95%, and the operation quality was significantly improved; no manual on-site operation was required throughout the process, only one user was needed to monitor the operation from the remote control center, which reduced labor costs.

[0120] The above description is merely a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any equivalent structural transformations made based on the content of the present invention specification and drawings under the premise of the present invention, or direct / indirect applications in other related technical fields, shall fall within the scope of protection of the present invention.

[0121] The embodiments of the present invention are given for illustrative and descriptive purposes only, and are not intended to be exhaustive or to limit the invention to the forms disclosed. Many modifications and variations will be apparent to those skilled in the art. The embodiments were chosen and described in order to better illustrate the principles and practical application of the invention, and to enable those skilled in the art to understand the invention and to design various embodiments with various modifications suitable for a particular purpose.

Claims

1. An intelligent agricultural machinery remote control system, characterized in that, The remote control center, the AI intelligent decision module, the agricultural machinery vehicle-mounted control unit, the multi-source perception module, the execution module and the communication transmission module are included. The remote control center is used for receiving user control instructions and sending them to the AI intelligent decision module, and receiving agricultural operation state data and environmental data fed back by the AI intelligent decision module. The multi-source perception module is used for collecting farmland environmental data, agricultural machinery self-running parameter data and operation object data, and transmitting the collected multi-dimensional data to the AI intelligent decision module. The AI intelligent decision module fuses and analyzes the multi-dimensional data collected by the multi-source perception module based on a deep learning model, combines the user control instructions issued by the remote control center, generates an optimal control strategy and sends it to the agricultural machinery vehicle-mounted control unit. The deep learning model is trained by a large amount of farmland operation sample data and has the ability of environmental self-adaptive adjustment and operation parameter optimization. The multi-dimensional data fusion adopts a weighted fusion algorithm, and the fusion data feature vector X fusion is calculated according to the formula: X fusion = ω1X environment + ω2X agricultural machinery + ω3X operation, wherein ω1, ω2 and ω3 are weight coefficients of the farmland environmental data, the agricultural machinery running parameter data and the operation object data, and ω1+ω2+ω3=1. The weight coefficients are optimized by a gradient descent algorithm.

2. The intelligent agricultural machinery remote control system according to claim 1, characterized in that, The agricultural machinery vehicle-mounted control unit is used for analyzing the optimal control strategy issued by the AI intelligent decision module, generating a driving signal and driving the execution module to execute corresponding operation actions. The communication transmission module adopts a 5G and satellite communication dual-mode redundancy design to realize low-delay and high-reliability data transmission between the remote control center, the AI intelligent decision module and the agricultural machinery vehicle-mounted control unit. The execution module includes a walking driving mechanism, an operation execution mechanism and a safety protection mechanism of the agricultural machinery, and is used for completing specified operation tasks and safety protection actions in response to the driving signal of the vehicle-mounted control unit. The AI intelligent decision module includes a data preprocessing submodule, an environmental situation assessment submodule, an operation strategy optimization submodule and a fault early warning submodule. The data preprocessing submodule is used for denoising, standardizing and data aligning the original data collected by the multi-source perception module, eliminating abnormal data and generating standardized data samples. The environmental situation assessment submodule performs image semantic segmentation on the standardized farmland environmental data based on a convolutional neural network (CNN), identifies obstacles, crop types, crop growth states and terrain features in the farmland, and generates a farmland environmental situation assessment report. The semantic segmentation adopts an improved U-Net network, and the loss function adopts a weighted sum of cross-entropy loss function and Dice loss function, and the specific expression is: Loss = α × CrossEntropyLoss + (1-α) × DiceLoss, wherein α is a weight coefficient, and the value range is 0.3-0.

7. The operation strategy optimization submodule dynamically optimizes the operation path, operation speed and operation parameters of the agricultural machinery based on a reinforcement learning algorithm, combines the environmental situation assessment report and the user control instructions, and generates an optimal operation strategy. The reinforcement learning adopts a DQN algorithm, and the action value function update formula is: Q(s,a) = Q(s,a) + γ × [r + maxQ(s',a') - Q(s,a)], wherein s is a current state, a is a current action, r is an immediate reward, s' is a next state, and γ is a discount factor, and the value range of γ is 0.8-0.95; The fault early warning sub-module performs time series analysis on the operation parameter data of the agricultural machine based on a recurrent neural network (RNN), predicts potential faults of the agricultural machine, and generates fault early warning information, which is pushed to the remote control center and the on-board control unit of the agricultural machine. The fault prediction adopts an LSTM network, and the fault early warning is achieved by calculating an abnormal score of the time series sequence of the operation parameters, and the abnormal score calculation formula is: Score = ||x^-x||2 / ||x||2, wherein x is an actual operation parameter vector, x^ is a model predicted parameter vector, and ||·||2 is an L2 norm.

3. The intelligent agricultural machinery remote control system according to claim 1, characterized in that, The multi-source perception module comprises a visual sensor, a laser radar, a millimeter wave radar, a GPS / Beidou positioning module, an inertial measurement unit, and an agricultural machine operation parameter sensor. The visual sensor is used to collect farmland environment images and work object images. The laser radar and the millimeter wave radar are used to detect farmland obstacles and terrain undulations. The GPS / Beidou positioning module and the inertial measurement unit are used to obtain real-time position and attitude data of the agricultural machine. The agricultural machine operation parameter sensor is used to collect engine speed, oil pressure, hydraulic system pressure, and battery capacity data.

4. The intelligent agricultural machinery remote control system according to claim 1, characterized in that, The AI intelligent decision module also has a model self-adaptive updating function, can receive new farmland work data in real time through an edge computing node, perform incremental training on a deep learning model, optimize the environmental recognition accuracy and work strategy generation efficiency of the model, and the model updating process does not affect normal work of the agricultural machine. The incremental training adopts a stochastic gradient descent (SGD) optimizer, and the parameter updating formula is: θt+1 = θt - η × ∇L(θt), wherein θt is a model parameter at time t, θt+1 is a model parameter at time t+1, η is a learning rate, the value range of η is 1e-5-1e-3, and ∇L(θt) is a gradient of a loss function at time t.

5. The intelligent agricultural machinery remote control system according to claim 1, characterized in that, The remote control center comprises an AI visual monitoring sub-module, which constructs a farmland work three-dimensional digital twin model based on environmental data and work state data transmitted by the AI intelligent decision module, renders an agricultural machine work trajectory, crop distribution, and environmental changes in real time, and supports a user to remotely control the agricultural machine through the three-dimensional model.

6. The intelligent agricultural machinery remote control system according to claim 1, characterized in that, A local backup decision unit is arranged between the AI intelligent decision module and the on-board control unit of the agricultural machine, when a communication transmission module fails and remote data transmission is interrupted, the local backup decision unit calls an emergency work strategy pre-stored by the AI intelligent decision module, drives the agricultural machine to complete an emergency escape or work finishing action, and synchronizes data to the remote control center after communication is restored.

7. The intelligent agricultural machinery remote control system according to claim 2, characterized in that, The operation strategy optimization submodule also has a multi-tractor cooperative operation optimization function. When the system simultaneously controls multiple tractors for operation, the operation data sharing and cooperative decision-making among the multiple tractors are realized through a federated learning algorithm, the operation area division and path planning of the multiple tractors are optimized, operation overlap and omission are avoided, and the overall operation efficiency is improved. The federated learning adopts a federated averaging algorithm, and the global model parameter update formula is: Wglobal=1 / N×ΣNi=1Wi, wherein Wglobal is the global model parameter, N is the number of tractors, and Wi is the local model parameter of the i-th tractor.

8. The intelligent agricultural machinery remote control system according to claim 1, characterized in that, The AI intelligent decision-making module also includes an operation effect evaluation submodule. Based on the post-operation farmland data and crop data collected by the multi-source perception module, in combination with a preset operation standard, the operation quality is quantitatively evaluated through an AI algorithm, an operation effect evaluation report is generated, and feedback is provided to the remote control center, thereby providing data support for subsequent operation instruction adjustment of the user.

9. The intelligent agricultural machinery remote control system according to claim 1, characterized in that, The communication transmission module is also provided with a data encryption submodule. An AI-driven dynamic encryption algorithm is used to encrypt the transmitted data, abnormal access behaviors in the data transmission process are identified and intercepted in real time, and the transmission safety of the remote control data is ensured.

10. The intelligent agricultural machine remote control system of claim 5, wherein, The AI visual monitoring submodule also has an AI-assisted decision-making prompting function. When the system identifies that the farmland environment is abnormal or the tractor operation parameters are unreasonable, an optimized control suggestion is automatically generated and highlighted in the three-dimensional digital twin model, thereby assisting the user to make more reasonable remote control decisions.

Images