Multi-mode positioning intelligent corn harvester operation auxiliary system

Abstract

The invention discloses a multi-mode positioning intelligent corn harvester operation auxiliary system, and relates to the technical field of agricultural machinery. The corn harvester operation auxiliary system comprises a multi-mode positioning fusion module, a multi-source data sensing and preprocessing module, an intelligent decision and control center module, a dynamic operation execution module, an edge calculation and data interaction module and an intelligent remote monitoring and fault diagnosis module. The dynamic operation execution module comprises an operation parameter adjusting unit and a driving control unit, the operation parameter adjusting unit adopts an electro-hydraulic proportional control technology, and the driving control unit is combined with a steer-by-wire and driving technology to form closed-loop control; the system and the method have the advantages that the limitation of traditional single satellite positioning is solved by using a four-mode fusion framework of satellite positioning, inertial navigation, visual positioning and UWB positioning, and inertial navigation, visual positioning and UWB positioning work cooperatively in mountainous areas and other environments in which satellite signals are easy to block.

Description

Technical Field

[0001] The present invention relates to the technical field of agricultural machinery, and in particular to an intelligent corn harvester operation auxiliary system with multi-mode positioning. Background Art

[0002] In modern agricultural production, corn harvesters generally use a single satellite positioning system for navigation and positioning. This method has serious technical defects in actual operation and has become a key bottleneck restricting the development of intelligent corn harvesting.

[0003] When a single satellite positioning system faces a complex and changeable farmland environment, the positioning accuracy is difficult to guarantee. In mountainous areas, the rolling hills can easily block satellite signals, resulting in signal interruption or weakening, causing deviations in the positioning of corn harvesters. When encountering severe weather such as heavy rain and sandstorms, signal transmission is interfered with and the positioning error increases significantly. Inaccurate positioning directly causes the corn harvester to be unable to operate accurately according to the preset path, resulting in a large number of missed and repeated harvests. Missed harvests will cause corn yield losses and reduce economic benefits, while repeated harvests will increase the workload of the corn harvester, aggravate mechanical wear, and increase operating costs. In addition, inaccurate positioning will also affect the continuity and smoothness of the corn harvester's operation, greatly reducing the harvesting efficiency and making it difficult to meet the large-scale and efficient production requirements of modern agriculture. To this end, we propose an intelligent corn harvester operation assistance system with multi-mode positioning. Summary of the Invention

[0004] The purpose of the present invention is to provide an intelligent corn harvester operation assistance system with multi-mode positioning.

[0005] To achieve the above-mentioned objectives, the present invention provides the following technical solutions: a multi-mode positioning intelligent corn harvester operation assistance system, including a corn harvester operation assistance system, the corn harvester operation assistance system including a multi-mode positioning fusion module, a multi-source data perception and preprocessing module, an intelligent decision-making and control center module, a dynamic operation execution module, an edge computing and data interaction module, and an intelligent remote monitoring and fault diagnosis module. The multi-mode positioning fusion module uses a four-mode fusion architecture of satellite positioning + inertial navigation + visual positioning + UWB positioning. The dynamic operation execution module includes an operation parameter adjustment unit and a driving control unit, wherein the operation parameter adjustment unit adopts electro-hydraulic proportional control technology, and the driving control unit combines wire-controlled steering and drive technology. Based on the positioning data of the multi-mode positioning fusion module and the path changes of intelligent decision-making, the drive wheel speed and steering angle are adjusted in real time according to the wire-controlled system. The built-in feedback sensor of the actuator transmits the working status of the component to the intelligent decision-making and control center module in real time at a frequency of 100Hz to form a closed-loop control.

[0006] As a further solution of the present invention: In the multi-mode positioning fusion module, the satellite positioning unit integrates the BeiDou-3 global satellite navigation system and the Galileo system. The inertial navigation unit is based on high-precision MEMS inertial devices and is combined with a zero-speed correction algorithm to control the positioning error to between 1m and 5m within 30 minutes after the satellite signal is lost. The visual positioning unit is equipped with a deep learning target detection model to perform real-time recognition of field crop rows and landmarks, and construct a visual map to achieve positioning operations. The UWB positioning unit needs to add and deploy a UWB base station to achieve high-precision local positioning. The module has a built-in adaptive weight fusion algorithm to automatically adjust the weight of each positioning source according to environmental characteristics. The calculation formula is as follows:

[0007]

[0008] Among them, W i is the i-th positioning source, i=1, 2, 3, 4 correspond to the weights of satellite positioning, inertial navigation, visual positioning and UWB positioning respectively, E i is the error evaluation value of the i-th positioning source, which is calculated by weighting the signal strength, historical positioning accuracy, and current environmental adaptability.

[0009] As a further solution of the present invention: the multi-source data perception and preprocessing module constructs a multi-source heterogeneous sensor network, including a corn growth status sensor, an obstacle detection sensor, a soil condition sensor, a crop density sensor, and a light intensity sensor. The corn growth status sensor uses hyperspectral imaging technology to monitor the 10m ahead every minute. 2 The system scans the entire area to acquire image data containing 200 spectral bands. The corn maturity and pest and disease conditions are analyzed based on the spectral characteristics. The obstacle detection sensor integrates lidar, millimeter-wave radar, and ultrasonic sensors. The soil condition sensor integrates temperature, humidity, conductivity, and pH parameters. Soil data is collected every 10 meters. The crop density sensor uses infrared radiation to count the number of corn plants per unit area in real time. The light intensity sensor uses silicon photocells to collect light data once per second. The collected data is processed by a data preprocessing algorithm and output as a data stream in a unified format.

[0010] As a further solution of the present invention: a multi-module joint analysis mechanism is established between the multi-mode positioning fusion module, the multi-source data perception and preprocessing module and the intelligent decision-making and control central module. The intelligent decision-making and control central module is based on the high-precision positioning data provided by the multi-mode positioning fusion module and the environmental information collected by the multi-source data perception and preprocessing module, and uses the spatiotemporal correlation analysis algorithm to perform predictive analysis on the corn growth trend, obstacle movement trajectory and soil change law. The operation path is planned based on the analysis of continuous positioning data and crop density sensor data, and the prediction of changes in the corn growth area. The obstacle movement trend is predicted by combining the historical obstacle detection data and positioning information.

[0011] As a further solution of the present invention: the intelligent decision-making and control center module constructs an intelligent decision-making model based on deep learning and reinforcement learning algorithms. The module receives the location information of the multi-mode positioning fusion module and the environmental data of the multi-source data perception and preprocessing module, extracts features of the visual image according to the convolutional neural network, and uses the long short-term memory network to analyze the time series data. The deep Q network is combined to realize the dynamic optimization of the operation strategy, and generates the operation parameters of harvesting speed, header height and threshing intensity according to the maturity, density and soil conditions of corn, and plans a collision-free driving path according to the obstacle distribution and positioning information.

[0012] As a further solution of the present invention: the dynamic operation execution module uses an intelligent actuator designed with hydraulic-electric hybrid drive, and the operation parameter adjustment unit adopts electro-hydraulic proportional control to adjust the cutting platform lifting, threshing drum, speed and straw crushing device. The driving control unit combines wire-controlled steering and drive technology. In the cutting platform lifting adjustment, the electro-hydraulic proportional valve response time is <100ms, and the cutting platform height can be quickly adjusted within the range of 10cm-50cm. The threshing drum speed adjustment range is 500r / min-1500r / min, and the adjustment accuracy is ±10r / min. The straw crushing device automatically adjusts the blade speed and cutting angle according to the hardness of the straw. The driving control unit controls the drive motor and steering motor according to the wire control system based on the positioning data of the multi-mode positioning fusion module and the path planning of intelligent decision-making, and controls the speed fluctuation of the corn harvester within ±0.2km / h when driving in a straight line, and the angle error during steering is ±1°.

[0013] As a further solution of the present invention: the edge computing and data interaction module introduces an edge computing model, sets an edge computing node at the corn harvester end, performs local processing and analysis on the real-time collected data, and sets a 5G+LoRa dual communication link. The 5G network is used to upload operation data to the management cloud platform, and the LoRa network is used for data interaction between the corn harvester and surrounding agricultural equipment. The edge computing node adopts a multi-core processor and GPU collaborative computing architecture to perform real-time filtering and feature extraction operations on sensor data, perform target detection and semantic segmentation processing on visual images, and perform error correction and trajectory prediction on positioning data. The 5G network uses slicing technology to allocate exclusive bandwidth according to the data type. The LoRa network uses an adaptive rate adjustment mechanism to automatically adjust the traffic rate according to the distance between devices and the signal strength.

[0014] As a further solution of the present invention: the intelligent remote monitoring and fault diagnosis module consists of two parts: a remote monitoring platform and a fault diagnosis system. The remote monitoring platform is built on the rural management cloud platform. Managers use computers and mobile terminals to log in to the platform to view the location, operation status and operation parameters of the corn harvester in real time. The fault diagnosis system analyzes the sensor data collected by the multi-source data perception and preprocessing module and the component working status data fed back by the dynamic operation execution module, and combines the preset fault diagnosis rule library and machine learning algorithm to perform real-time diagnosis of the corn harvester's fault. When a fault or abnormality is detected, an early warning information is generated and pushed to the manager terminal through 5G. At the same time, a maintenance method is given according to the fault cause analysis results.

[0015] As a further solution of the present invention: the corn harvester sets a multi-system linkage energy-saving strategy, the edge computing and data interaction module evaluates the system energy consumption status according to the operation intensity and ambient temperature data, and transmits the information to the intelligent decision-making and control center module. The intelligent decision-making and control center module combines the path information of the multi-mode positioning fusion module and the load data of the dynamic operation execution module, and dynamically adjusts the corn harvester engine power, hydraulic system pressure and power equipment operating status according to the energy consumption optimization algorithm.

[0016] By adopting the above technical solution, compared with the prior art, the beneficial effects of the present invention are:

[0017] 1. The present invention overcomes the limitations of traditional single satellite positioning by using a four-mode fusion architecture of "satellite positioning + inertial navigation + visual positioning + UWB positioning". In environments where satellite signals are easily blocked, such as mountainous areas, inertial navigation, visual positioning and UWB positioning work together. Inertial navigation is combined with a zero-speed correction algorithm. UWB positioning achieves high-precision local positioning by deploying base stations. Visual positioning uses a deep learning model to identify crop rows and construct maps to assist positioning. In severe weather, multiple positioning sources automatically adjust their weights and fuse according to the environment to ensure positioning accuracy, effectively avoid missed harvests and duplicate harvests due to inaccurate positioning, significantly improve harvest quality and economic benefits, and ensure operational continuity and smoothness, thereby improving harvest efficiency.

[0018] 2. The present invention constructs a multi-source heterogeneous sensor network through a multi-source data perception and preprocessing module to collect information such as corn growth status, obstacle distribution, and soil conditions in real time. The standardized data stream is output through the spatiotemporal calibration and preprocessing algorithm of multi-sensor data. The intelligent decision-making and control center module conducts in-depth analysis of this data based on deep learning and reinforcement learning algorithms, and uses spatiotemporal correlation analysis algorithms to predict corn growth trends, obstacle movement trajectories, etc. Combined with an improved path planning algorithm and a dynamic optimization model for operation strategies, the system automatically generates operation parameters including harvesting speed, header height, and threshing intensity according to corn maturity, density, and soil conditions, and plans a collision-free driving path. Compared with traditional harvesters, the system has significantly enhanced adaptability to different operating environments and can effectively reduce the cost of corn harvesting operations.

[0019] 3. The present invention uses an intelligent remote monitoring and fault diagnosis module through a remote monitoring platform, so that managers can view the harvester's location, operating status and parameters in real time to achieve remote management. The fault diagnosis system adopts transfer learning, integrated learning and fault probability map analysis methods, combined with multi-source sensor data and component working status data, to perform real-time diagnosis of harvester faults, and the diagnostic accuracy is greatly improved. When a potential fault or abnormality is detected, the system promptly generates early warning information and pushes it to the terminal, while pointing out the faulty components and providing maintenance guidance plans, reducing equipment downtime and ensuring safe and efficient operations. The multi-system linkage energy-saving strategy set by the system, through the coordination of edge computing and data interaction modules and intelligent decision-making and control center modules, optimizes energy consumption based on genetic algorithms, and further improves the comprehensive benefits of agricultural production. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a workflow diagram of the corn harvester operation assistance system in an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The specific embodiments of the present invention will be further described below in conjunction with the accompanying drawings. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0022] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0023] Please see the attached Figure 1 The present invention provides a multi-mode positioning intelligent corn harvester operation assistance system, including a corn harvester operation assistance system, which includes a multi-mode positioning fusion module, a multi-source data perception and preprocessing module, an intelligent decision-making and control center module, a dynamic operation execution module, an edge computing and data interaction module, and an intelligent remote monitoring and fault diagnosis module. The multi-mode positioning fusion module uses a four-mode fusion architecture of satellite positioning + inertial navigation + visual positioning + UWB positioning. The dynamic operation execution module includes an operation parameter adjustment unit and a driving control unit, wherein the operation parameter adjustment unit adopts electro-hydraulic proportional control technology, and the driving control unit combines wire-controlled steering and drive technology. Based on the positioning data of the multi-mode positioning fusion module and the path change of intelligent decision-making, the driving wheel speed and steering angle are adjusted in real time according to the wire-controlled system. The built-in feedback sensor of the actuator transmits the working status of the component to the intelligent decision-making and control center module in real time at a frequency of 100Hz to form a closed-loop control.

[0024] In one embodiment of the present invention, in a multi-mode positioning fusion module, the satellite positioning unit integrates the BeiDou-3 global satellite navigation system and the Galileo system. The inertial navigation unit is based on high-precision MEMS inertial devices and is combined with a zero-speed correction algorithm to control the positioning error to between 1m and 5m within 30 minutes after the satellite signal is lost. The visual positioning unit is equipped with a deep learning target detection model to perform real-time recognition of field crop rows and landmarks, and construct a visual map to achieve positioning operations. The UWB positioning unit needs to add and deploy a UWB base station to achieve high-precision local positioning. The module has a built-in adaptive weight fusion algorithm to automatically adjust the weight of each positioning source according to environmental characteristics. The calculation formula is as follows:

[0025]

[0026] Among them, W i is the i-th positioning source, i=1, 2, 3, 4 correspond to the weights of satellite positioning, inertial navigation, visual positioning and UWB positioning respectively, E i is the error evaluation value of the i-th positioning source, which is calculated by weighting the signal strength, historical positioning accuracy, and current environmental adaptability.

[0027] In one embodiment of the present invention, the multi-source data perception and preprocessing module constructs a multi-source heterogeneous sensor network, including a corn growth status sensor, an obstacle detection sensor, a soil condition sensor, a crop density sensor, and a light intensity sensor. The corn growth status sensor uses hyperspectral imaging technology to monitor the surrounding area 10m ahead every minute. 2 The system scans the entire area to acquire image data containing 200 spectral bands. The corn maturity and pest and disease conditions are analyzed based on the spectral characteristics. The obstacle detection sensor integrates lidar, millimeter-wave radar, and ultrasonic sensors. The soil condition sensor integrates temperature, humidity, conductivity, and pH parameters. Soil data is collected every 10 meters. The crop density sensor uses infrared radiation to count the number of corn plants per unit area in real time. The light intensity sensor uses silicon photocells to collect light data once per second. The collected data is processed by a data preprocessing algorithm and output as a data stream in a unified format.

[0028] In one embodiment of the present invention: a multi-module joint analysis mechanism is established between the multi-mode positioning fusion module, the multi-source data perception and preprocessing module and the intelligent decision-making and control central module. The intelligent decision-making and control central module uses the high-precision positioning data provided by the multi-mode positioning fusion module and the environmental information collected by the multi-source data perception and preprocessing module, and uses the spatiotemporal correlation analysis algorithm to perform predictive analysis on the corn growth trend, obstacle movement trajectory and soil change law. The operation path is planned based on the analysis of continuous positioning data and crop density sensor data, and the prediction of changes in the corn growth area. The obstacle movement trend is predicted by combining the historical obstacle detection data and positioning information.

[0029] In one embodiment of the present invention: the intelligent decision-making and control center module is an intelligent decision-making model constructed based on deep learning and reinforcement learning algorithms. The module receives the location information of the multi-mode positioning fusion module and the environmental data of the multi-source data perception and preprocessing module, extracts features of the visual image according to the convolutional neural network, and uses the long short-term memory network to analyze the time series data. The deep Q network is combined to realize the dynamic optimization of the operation strategy, and generates the operation parameters of the harvesting speed, header height and threshing intensity according to the maturity, density and soil conditions of corn, and plans a collision-free driving path according to the obstacle distribution and positioning information.

[0030] In one embodiment of the present invention: the dynamic operation execution module uses an intelligent actuator designed with hydraulic-electric hybrid drive, the operation parameter adjustment unit adopts electro-hydraulic proportional control to adjust the cutting platform lifting, threshing drum, speed and straw crushing device, and the driving control unit combines wire-controlled steering and drive technology. In the cutting platform lifting adjustment, the electro-hydraulic proportional valve response time is <100ms, and the cutting platform height can be quickly adjusted within the range of 10cm-50cm. The threshing drum speed adjustment range is 500r / min-1500r / min, and the adjustment accuracy is ±10r / min. The straw crushing device automatically adjusts the blade speed and cutting angle according to the hardness of the straw. The driving control unit controls the drive motor and steering motor according to the wire control system based on the positioning data of the multi-mode positioning fusion module and the path planning of intelligent decision-making, and controls the speed fluctuation of the corn harvester within ±0.2km / h when driving in a straight line, and the angle error during steering is ±1°.

[0031] In one embodiment of the present invention: the edge computing and data interaction module introduces an edge computing model, an edge computing node is set at the corn harvester end, the real-time collected data is locally processed and analyzed, and a 5G+LoRa dual communication link is set. The 5G network is used to upload operation data to the management cloud platform, and the LoRa network is used for data interaction between the corn harvester and surrounding agricultural equipment. The edge computing node adopts a multi-core processor and GPU collaborative computing architecture to perform real-time filtering and feature extraction operations on sensor data, perform target detection and semantic segmentation processing on visual images, and perform error correction and trajectory prediction on positioning data. The 5G network uses slicing technology to allocate exclusive bandwidth according to the data type. The LoRa network uses an adaptive rate adjustment mechanism to automatically adjust the traffic rate according to the distance between devices and the signal strength.

[0032] In one embodiment of the present invention: the intelligent remote monitoring and fault diagnosis module consists of two parts: a remote monitoring platform and a fault diagnosis system. The remote monitoring platform is built on the rural management cloud platform. The management personnel use computers and mobile phone terminals to log in to the platform to view the location, operation status and operation parameters of the corn harvester in real time. The fault diagnosis system analyzes the sensor data collected by the multi-source data perception and preprocessing module and the component working status data fed back by the dynamic operation execution module, and combines the preset fault diagnosis rule library and machine learning algorithm to perform real-time diagnosis of the corn harvester's fault. When a fault or abnormality is detected, an early warning information is generated and pushed to the management personnel terminal through 5G. At the same time, a maintenance method is given according to the fault cause analysis results.

[0033] In one embodiment of the present invention: a corn harvester sets a multi-system linkage energy-saving strategy, the edge computing and data interaction module evaluates the system energy consumption status according to the operation intensity and ambient temperature data, and transmits the information to the intelligent decision-making and control center module, the intelligent decision-making and control center module combines the path information of the multi-mode positioning fusion module and the load data of the dynamic operation execution module, and dynamically adjusts the corn harvester engine power, hydraulic system pressure and power equipment operating status according to the energy consumption optimization algorithm.

[0034] In one embodiment of the present invention: In the intelligent decision-making and control central module, for newly emerging operation scenarios, an online reinforcement learning algorithm is adopted, and trial and error learning is continuously carried out during the operation process to optimize the operation strategy. According to the corn maturity, density and soil conditions, the operation parameters of harvesting speed, header height and threshing intensity are generated. Under normal circumstances, based on historical operation data and machine learning model training, when the corn maturity reaches 90%, the density is 6000 plants / mu, and the soil hardness is moderate, a parameter combination of harvesting speed 5km / h, header height 30cm, and threshing intensity level 6 is generated. According to the obstacle distribution and positioning information, a collision-free driving path is planned. When planning the path, the dynamic constraints such as the minimum turning radius of the harvester and the driving speed limit are considered to ensure the feasibility of the path.

[0035] In one embodiment of the present invention: the satellite positioning unit in the multi-mode positioning fusion module uses the dual-frequency signals of the BeiDou-3 and Galileo systems through a joint tracking loop processing, and adopts a multi-frequency combination positioning algorithm based on the least squares method. The positioning accuracy in open areas can reach 2cm-5cm, the signal update frequency is set to 1Hz, and multi-constellation interoperability protocols are supported to achieve seamless switching. The inertial navigation unit uses a MEMS gyroscope with a zero bias stability of 0.05° / h and an accelerometer with a zero bias of 50μg. The zero-speed correction algorithm performs a gait detection every 10s and triggers the correction when the vehicle is detected to be stationary. The zero bias, scale factor and other parameters of the inertial device are estimated and compensated online through the error state Kalman filter to ensure that the positioning error is controlled within 1m-5m within 30 minutes of satellite signal loss. The visual positioning unit adopts a 12MP global The shutter camera has a frame rate of 30fps, an installation height of 1.5m, a horizontal viewing angle of 90° and a vertical viewing angle of 60°. It is based on an improved target detection model based on YOLOv7. The pre-trained dataset contains 200,000 farmland scene images. By adding a crop row feature enhancement module, the row recognition accuracy is achieved at least 95%. The ORB-SLAM3 algorithm is used to build a visual map with a map update frequency of 5Hz. The positioning accuracy can reach 10cm in sufficient light. The UWB positioning unit uses the Decawave DW1000 chip, with an operating frequency band of 3.5GHz-6.5GHz and a transmission power of 20dBm. It is deployed in the field in a 50m×50m grid, forming an equilateral triangle distribution. Each base station is equipped with a dual power supply system of solar power supply and lithium battery, adopts a bilateral two-way ranging protocol, has a positioning refresh rate of 10Hz, and a positioning accuracy of up to 5cm.

[0036] In one embodiment of the present invention, the error evaluation value E in the adaptive weight fusion algorithm is i The quantitative formula is:

[0037]

[0038] Among them, S i is the current positioning source signal strength (satellite carrier-to-noise ratio / UWB RSSI), S min H is the minimum signal strength threshold for the normal operation of the positioning source. i is the average error of the past 10 positioning times, H avg is the historical average error, C i is the environmental adaptability coefficient (mountainous area / bad weather=1.5, plain / good weather=1).

[0039] In one embodiment of the present invention: Multi-source data perception and pre-processing module sensor parameters and layout:

[0040] Corn growth status sensor: Hyperspectral imager with spectral resolution of 3nm, wavelength range of 400nm-1000nm, installed 1m in front of the header, 1.2m above the ground, with a scanning field angle of 45°, using push-broom imaging mode, scanning 10m ahead every minute 2 The area was scanned, 200 spectral band data were collected each time, and 7 characteristic bands (450nm, 550nm, 680nm, 720nm, 760nm, 850nm, 950nm) were screened out by continuous projection algorithm. The corn maturity (R 2 ≥0.92) and pest and disease (recall rate ≥85%) prediction models;

[0041] Obstacle detection sensors:

[0042] LiDAR: 16-line mechanical LiDAR, with a horizontal angular resolution of 0.25°, a vertical angular resolution of 2°, a detection range of 20m, a scanning frequency of 10Hz, installed at the center of the roof, 2m above the ground, and a pitch angle of 5°;

[0043] Millimeter-wave radar: 77GHz millimeter-wave radar, with a detection speed range of 0km / h-80km / h, an angular resolution of ±3°, installed in the center of the front bumper, and a detection distance of 15m;

[0044] Ultrasonic sensors: 6 sets of ultrasonic sensors with a ranging range of 0.1m-5m and an accuracy of 1cm, installed at the front, rear, left, right, and center corners of the vehicle body to form a 360-degree protection circle;

[0045] Data fusion uses a decision-level fusion method based on DS evidence theory, outputting the location, speed, type (pedestrian / vehicle / tree) and confidence level of obstacles every 100ms;

[0046] Soil condition sensor: A composite probe integrating temperature, humidity, conductivity, and pH sensors. The probe is 20 cm long and inserted into the soil 15 cm deep. The temperature and humidity sensor has an accuracy of ±0.5°C / ±3% RH, a conductivity accuracy of ±5 μS / cm, and a pH accuracy of ±0.1. It automatically touches the ground every 10 m with a sampling interval of 30 seconds. Data is stored in a circular buffer, retaining the most recent 100 sets of data.

[0047] Crop density sensor: The infrared sensor uses a dual-beam design, with a detection distance of 2m and an accuracy of ±1 plant. It is installed at a height of 0.8m and a spacing of 0.5m. It is arranged horizontally along the vehicle body. It calculates crop density by counting the number of plant occlusions per unit area and combining it with the driving speed. The calculation cycle is 1s.

[0048] Light intensity sensor: Silicon photocell sensor with a sensitivity of 0.1 lux, a measurement range of 0-100,000 lux, a response time of <100ms, installed in an unobstructed position on the top of the cab, and a data sampling frequency of 1Hz;

[0049] Data preprocessing process,

[0050] Spatiotemporal alignment: All sensors are connected to a high-precision clock synchronization module based on the IEEE1588 protocol, with a clock synchronization accuracy of ±1μs. Linear interpolation is used to unify sensor data with different sampling frequencies to 100Hz. A rotation and translation matrix is ​​used to transform each sensor's coordinate system into the harvester's global coordinate system (with the origin at the center of the rear wheel axle).

[0051] Data cleaning:

[0052] The 3σ principle is used to eliminate sensor outliers, and interpolation repair is performed for data that exceeds 3 times the standard deviation for 3 consecutive sampling points;

[0053] Use the isolation forest algorithm to detect outliers in the obstacle detection sensor data and filter the detection results with a confidence level of less than 0.5;

[0054] Feature extraction:

[0055] Principal component analysis (PCA) dimensionality reduction was performed on the hyperspectral data, and the first 10 principal components were extracted as eigenvectors;

[0056] Obstacle detection data extracts features such as bounding box coordinates, velocity vector, aspect ratio, etc.

[0057] The soil data were normalized and the temperature, humidity, conductivity and pH value data were mapped to the [0,1] interval.

[0058] Example

[0059] At a corn planting base in a hilly area, there is a corn field covering an area of ​​approximately 500 mu (approximately 1,000 acres) with a planting density of 6,000 plants per mu. The current corn maturity is generally 90%, and the soil hardness is moderate. Some mountainous areas in the area block satellite signals, and short periods of rainfall are possible during operations. A multi-mode positioning intelligent corn harvester operation assistance system is used for corn harvesting. The specific implementation process is as follows:

[0060] 1. Preparation before operation

[0061] Before starting the corn harvester, the operator inputs the electronic map information of the operation plot through the remote monitoring platform of the intelligent remote monitoring and fault diagnosis module, and sets the operation starting point and approximate operation route. At this time, the multi-mode positioning fusion module is activated, and the satellite positioning unit quickly receives the dual-frequency signals of the Beidou-3 and Galileo systems. It adopts the multi-frequency combination positioning algorithm based on the least squares method. In the open operation starting area, the positioning accuracy reaches 3cm and the signal update frequency is 1Hz. At the same time, the multi-source data perception and preprocessing module completes the initialization, and the corn growth status sensor, obstacle detection sensor (combination of lidar, millimeter-wave radar, and ultrasonic sensor), soil condition sensor, etc. begin self-test to ensure that each sensor is working properly.

[0062] 2. Operation process

[0063] (1) Positioning and environmental perception,

[0064] After the harvester enters the cornfield, the multi-mode positioning fusion module senses environmental changes in real time. When entering a mountainous section, the satellite signal is blocked by the mountain and the signal strength decreases. At this time, the module's built-in adaptive weight fusion algorithm quickly takes effect. According to the formula:

[0065]

[0066] Calculate the weight of each positioning source, where k is the environmental adaptability coefficient and the signal strength S of satellite positioning i Lower than the minimum signal strength threshold S for normal operation min , its error evaluation value E i Increase, weight reduced, the inertial navigation unit is based on high-precision MEMS inertial devices (gyroscope with zero bias stability of 0.05° / h and accelerometer with zero bias of 50μg), combined with zero-speed correction algorithm, gait detection every 10s, triggering correction when the vehicle briefly stops to adjust the operating direction, and online estimation and compensation of inertial device parameters through error state Kalman filtering. Within 2 minutes of satellite signal loss, the positioning error is controlled at about 3m, and the weight is increased to 40%. The 12MP global shutter camera of the visual positioning unit is based on the improved The YOLOv7 target detection model performs real-time recognition of crop rows and landmarks in the field with an accuracy of 96%. The ORB-SLAM3 algorithm is used to build a visual map with a map update frequency of 5Hz, maintaining a positioning accuracy of 10cm and a weighting of 30%. UWB positioning units are deployed in base stations arranged in a 50m×50m grid in the field. They use a bilateral two-way ranging protocol with a positioning refresh rate of 10Hz, a positioning accuracy of up to 5cm, and a weighting of 20%. Through the fusion of four modes, the harvester can still accurately locate itself in complex terrain.

[0067] At the same time, the multi-source data perception and preprocessing module continuously collects environmental data, and the corn growth status sensor monitors the corn 10m ahead every minute. 2 The system scans the area and obtains 200 spectral band data. Seven characteristic bands are selected through the continuous projection algorithm. The analysis based on the random forest algorithm shows that the current corn maturity is 91% accurate and no obvious pests and diseases are detected. The obstacle detection sensor fusion lidar is set to 16 lines, a horizontal angular resolution of 0.25° and a detection distance of 20m. The millimeter wave radar is set to 77GHz, with a detection speed range of 0km / h-80km / h. The ultrasonic sensor is set to 6 groups with a ranging range of 0.1m-5m. The DS evidence theory is used for decision-level fusion, and obstacle information is output every 100ms. During the operation, a temporarily parked agricultural tricycle 5m ahead was successfully detected, and its position, speed and type were accurately determined. The soil condition sensor automatically touches the ground for sampling every 10m. The collected temperature, humidity, conductivity and pH value data are transmitted to the intelligent decision-making and control central module after normalization.

[0068] (2) Intelligent decision-making and task execution

[0069] After receiving positioning data from the multi-mode positioning fusion module and environmental information from the multi-source data perception and preprocessing module, the intelligent decision-making and control center module conducts analysis and decision-making based on deep learning and reinforcement learning algorithms. Based on the corn maturity (91%), density (6,000 plants / mu), and soil conditions (moderate), combined with a preset fuzzy rule library, it generates operating parameters: the harvesting speed is set to 5.2 km / h, the header height is adjusted to 32 cm, and the threshing intensity level is set to 6. When an agricultural tricycle obstacle is detected in front, the improved RRT*-Hybrid algorithm is used to plan a collision-free driving path within 180 milliseconds, while taking into account the harvester's minimum turning radius (3 meters) and driving speed limit to ensure the path is feasible.

[0070] After the dynamic operation execution module receives the command, the electro-hydraulic proportional control technology of the operation parameter adjustment unit responds quickly. The electro-hydraulic proportional valve for raising and lowering the harvesting platform operates within 80ms, adjusting the harvesting platform height from the initial position to 32cm. The threshing drum speed is accurately adjusted to 1050r / min by the frequency converter. The straw crushing device automatically adjusts the blade speed to 1200r / min and the cutting angle to 25° based on the real-time detection of straw hardness. The driving control unit combines wire-controlled steering and drive technology. Based on positioning data and planned paths, the drive motor and steering motor are controlled by the wire control system, so that the speed fluctuation of the harvester is always controlled within ±0.15km / h when driving in a straight line, and the angle error is kept at ±0.8° during steering. It accurately bypasses obstacles along the planned path and continues to operate. The feedback sensor built into the actuator transmits the working status of the components to the intelligent decision-making and control center module in real time at a frequency of 100Hz, forming a closed-loop control to ensure stable operation parameters.

[0071] (3) Data processing and communication,

[0072] The edge computing and data interaction module processes real-time collected data locally at the edge computing node on the harvester side, performs real-time filtering and feature extraction on sensor data, performs principal component analysis and dimensionality reduction on hyperspectral data, performs target detection and semantic segmentation on visual images, and performs error correction and trajectory prediction on positioning data, reducing data transmission by 75%. The 5G network uses slicing technology to allocate exclusive bandwidth for operation control data, ensuring that control instructions are uploaded to the management cloud platform with a delay of less than 10ms. The LoRa network exchanges data with surrounding unmanned plant protection aircraft and intelligent transport vehicles in a low-power, long-distance manner to achieve collaborative allocation of operation tasks. In one interaction with an intelligent transport vehicle, the harvested corn transportation task was successfully assigned to a nearby transport vehicle. Based on the received information, the transport vehicle planned the driving route in advance and headed to the harvester location to wait for loading.

[0073] (4) Remote monitoring and fault diagnosis,

[0074] The remote monitoring platform of the intelligent remote monitoring and fault diagnosis module uses WebGL technology to provide managers with a 3D visualization interface. Managers can view the harvester's location, operating status, and operating parameters in real time through computer terminals in the office. The fault diagnosis system continuously analyzes sensor data and component operating conditions. During operation, when abnormal fluctuations in the load current of the threshing drum are detected, the system combines the preset fault diagnosis rule library and machine learning algorithm to determine the potential failure of the threshing drum to be blocked. Fault warning information is immediately generated and pushed to the mobile terminals of managers and on-site workers via the 5G network. At the same time, the system uses AR technology to display a virtual model of the threshing drum on the mobile phone, highlighting the location of components that may be blocked, and providing maintenance instructions including animated demonstrations of disassembly steps and a list of required tools. According to the instructions, the operator can complete the troubleshooting within 20 minutes and resume normal operation.

[0075] 3. Summary after homework

[0076] After the operation is completed, the edge computing and data interaction module uploads the data of the entire operation process to the management cloud platform. According to statistics, when using the intelligent corn harvester operation assistance system, the harvesting efficiency of the 500-acre corn field is improved by 35% compared with the traditional harvesting method, the corn missed rate is reduced from the original 8% to 2%, the double harvesting phenomenon is basically eliminated, and the corn harvesting quality is significantly improved. At the same time, through the multi-system linkage energy-saving strategy, combined with the energy consumption optimization algorithm to dynamically adjust the engine power, hydraulic system pressure, etc., the overall energy consumption of this operation is reduced by 18%, which effectively saves the operation cost, demonstrating the efficiency, accuracy and energy saving of the system of the present invention in practical applications.

[0077] Although the present invention is disclosed above with reference to preferred embodiments, this is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications without departing from the spirit and scope of the present invention. Therefore, any modifications, equivalent variations, and modifications made to the above embodiments in accordance with the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the scope of protection defined by the claims of the present invention.

Claims

1. A multi-mode positioning intelligent corn harvester operation assistance system, including a corn harvester operation assistance system, characterized by: The corn harvester operation assistance system includes a multi-mode positioning fusion module, a multi-source data perception and preprocessing module, an intelligent decision-making and control center module, a dynamic operation execution module, an edge computing and data interaction module, and an intelligent remote monitoring and fault diagnosis module. The multi-mode positioning fusion module uses a four-mode fusion architecture of satellite positioning + inertial navigation + visual positioning + UWB positioning. The dynamic operation execution module includes an operation parameter adjustment unit and a driving control unit. The operation parameter adjustment unit adopts electro-hydraulic proportional control technology. The driving control unit combines wire-controlled steering and drive technology. Based on the positioning data of the multi-mode positioning fusion module and the path changes of intelligent decision-making, the drive wheel speed and steering angle are adjusted in real time according to the wire-controlled system. The built-in feedback sensor of the actuator transmits the working status of the component to the intelligent decision-making and control center module in real time at a frequency of 100Hz to form a closed-loop control.

2. The multi-mode positioning intelligent corn harvester operation assistance system according to claim 1, characterized in that: In the multi-mode positioning fusion module, the satellite positioning unit integrates the BeiDou-3 global satellite navigation system and the Galileo system. The inertial navigation unit is based on high-precision MEMS inertial devices and is combined with a zero-speed correction algorithm to control the positioning error to between 1m and 5m within 30 minutes after the satellite signal is lost. The visual positioning unit is equipped with a deep learning target detection model to perform real-time recognition of field crop rows and landmarks, and build a visual map to achieve positioning operations. The UWB positioning unit requires the addition and deployment of a UWB base station to achieve high-precision local positioning. The module has a built-in adaptive weight fusion algorithm that automatically adjusts the weight of each positioning source according to environmental characteristics. The calculation formula is as follows: Among them, W i is the i-th positioning source, i=1, 2, 3, 4 correspond to the weights of satellite positioning, inertial navigation, visual positioning and UWB positioning respectively, E i is the error evaluation value of the i-th positioning source, which is calculated by weighting the signal strength, historical positioning accuracy, and current environmental adaptability.

3. The multi-mode positioning intelligent corn harvester operation assistance system according to claim 2, characterized in that: The multi-source data perception and preprocessing module constructs a multi-source heterogeneous sensor network, including corn growth status sensor, obstacle detection sensor, soil condition sensor, crop density sensor and light intensity sensor. The corn growth status sensor uses hyperspectral imaging technology to monitor the 10m ahead every minute. 2 The system scans the entire area to acquire image data containing 200 spectral bands. The corn maturity and pest and disease conditions are analyzed based on the spectral characteristics. The obstacle detection sensor integrates lidar, millimeter-wave radar, and ultrasonic sensors. The soil condition sensor integrates temperature, humidity, conductivity, and pH parameters. Soil data is collected every 10 meters. The crop density sensor uses infrared radiation to count the number of corn plants per unit area in real time. The light intensity sensor uses silicon photocells to collect light data once per second. The collected data is processed by a data preprocessing algorithm and output as a data stream in a unified format.

4. The multi-mode positioning intelligent corn harvester operation assistance system according to claim 3 is characterized by: A multi-module joint analysis mechanism is established between the multi-mode positioning fusion module, the multi-source data perception and preprocessing module, and the intelligent decision-making and control central module. The intelligent decision-making and control central module uses the high-precision positioning data provided by the multi-mode positioning fusion module and the environmental information collected by the multi-source data perception and preprocessing module, and uses a spatiotemporal correlation analysis algorithm to perform predictive analysis on corn growth trends, obstacle movement trajectories, and soil change patterns. The operation path is planned based on the analysis of continuous positioning data and crop density sensor data, and the prediction of changes in corn growth areas. The obstacle movement trend is predicted by combining historical obstacle detection data and positioning information.

5. The multi-mode positioning intelligent corn harvester operation assistance system according to claim 4 is characterized in that: The intelligent decision-making and control center module builds an intelligent decision-making model based on deep learning and reinforcement learning algorithms. The module receives location information from the multi-mode positioning fusion module and environmental data from the multi-source data perception and preprocessing module, extracts features from visual images based on the convolutional neural network, and uses the long short-term memory network to analyze the time series data. It combines the deep Q network to achieve dynamic optimization of the operation strategy, generates operation parameters such as harvesting speed, header height and threshing intensity according to the maturity, density and soil conditions of corn, and plans a collision-free driving path according to the obstacle distribution and positioning information.

6. The multi-mode positioning intelligent corn harvester operation assistance system according to claim 5, characterized in that: The dynamic operation execution module uses an intelligent actuator designed with hydraulic-electric hybrid drive. The operation parameter adjustment unit adopts electro-hydraulic proportional control to adjust the cutting platform lifting, threshing drum, speed and straw crushing device. The driving control unit combines wire-controlled steering and drive technology. In the cutting platform lifting adjustment, the electro-hydraulic proportional valve response time is less than 100ms, and the cutting platform height can be quickly adjusted within the range of 10cm-50cm. The threshing drum speed adjustment range is 500r / min-1500r / min, and the adjustment accuracy is ±10r / min. The straw crushing device automatically adjusts the blade speed and cutting angle according to the hardness of the straw. The driving control unit controls the drive motor and steering motor according to the wire control system based on the positioning data of the multi-mode positioning fusion module and the path planning of intelligent decision-making, and controls the speed fluctuation of the corn harvester within ±0.2km / h when driving in a straight line, and the angle error during steering is ±1°.

7. The multi-mode positioning intelligent corn harvester operation assistance system according to claim 6, characterized in that: The edge computing and data interaction module introduces an edge computing model, sets an edge computing node at the corn harvester end, performs local processing and analysis on the real-time collected data, and sets up a 5G+LoRa dual communication link. The 5G network is used to upload operation data to the management cloud platform, and the LoRa network is used for data interaction between the corn harvester and surrounding agricultural equipment. The edge computing node adopts a multi-core processor and GPU collaborative computing architecture to perform real-time filtering and feature extraction operations on sensor data, perform target detection and semantic segmentation processing on visual images, and perform error correction and trajectory prediction on positioning data. The 5G network uses slicing technology to allocate exclusive bandwidth according to the data type. The LoRa network uses an adaptive rate adjustment mechanism to automatically adjust the traffic rate according to the distance between devices and the signal strength.

8. The multi-mode positioning intelligent corn harvester operation assistance system according to claim 7, characterized in that: The intelligent remote monitoring and fault diagnosis module consists of two parts: a remote monitoring platform and a fault diagnosis system. The remote monitoring platform is built on the rural management cloud platform. Managers use computers and mobile terminals to log in to the platform to view the location, operating status and operating parameters of the corn harvester in real time. The fault diagnosis system analyzes the sensor data collected by the multi-source data perception and preprocessing module and the component working status data feedback from the dynamic operation execution module, and combines the preset fault diagnosis rule library and machine learning algorithm to perform real-time diagnosis of the corn harvester's fault. When a fault or abnormality is detected, an early warning information is generated and pushed to the manager terminal through 5G. At the same time, a maintenance method is given based on the fault cause analysis results.

9. The multi-mode positioning intelligent corn harvester operation assistance system according to claim 8, characterized in that: The corn harvester sets up a multi-system linkage energy-saving strategy. The edge computing and data interaction module evaluates the system energy consumption status according to the operation intensity and ambient temperature data, and transmits the information to the intelligent decision-making and control center module. The intelligent decision-making and control center module combines the path information of the multi-mode positioning fusion module and the load data of the dynamic operation execution module, and dynamically adjusts the corn harvester engine power, hydraulic system pressure and power equipment operating status according to the energy consumption optimization algorithm.

Images