An environmental perception-driven automatic navigation system for barley harvesters

Abstract

The present invention relates to the technical field of agricultural mechanization, and specifically to an automatic navigation system for a highland barley harvester driven by environmental perception, including an enhanced environmental perception module, a data processing and analysis module, a nighttime intelligent navigation module, an optimal path planning module, a dynamic environmental adaptation module, and an execution control module; wherein: the enhanced environmental perception module is used to collect terrain, weather, and crop growth information in a working environment in real time under various lighting conditions; the data processing and analysis module is used to generate environmental characteristic data; the nighttime intelligent navigation module is used to generate a high-precision nighttime working map; and the optimal path planning module is used to calculate and generate an optimal navigation path. The present invention significantly improves the navigation accuracy and environmental adaptability of the highland barley harvester in complex environments by integrating advanced sensing technology and intelligent navigation algorithms, and ensures efficient and continuous operation under various lighting and meteorological conditions.

Description

Technical Field

[0001] The present invention relates to the technical field of agricultural mechanization, and in particular to an automatic navigation system for a highland barley harvester driven by environment perception. Background Art

[0002] In modern agricultural production, highland barley is an important food crop, and the level of automation and intelligence in its harvesting process directly affects agricultural efficiency and crop yield. Although a variety of harvesting machines have been widely used, they mainly rely on basic GPS navigation systems for operation. These systems usually have poor navigation accuracy and environmental adaptability at night or in low light conditions. In addition, existing automatic navigation systems often ignore the impact of complex terrain and sudden meteorological conditions on harvesting operations, such as changes in terrain slope, soil moisture, obstacle locations and other factors, all of which may lead to reduced harvesting efficiency or operation interruptions.

[0003] The present invention aims to solve the problems existing in the prior art, especially to improve the operating accuracy and efficiency of highland barley harvesters under complex environmental conditions. Specifically, the prior art lacks effective navigation and environmental perception capabilities at night or under low light conditions, and it is difficult to respond to sudden terrain and weather changes in real time. In addition, the existing system is deficient in processing multi-source environmental data, such as the integration of terrain, obstacles and crop growth status information, resulting in the inability to accurately plan and adjust the operating path. Summary of the invention

[0004] Based on the above objectives, the present invention provides an automatic navigation system for a highland barley harvester driven by environment perception.

[0005] An automatic navigation system for a highland barley harvester driven by environmental perception includes an enhanced environmental perception module, a data processing and analysis module, a nighttime intelligent navigation module, an optimal path planning module, a dynamic environmental adaptation module, and an execution control module; wherein:

[0006] Enhanced environmental perception module: Integrates lidar, infrared sensors, ultrasonic sensors and multispectral cameras to collect real-time information on terrain, weather and crop growth conditions in the operating environment under various lighting conditions;

[0007] Data processing and analysis module: Receives multi-source data from the enhanced environmental perception module, processes and analyzes the point cloud data using the point cloud network algorithm, identifies obstacles, terrain features, and crop maturity in the operating area, and generates environmental feature data;

[0008] Nighttime intelligent navigation module: Generates high-precision nighttime operation maps using environmental feature data provided by the data processing and analysis module;

[0009] Optimal path planning module: Calculates and generates the optimal navigation path based on the environmental feature data and night operation map generated by the dynamic environment adaptation module and the night intelligent navigation module;

[0010] Dynamic environment adaptation module: obtains the navigation path generated from the optimal path planning module, evaluates environmental feature data in real time, and generates real-time adjustment instructions based on the navigation path and the operating speed of the highland barley harvester to cope with sudden weather changes and dynamic terrain changes;

[0011] Execution control module: receives the navigation path generated by the optimal path planning module, controls the speed and direction of the harvester, and uses the real-time adjustment instructions of the dynamic environment adaptation module to ensure that the harvester responds in time when encountering sudden obstacles.

[0012] Optionally, the enhanced environment perception module includes a laser radar unit, an infrared sensor unit, an ultrasonic sensor unit and a multi-spectral camera unit; wherein:

[0013] LiDAR unit: used to emit laser beams under various lighting conditions and receive reflected signals, and to obtain terrain data by calculating the flight time of the laser beam to measure the height and slope of the terrain;

[0014] Infrared sensor unit: used to detect thermal radiation signals in the working environment at night and in low light conditions. It collects thermal imaging data in real time through infrared sensors to identify crops and obstacles to supplement environmental information during the day and at night.

[0015] Ultrasonic sensor unit: used to transmit ultrasonic signals and receive reflected echoes under various lighting conditions, measure the position and distance of close-range obstacles by calculating the propagation time of the echoes, and generate obstacle detection data in real time;

[0016] Multispectral camera unit: used to capture spectral images of different bands under various lighting conditions, and to detect the growth and health of crops in real time by analyzing the reflected light information in the multispectral images, and to generate data on the growth and health of crops.

[0017] Optionally, the data processing and analysis module includes a data receiving unit, a point cloud processing unit, a feature extraction unit and a feature analysis unit; wherein:

[0018] Data receiving unit: used to receive multi-source data from the enhanced environmental perception module, including lidar data, infrared data, ultrasonic data and multispectral image data, and perform data preprocessing to ensure a unified data format;

[0019] Point cloud processing unit: Use point cloud network algorithm to process the point cloud data generated by LiDAR, extract and classify the features of each point in the point cloud data, identify the position and shape of obstacles in the working area, and generate preliminary terrain feature data;

[0020] Feature extraction unit: extracts terrain height and slope information from the terrain feature data generated by the point cloud processing unit, and combines infrared data and multispectral image data to extract crop health and growth characteristics, including leaf color and plant density parameters;

[0021] Feature Analysis Unit: Comprehensively analyzes the extracted terrain features and crop features, uses deep learning algorithms to identify the maturity of crops, and combines real-time meteorological data to generate comprehensive environmental feature data, including terrain models, obstacle locations, crop health status and maturity assessment.

[0022] Optionally, the point cloud processing unit includes:

[0023] Data input and preprocessing: Receive the point cloud data generated by the lidar, and each point cloud data point is represented as: ,in, , , Respectively indicate points The spatial coordinates of the point cloud data are normalized so that the coordinate range of all points is between;

[0024] Feature extraction: Use the point cloud network algorithm to extract features from each point, and first use a multi-layer perceptron to perform nonlinear transformation on the coordinates of each point;

[0025] Global feature aggregation: The local features of all points are aggregated into a global feature vector through the maximum pooling operation;

[0026] Feature classification: connect the global feature vector to the local feature vector of each point to form a new feature representation ; And classify each point through the classifier to identify whether each point is a terrain feature point or an obstacle point;

[0027] Obstacle position and shape recognition: According to the classification results, all points classified as obstacles are clustered, and the center position and shape characteristics of each obstacle are calculated. The center position of the obstacle is obtained by calculating the average coordinates of the obstacle points. The formula is: ,in, represents the center position of the obstacle, M is the number of obstacle points, , , Respectively represent the spatial coordinates of the j-th obstacle point;

[0028] Generate terrain feature data: Triangulate or grid the terrain feature points to generate a terrain model, and combine the location and shape data of obstacles to generate preliminary terrain feature data.

[0029] Optionally, the feature extraction unit specifically includes:

[0030] Terrain height extraction: Extract each point from point cloud data Height information , calculate the terrain height distribution in the operation area, and generate the height matrix H by sampling the height of each point, which is expressed as: , where H is the terrain height matrix and N is the total number of points in the point cloud data;

[0031] Slope information extraction: Use the height matrix H to calculate the slope information of the terrain. For each point , its slope It can be calculated by the height difference of adjacent points, the formula is: ,in, and Represent the rate of change of height in the x and y directions respectively;

[0032] Infrared data processing: Receives thermal imaging data from infrared sensor units , the infrared radiation value of each point is expressed as , by analyzing the thermal imaging data, the thermal radiation characteristics of the crops are extracted and the thermal radiation matrix T is generated, which is expressed as: , where T is the thermal radiation matrix and N is the total number of points;

[0033] Multispectral image data processing: Receive multispectral image data from the multispectral camera unit, extract the reflectance information of each band, and generate a multispectral feature matrix M, which is expressed as: ,in, Indicate point The multispectral feature vector of includes the reflectance values ​​of different bands;

[0034] Crop health status extraction: Combine the thermal radiation matrix T and the multispectral feature matrix M, and use the normalized difference vegetation index to calculate the crop health status. , its difference vegetation index value Calculated by the following formula: ,in, and Respectively indicate points Reflectance values ​​in the near-infrared and red bands;

[0035] Crop growth feature extraction: By analyzing the reflectance information of each band in the multispectral image data, combined with the thermal radiation characteristics and differential vegetation index values, the growth characteristics of crops, including leaf color, plant density and growth status, are extracted.

[0036] Optionally, the feature analysis unit specifically includes:

[0037] Feature input and preprocessing: receiving terrain feature data extracted from the point cloud processing unit and crop characteristics data , and normalize the input data;

[0038] Deep learning model construction: A feature analysis model is constructed using a deep learning algorithm based on a convolutional neural network. and crop characteristics data Combined into a comprehensive feature vector ;

[0039] Feature extraction and classification: Comprehensive feature vectors are extracted through deep learning models Perform multi-layer convolution operations to extract high-level feature representations;

[0040] Maturity recognition: After the feature extraction layer, a fully connected layer is added to classify the extracted high-level features and identify the maturity of the crop;

[0041] Meteorological data integration: Receive real-time meteorological data, normalize the meteorological data, and combine it with the extracted feature data to form a comprehensive environmental feature vector ;

[0042] Generate comprehensive environmental feature data: Comprehensive environmental feature vector Input into the feature analysis model, analyze the environmental features through the multi-layer perceptron, and generate comprehensive environmental feature data , including terrain models, crop health and maturity assessments, and real-time weather information.

[0043] Optionally, the nighttime intelligent navigation module includes a data receiving unit, an image processing unit, a feature fusion unit and a map generating unit; wherein:

[0044] Data receiving unit: used to receive environmental feature data provided by the data processing and analysis module, including terrain feature data, obstacle detection data and crop growth status data;

[0045] Image processing unit: receives infrared sensor data and multispectral image data from the enhanced environmental perception module and performs preprocessing, including denoising, contrast enhancement, and image correction;

[0046] Feature fusion unit: fuses the infrared image and multispectral image data provided by the image processing unit with the terrain feature data and obstacle detection data provided by the data receiving unit. The fusion method includes feature splicing and multimodal fusion technology to generate a comprehensive feature vector , the formula is: ,in, is the infrared image data, is the multispectral image data, is the terrain feature data, Obstacle detection data;

[0047] Map generation unit: using the fused comprehensive feature vector , a high-precision night operation map is constructed through a convolutional neural network. The convolutional neural network extracts high-level features through multi-layer convolution and pooling operations to generate a high-precision operation map , the specific steps include:

[0048] Step 1: Comprehensive feature vector Perform initial convolution operations to extract low-level features , the formula is: ,in, is the initial convolution kernel weight, is the bias of the initial convolutional layer;

[0049] Step 2: Perform multi-layer convolution and pooling operations on low-level features to extract high-level features , the formula is: ,in, For the The convolution kernel weights of the layer, For the Bias of the layer;

[0050] Step 3: After the last convolution operation, a high-precision nighttime operation map is generated , the formula is: ,in, is the weight of the fully connected layer, is the bias of the fully connected layer, is the normalized exponential function, and L is the last layer of the convolutional layer.

[0051] Optionally, the optimal path planning module includes a data receiving unit, a path calculation unit, a path optimization unit and a path generation unit; wherein:

[0052] Data receiving unit: used to receive environmental feature data and night operation maps generated by the dynamic environment adaptation module and the night intelligent navigation module;

[0053] Path calculation unit: After receiving environmental feature data and night operation map, it uses Dijkstra algorithm or A* algorithm to perform preliminary path calculation on the operation area and generate multiple candidate paths. The path calculation is based on the following formula: ,in, represents the path distance from the starting point s to the end point t, is the weight of the ith edge on the path, and n is the number of edges on the path;

[0054] Path optimization unit: Optimize the candidate paths generated by the path calculation unit, taking into account the factors of path smoothness, crop density and health, and terrain slope. Path optimization is achieved through an optimization method based on genetic algorithm. The specific optimization objective function is:

[0055] ,in, is the optimization target value of path P, is the distance on the path, for crop density and health, is the slope of the terrain, , , is the weight coefficient, n, m, and p are the distance, number of crops, and slope on the path, respectively;

[0056] Path generation unit: processes the optimized path to generate the optimal navigation path. The path generation unit uses the Bezier curve to smooth the path. The Bezier curve is generated by the following formula: ,in, represents the Bezier curve, t is the parameter, is the control point, is the binomial coefficient.

[0057] Optionally, the dynamic environment adaptation module includes a data receiving unit, an environment evaluation unit, a speed calculation unit, a path adjustment unit and an instruction generation unit; wherein:

[0058] Data receiving unit: used to receive navigation path data from the optimal path planning module and real-time environmental feature data, including meteorological data, terrain features and obstacle locations;

[0059] Environmental assessment unit: evaluates the received environmental characteristic data in real time, and specifically uses the following formula to evaluate the impact of meteorological changes on operations: ,in, represents the meteorological impact assessment value, T is temperature, H is humidity, P is air pressure, and f is the meteorological impact assessment function;

[0060] Speed ​​calculation unit: Based on the evaluated environmental feature data and navigation path, the optimal operating speed of the highland barley harvester is calculated. The operating speed is V and the position point on the path is , the calculation formula is: ,in, is the distance between adjacent position points on the path, and t is the time interval; path adjustment unit: combines the environmental assessment results and the optimal operating speed to adjust the navigation path in real time. The adjusted path is expressed as , the adjustment formula is: ,in, is the path adjustment amount calculated based on the environmental characteristic data;

[0061] Instruction generation unit: According to the adjusted path and the optimal operating speed V, generate a real-time adjustment instruction I, which includes a new driving direction and speed. The instruction is expressed as: , where I is the real-time adjustment instruction, The adjusted driving direction.

[0062] Optionally, the execution control module includes a path receiving unit, a speed control unit, a direction control unit and a response unit; wherein:

[0063] Path receiving unit: used for receiving the navigation path generated by the optimal path planning module and the real-time adjustment instruction I generated by the dynamic environment adaptation module;

[0064] Speed ​​control unit: Calculates the driving speed of the barley harvester based on the received navigation path , each path point is calculated by the following formula Target speed: ,in, For waypoints and The distance between them, t is the set time interval;

[0065] Direction control unit: Calculates the driving direction of the barley harvester based on the received navigation path , each path point is calculated by the following formula Target direction: ,in, , Waypoints The coordinates of represents the inverse tangent function;

[0066] Response unit: Use the real-time adjustment instruction I generated by the dynamic environment adaptation module to timely adjust the driving speed and direction of the highland barley harvester. The real-time adjustment instruction includes the new speed and direction , the final driving speed and direction are determined by the following formula: ; ;in, is the adjusted driving speed, The adjusted driving direction.

[0067] Beneficial effects of the present invention:

[0068] The present invention integrates multiple sensors and advanced image processing technology. The system can accurately identify and analyze the working environment, including terrain, obstacles and crop growth conditions, at night or in low light conditions. This not only optimizes the accuracy of the navigation path, but also realizes stable operation under various climate and lighting changes, greatly reducing operation interruptions or efficiency reductions caused by environmental factors.

[0069] The present invention, through the efficient collaboration of the dynamic environment adaptation module and the optimal path planning module, can adjust the driving speed and path of the harvester in real time to adapt to sudden terrain and weather changes. This real-time path adjustment and speed control ensures the harvester's rapid response capability when encountering unexpected obstacles or complex terrain, thereby ensuring the continuity and high efficiency of the operation. In addition, the system's comprehensive environmental perception and intelligent decision-making functions significantly improve the automation level of the operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0071] Figure 1 A schematic diagram of an automatic navigation system for a highland barley harvester according to an embodiment of the present invention;

[0072] Figure 2 Schematic diagram of a data processing and analysis module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0073] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. At the same time, it is explained here that in order to make the embodiments more detailed, the following embodiments are the best and preferred embodiments, and those skilled in the art may also adopt other alternatives to implement some known technologies; and the accompanying drawings are only for more specific description of the embodiments, and are not intended to specifically limit the present invention.

[0074] It should be noted that the references to "one embodiment", "embodiment", "exemplary embodiments", "some embodiments" and the like in the specification indicate that the embodiments described may include specific features, structures or characteristics, but not every embodiment may include the specific features, structures or characteristics. In addition, when a specific feature, structure or characteristic is described in conjunction with an embodiment, it should be within the knowledge of a person skilled in the art to implement such feature, structure or characteristic in conjunction with other embodiments (whether or not explicitly described).

[0075] In general, a term can be understood, at least in part, from its use in context. For example, depending, at least in part, on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in the singular sense, or can be used to describe a combination of features, structures, or characteristics in the plural sense. Additionally, the term "based on" can be understood as not necessarily intended to convey an exclusive set of factors, but can instead, depending, at least in part, on the context, allow for the presence of other factors that are not necessarily explicitly described.

[0076] like Figure 1-Figure 2 As shown, an automatic navigation system for a highland barley harvester driven by environmental perception includes an enhanced environmental perception module, a data processing and analysis module, a nighttime intelligent navigation module, an optimal path planning module, a dynamic environmental adaptation module, and an execution control module; wherein:

[0077] Enhanced environmental perception module: Integrates lidar, infrared sensor, ultrasonic sensor and multispectral camera to collect real-time information on terrain, weather and crop growth in the working environment under various lighting conditions. Lidar is used to measure terrain height and slope, infrared sensor and multispectral camera are used for crop and obstacle detection at night and in low light conditions, and ultrasonic sensor is used for close-range obstacle detection.

[0078] Data processing and analysis module: Receives multi-source data from the enhanced environmental perception module, processes and analyzes the point cloud data using the PointNet algorithm, identifies obstacles, terrain features, and crop maturity in the operating area, and generates environmental feature data;

[0079] Nighttime intelligent navigation module: Generates high-precision nighttime operation maps using environmental feature data provided by the data processing and analysis module;

[0080] Optimal path planning module: Based on the environmental feature data and night operation map generated by the dynamic environment adaptation module and the night intelligent navigation module, it calculates and generates the optimal navigation path to avoid obstacles and optimize the driving route, thereby improving harvesting efficiency and operation accuracy;

[0081] Dynamic environment adaptation module: obtains the navigation path generated from the optimal path planning module, evaluates the environmental characteristic data in real time, and generates real-time adjustment instructions based on the navigation path and the operating speed of the highland barley harvester to cope with sudden weather changes and dynamic terrain changes, ensuring the stability and efficient operation of the harvester;

[0082] Execution control module: receives the navigation path generated by the optimal path planning module, controls the speed and direction of the harvester, and uses the real-time adjustment instructions of the dynamic environment adaptation module to ensure that the harvester responds in time when encountering sudden obstacles.

[0083] The enhanced environment perception module includes a laser radar unit, an infrared sensor unit, an ultrasonic sensor unit and a multi-spectral camera unit; wherein:

[0084] LiDAR unit: used to emit laser beams under various lighting conditions and receive reflected signals, and to obtain terrain data by calculating the flight time of the laser beam to measure the height and slope of the terrain;

[0085] Infrared sensor unit: used to detect thermal radiation signals in the working environment at night and in low light conditions. It collects thermal imaging data in real time through infrared sensors to identify crops and obstacles to supplement environmental information during the day and at night.

[0086] Ultrasonic sensor unit: used to transmit ultrasonic signals and receive reflected echoes under various lighting conditions, measure the position and distance of close-range obstacles by calculating the propagation time of the echoes, and generate obstacle detection data in real time;

[0087] Multispectral camera unit: used to capture spectral images of different bands under various lighting conditions, and to detect the growth and health status of crops in real time by analyzing the reflected light information in the multispectral images, and to generate data on the growth and health status of crops; through the collaborative work of the above units, the enhanced environmental perception module can collect terrain, meteorological and crop growth status information in the working environment in real time under various lighting conditions, providing a comprehensive and accurate environmental data foundation for the system.

[0088] The data processing and analysis module includes a data receiving unit, a point cloud processing unit, a feature extraction unit and a feature analysis unit; wherein:

[0089] Data receiving unit: used to receive multi-source data from the enhanced environmental perception module, including lidar data, infrared data, ultrasonic data and multispectral image data, and perform data preprocessing to ensure a unified data format;

[0090] Point cloud processing unit: Use the point cloud network (PointNet) algorithm to process the point cloud data generated by the lidar, extract and classify the features of each point in the point cloud data, identify the location and shape of obstacles in the working area, and generate preliminary terrain feature data;

[0091] Feature extraction unit: extracts terrain height and slope information from the terrain feature data generated by the point cloud processing unit, and combines infrared data and multispectral image data to extract crop health and growth characteristics, including leaf color and plant density parameters;

[0092] Feature analysis unit: conducts a comprehensive analysis of the extracted terrain features and crop features, uses deep learning algorithms to identify the maturity of crops, and combines real-time meteorological data to generate comprehensive environmental feature data, including terrain models, obstacle locations, crop health status and maturity assessments; through the collaborative work of the above units, the data processing and analysis module can efficiently and accurately process and analyze multi-source data from the enhanced environmental perception module, generate comprehensive environmental feature data within the operating area, and provide key support for the system's navigation and operating strategies.

[0093] The point cloud processing unit includes:

[0094] Data input and preprocessing: Receive the point cloud data generated by the lidar, and each point cloud data point is represented as: ,in, , , Respectively indicate points The spatial coordinates of the point cloud data are normalized so that the coordinate range of all points is Improve the stability of subsequent calculations;

[0095] Feature extraction: Use the PointNet algorithm to extract features from each point, and first use a multi-layer perceptron (MLP) to perform nonlinear transformation on the coordinates of each point;

[0096] Global feature aggregation: The local features of all points are aggregated into a global feature vector through the maximum pooling operation;

[0097] Feature classification: connect the global feature vector to the local feature vector of each point to form a new feature representation , the formula is: ,in is a local feature, g is a global feature vector; and each point is classified by a classifier to identify whether each point is a terrain feature point or an obstacle point;

[0098] Obstacle position and shape recognition: According to the classification results, all points classified as obstacles are clustered, and the center position and shape characteristics of each obstacle are calculated. The center position of the obstacle is obtained by calculating the average coordinates of the obstacle points. The formula is: ,in, represents the center position of the obstacle, M is the number of obstacle points, , , Respectively represent the spatial coordinates of the j-th obstacle point;

[0099] Generate terrain feature data: Triangulate or grid the terrain feature points to generate a high-precision terrain model, and combine the location and shape data of obstacles to generate preliminary terrain feature data.

[0100] The feature extraction unit specifically includes:

[0101] Terrain height extraction: Extract each point from point cloud data Height information , calculate the terrain height distribution in the operation area, and generate the height matrix H by sampling the height of each point, which is expressed as: , where H is the terrain height matrix and N is the total number of points in the point cloud data;

[0102] Slope information extraction: Use the height matrix H to calculate the slope information of the terrain. For each point , its slope It can be calculated by the height difference of adjacent points, the formula is: ,in, and Represent the rate of change of height in the x and y directions respectively;

[0103] Infrared data processing: Receives thermal imaging data from infrared sensor units , the infrared radiation value of each point is expressed as , by analyzing the thermal imaging data, the thermal radiation characteristics of the crops are extracted and the thermal radiation matrix T is generated, which is expressed as: , where T is the thermal radiation matrix and N is the total number of points;

[0104] Multispectral image data processing: Receive multispectral image data from the multispectral camera unit, extract the reflectance information of each band, and generate a multispectral feature matrix M, which is expressed as: ,in, Indicate point The multispectral feature vector of includes the reflectance values ​​of different bands;

[0105] Crop health status extraction: Combine the thermal radiation matrix T and the multispectral feature matrix M, and use the Normalized Difference Vegetation Index (NDVI) to calculate the crop health status. , its difference vegetation index value Calculated by the following formula: ,in, and Respectively indicate points Reflectance values ​​in the near-infrared and red bands;

[0106] Crop growth feature extraction: By analyzing the reflectance information of each band in the multispectral image data, combined with the thermal radiation characteristics and differential vegetation index values, the growth characteristics of crops, including leaf color, plant density and growth status, are extracted; through the above steps, the point cloud processing unit can extract terrain height and slope information from the generated terrain feature data, and combine infrared data and multispectral image data to accurately extract the health status and growth characteristics of crops, providing key operating environment data support for highland barley harvesters.

[0107] The feature analysis unit specifically includes:

[0108] Feature input and preprocessing: receiving terrain feature data extracted from the point cloud processing unit and crop characteristics data , and normalize the input data to improve the stability of subsequent calculations;

[0109] Deep learning model construction: A feature analysis model is constructed using a deep learning algorithm based on a convolutional neural network. and crop characteristics data Combined into a comprehensive feature vector ;

[0110] Feature extraction and classification: Comprehensive feature vectors are extracted through deep learning models Perform multi-layer convolution operations to extract high-level feature representations. Let W be the convolution kernel weight and b be the bias. Then the output of the lth layer is , calculated by the following formula: , where ReLU represents the activation function, l represents the number of network layers, and * represents the convolution operation;

[0111] Maturity recognition: After the feature extraction layer, a fully connected layer is added to classify the extracted high-level features and identify the maturity of the crops. is the weight of the fully connected layer, is biased, then the maturity classification result Calculated by the following formula: ,in, represents the normalized exponential function, L represents the last layer of the network;

[0112] Meteorological data integration: Receive real-time meteorological data, normalize the meteorological data, and combine it with the extracted feature data to form a comprehensive environmental feature vector ;

[0113] Generate comprehensive environmental feature data: Comprehensive environmental feature vector Input into the feature analysis model, analyze the environmental features through the multi-layer perceptron (MLP) to generate comprehensive environmental feature data , including terrain models, crop health and maturity assessments, and real-time weather information, assuming is the weight of the multi-layer perceptron, is the bias, then the environmental characteristic data Calculated by the following formula: ; Through the above steps, the feature analysis unit can conduct a comprehensive analysis of the extracted terrain features and crop features, use the self-convolutional neural network (CNN) to identify the maturity of crops, and generate comprehensive environmental feature data in combination with real-time meteorological data.

[0114] The nighttime intelligent navigation module includes a data receiving unit, an image processing unit, a feature fusion unit and a map generation unit; wherein:

[0115] Data receiving unit: used to receive environmental feature data provided by the data processing and analysis module, including terrain feature data, obstacle detection data and crop growth status data. This unit ensures the real-time and accuracy of all data at night and under low light conditions;

[0116] Image processing unit: Receives infrared sensor data and multispectral image data from the enhanced environmental perception module and performs preprocessing, including denoising, contrast enhancement, and image correction, to ensure high-quality image information at night and in low-light conditions;

[0117] Feature fusion unit: fuses the infrared image and multispectral image data provided by the image processing unit with the terrain feature data and obstacle detection data provided by the data receiving unit. The fusion method includes feature splicing and multimodal fusion technology to generate a comprehensive feature vector , the formula is: ,in, is the infrared image data, is the multispectral image data, is the terrain feature data, Obstacle detection data;

[0118] Map generation unit: using the fused comprehensive feature vector , a high-precision night operation map is constructed through a convolutional neural network (CNN). The convolutional neural network extracts high-level features through multi-layer convolution and pooling operations to generate a high-precision operation map , the specific steps include:

[0119] Step 1: Comprehensive feature vector Perform initial convolution operations to extract low-level features , the formula is: ,in, is the initial convolution kernel weight, is the bias of the initial convolutional layer;

[0120] Step 2: Perform multi-layer convolution and pooling operations on low-level features to extract high-level features , the formula is: ,in, For the The convolution kernel weights of the layer, For the Bias of the layer;

[0121] Step 3: After the last convolution operation, a high-precision nighttime operation map is generated , the formula is: ,in, is the weight of the fully connected layer, is the bias of the fully connected layer, is a normalized exponential function, and L is the last layer of the convolutional layer. Through the collaborative work of the above units, the night intelligent navigation module can use the environmental feature data provided by the data processing and analysis module and the infrared and multi-spectral image data provided by the enhanced environmental perception module to generate a high-precision night operation map, providing accurate and reliable support for the navigation of the barley harvester at night and in low light conditions.

[0122] The optimal path planning module includes a data receiving unit, a path calculation unit, a path optimization unit and a path generation unit; wherein:

[0123] Data receiving unit: used to receive environmental feature data and night operation maps generated by the dynamic environment adaptation module and the night intelligent navigation module. The environmental feature data includes terrain features, obstacle locations, crop health status and real-time meteorological information;

[0124] Path calculation unit: After receiving environmental feature data and night operation map, it uses Dijkstra algorithm or A* algorithm to perform preliminary path calculation on the operation area and generate multiple candidate paths. The path calculation is based on the following formula: ,in, represents the path distance from the starting point s to the end point t, is the weight of the ith edge on the path, and n is the number of edges on the path;

[0125] Path optimization unit: Optimize the candidate paths generated by the path calculation unit, taking into account the factors of path smoothness, crop density and health, and terrain slope. Path optimization is achieved through an optimization method based on a genetic algorithm (GA). The specific optimization objective function is:

[0126] ,in, is the optimization target value of path P, is the distance on the path, for crop density and health, is the slope of the terrain, , , is the weight coefficient, n, m, and p are the distance, number of crops, and slope on the path, respectively;

[0127] Path generation unit: Process the optimized path to generate the optimal navigation path. The path generation unit uses the Bezier Curve to smooth the path to ensure the feasibility and stability of the path. The Bezier Curve is generated by the following formula: ,in, represents the Bezier curve, t is the parameter, is the control point, is the binomial coefficient; through the collaborative work of the above units, the optimal path planning module can calculate and generate the optimal navigation path based on the environmental feature data and night operation map generated by the dynamic environment adaptation module and the night intelligent navigation module, so as to provide an efficient and accurate operation route for the highland barley harvester.

[0128] The dynamic environment adaptation module includes a data receiving unit, an environment evaluation unit, a speed calculation unit, a path adjustment unit and an instruction generation unit; wherein:

[0129] Data receiving unit: used to receive navigation path data from the optimal path planning module and real-time environmental feature data, including meteorological data, terrain features and obstacle locations;

[0130] Environmental assessment unit: evaluates the received environmental characteristic data in real time, and specifically uses the following formula to evaluate the impact of meteorological changes on operations: ,in, represents the meteorological impact assessment value, T is temperature, H is humidity, P is air pressure, and f is the meteorological impact assessment function;

[0131] Speed ​​calculation unit: Based on the evaluated environmental feature data and navigation path, the optimal operating speed of the highland barley harvester is calculated. The operating speed is V and the position point on the path is , the calculation formula is: ,in, is the distance between adjacent position points on the path, and t is the time interval; path adjustment unit: Combined with the environmental assessment results and the optimal operating speed, the navigation path is adjusted in real time to ensure the stability and efficiency of the operation. The adjusted path is expressed as , the adjustment formula is: ,in, is the path adjustment amount calculated based on the environmental characteristic data;

[0132] Instruction generation unit: According to the adjusted path and the optimal operating speed V, generate real-time adjustment instructions I, which include new driving direction and speed, to ensure that the highland barley harvester can respond to sudden weather changes and dynamic terrain changes in a timely manner. The instructions are expressed as: , where I is the real-time adjustment instruction, The adjusted driving direction; through the collaborative work of the above units, the dynamic environment adaptation module can obtain the generated navigation path from the optimal path planning module, evaluate the environmental characteristic data in real time, and generate real-time adjustment instructions based on the navigation path and the operating speed of the barley harvester to respond to sudden weather changes and dynamic terrain changes, ensuring the continuity and efficiency of the operation.

[0133] The execution control module includes a path receiving unit, a speed control unit, a direction control unit and a response unit; wherein:

[0134] Path receiving unit: used to receive the navigation path generated by the optimal path planning module and the real-time adjustment instruction I generated by the dynamic environment adaptation module. The path receiving unit ensures the integrity and accuracy of all received paths and instructions during the transmission process;

[0135] Speed ​​control unit: Calculates the driving speed of the barley harvester based on the received navigation path , each path point is calculated by the following formula Target speed: ,in, For waypoints and The distance between them, t is the set time interval;

[0136] Direction control unit: Calculates the driving direction of the barley harvester based on the received navigation path , each path point is calculated by the following formula Target direction: ,in, , Waypoints The coordinates of represents the inverse tangent function;

[0137] Response unit: Use the real-time adjustment instruction I generated by the dynamic environment adaptation module to timely adjust the driving speed and direction of the highland barley harvester. The real-time adjustment instruction includes the new speed and direction , the final driving speed and direction are determined by the following formula: ; ;in, is the adjusted driving speed, is the adjusted driving direction; through the collaborative work of the above units, the execution control module can control the driving speed and direction of the harvester after receiving the navigation path generated by the optimal path planning module, and at the same time use the real-time adjustment instructions of the dynamic environment adaptation module to ensure that the harvester responds in time when encountering sudden obstacles.

[0138] The present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention. In order to make the public have a thorough understanding of the present invention, specific details are described in detail in the following preferred embodiments of the present invention, but those skilled in the art can fully understand the present invention without the description of these details. In addition, in order to avoid unnecessary confusion about the essence of the present invention, well-known methods, processes, procedures, components and circuits are not described in detail.

[0139] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An automatic navigation system for a highland barley harvester driven by environmental perception, characterized in that: It includes an enhanced environmental perception module, a data processing and analysis module, a nighttime intelligent navigation module, an optimal path planning module, a dynamic environmental adaptation module, and an execution control module; among which: Enhanced environmental perception module: Integrates lidar, infrared sensors, ultrasonic sensors and multispectral cameras to collect real-time information on terrain, weather and crop growth conditions in the operating environment under various lighting conditions; Data processing and analysis module: Receives multi-source data from the enhanced environmental perception module, processes and analyzes the point cloud data using the point cloud network algorithm, identifies obstacles, terrain features, and crop maturity in the operating area, and generates environmental feature data; The data processing and analysis module includes a data receiving unit, a point cloud processing unit, a feature extraction unit and a feature analysis unit; wherein: Data receiving unit: used to receive multi-source data from the enhanced environmental perception module, including lidar data, infrared data, ultrasonic data and multispectral image data, and perform data preprocessing to ensure a unified data format; Point cloud processing unit: Use point cloud network algorithm to process the point cloud data generated by LiDAR, extract and classify the features of each point in the point cloud data, identify the position and shape of obstacles in the working area, and generate preliminary terrain feature data; Feature extraction unit: extracts terrain height and slope information from the terrain feature data generated by the point cloud processing unit, and combines infrared data and multispectral image data to extract crop health and growth characteristics, including leaf color and plant density parameters; Feature analysis unit: Comprehensively analyzes the extracted terrain features and crop features, uses deep learning algorithms to identify the maturity of crops, and combines real-time meteorological data to generate comprehensive environmental feature data, including terrain models, obstacle locations, crop health status, and maturity assessments; The point cloud processing unit comprises: Data input and preprocessing: Receive the point cloud data generated by the laser radar, each point cloud data point is represented by: P i =(x i ,y i ,z i ), where x i ,y i ,z i Respectively represent point P i The point cloud data is normalized so that the coordinate range of all points is between [-1, 1]; Feature extraction: Use the point cloud network algorithm to extract features from each point, and first use a multi-layer perceptron to perform nonlinear transformation on the coordinates of each point; Global feature aggregation: The local features of all points are aggregated into a global feature vector through the maximum pooling operation; Feature classification: Connect the global feature vector to the local feature vector of each point to form a new feature representation f i ; And classify each point through the classifier to identify whether each point is a terrain feature point or an obstacle point; Obstacle position and shape recognition: According to the classification results, all points classified as obstacles are clustered, and the center position and shape characteristics of each obstacle are calculated. The center position of the obstacle is obtained by calculating the average coordinates of the obstacle points. The formula is: Among them, C obs represents the center position of the obstacle, M is the number of obstacle points, x j ,y j ,z j Respectively represent the spatial coordinates of the j-th obstacle point; Generate terrain feature data: Triangulate or grid the terrain feature points to generate a terrain model, and combine the location and shape data of obstacles to generate preliminary terrain feature data; The feature extraction unit specifically includes: Terrain height extraction: Extract each point P from the point cloud data i Height information z i , calculate the terrain height distribution in the operation area, and generate the height matrix H by sampling the height of each point, which is expressed as: H = {z1, z2, ..., z N }, where H is the terrain height matrix and N is the total number of points in the point cloud data; Slope information extraction: Use the height matrix H to calculate the slope information of the terrain. For each point P i , whose slope s i It can be calculated by the height difference of adjacent points, the formula is: in, and Represent the rate of change of height in the x and y directions respectively; Infrared data processing: Receive thermal imaging data from the infrared sensor unit I IR , the infrared radiation value of each point is expressed as T i By analyzing the thermal imaging data, the thermal radiation characteristics of the crop are extracted and the thermal radiation matrix T is generated, which is expressed as: T = {T1, T2, ..., T N }, where T is the thermal radiation matrix and N is the total number of points; Multispectral image data processing: Receive multispectral image data from the multispectral camera unit, extract the reflectance information of each band, and generate a multispectral feature matrix M, which is expressed as: M = {M1, M2, ..., M N }, where M i Indicates point P i The multispectral feature vector of includes the reflectance values ​​of different bands; Crop health status extraction: Combine the thermal radiation matrix T and the multispectral feature matrix M, and use the normalized difference vegetation index to calculate the crop health status. For each point P i , its difference vegetation index value V i Calculated by the following formula: Among them, M NIR，i and M RED，i Respectively represent point P i Reflectance values ​​in the near-infrared and red bands; Crop growth feature extraction: By analyzing the reflectance information of each band in the multispectral image data, combined with the thermal radiation characteristics and differential vegetation index values, the crop growth characteristics, including leaf color, plant density and growth status, are extracted; The feature analysis unit specifically includes: Feature input and preprocessing: receiving terrain feature data D extracted from the point cloud processing unit t and crop characteristic data C p , and normalize the input data; Deep learning model construction: A feature analysis model is constructed using a deep learning algorithm based on a convolutional neural network. t and crop characteristic data C p Merge into comprehensive feature vector F i ; Feature extraction and classification: The comprehensive feature vector F is extracted through a deep learning model. i Perform multi-layer convolution operations to extract high-level feature representations; Maturity recognition: After the feature extraction layer, a fully connected layer is added to classify the extracted high-level features and identify the maturity of the crop; Meteorological data integration: Receive real-time meteorological data, normalize the meteorological data, and combine it with the extracted feature data to form a comprehensive environmental feature vector E i ; Generate comprehensive environmental feature data: The comprehensive environmental feature vector E i Input into the feature analysis model, analyze the environmental features through the multi-layer perceptron, and generate comprehensive environmental feature data E f , including terrain models, crop health and maturity assessments, and real-time weather information; Nighttime intelligent navigation module: Generates high-precision nighttime operation maps using environmental feature data provided by the data processing and analysis module; The nighttime intelligent navigation module includes a data receiving unit, an image processing unit, a feature fusion unit and a map generation unit; wherein: Data receiving unit: used to receive environmental feature data provided by the data processing and analysis module, including terrain feature data, obstacle detection data and crop growth status data; Image processing unit: receives infrared sensor data and multispectral image data from the enhanced environmental perception module and performs preprocessing, including denoising, contrast enhancement, and image correction; Feature fusion unit: fuses the infrared image and multispectral image data provided by the image processing unit with the terrain feature data and obstacle detection data provided by the data receiving unit. The fusion method includes feature splicing and multimodal fusion technology to generate a comprehensive feature vector F n , the formula is: F n =[I IR , I MS , D t , O d ], where I IR is the infrared image data, I MS is the multispectral image data, D t is the terrain feature data, O d Obstacle detection data; Map generation unit: using the fused comprehensive feature vector F n , a high-precision night operation map is constructed through a convolutional neural network. The convolutional neural network extracts high-level features through multi-layer convolution and pooling operations to generate a high-precision operation map M n , the specific steps include: Step 1: Comprehensive feature vector F n Perform the initial convolution operation to extract the low-level feature O1. The formula is: O1 = ReLU (W1*F n +b1), where W1 is the initial convolution kernel weight and b1 is the bias of the initial convolution layer; Step 2: Perform multi-layer convolution and pooling operations on low-level features to extract high-level features O l , the formula is: l =ReLU(W l *O l-1 +b l ), where W l is the convolution kernel weight of the lth layer, b l is the bias of the lth layer; Step 3: After the last convolution operation, a high-precision nighttime operation map M is generated. n , the formula is: M n =Softmax(W f ·O L +b f ), where W f is the weight of the fully connected layer, b f is the bias of the fully connected layer, Softmax is the normalized exponential function, and L is the last layer of the convolutional layer; Optimal path planning module: Calculates and generates the optimal navigation path based on the environmental feature data and night operation map generated by the dynamic environment adaptation module and the night intelligent navigation module; Dynamic environment adaptation module: obtains the navigation path generated from the optimal path planning module, evaluates environmental feature data in real time, and generates real-time adjustment instructions based on the navigation path and the operating speed of the highland barley harvester to cope with sudden weather changes and dynamic terrain changes; Execution control module: receives the navigation path generated by the optimal path planning module, controls the speed and direction of the harvester, and uses the real-time adjustment instructions of the dynamic environment adaptation module to ensure that the harvester responds in time when encountering sudden obstacles.

2. The environment-aware driven highland barley harvester automatic navigation system according to claim 1 is characterized in that: The enhanced environment perception module includes a laser radar unit, an infrared sensor unit, an ultrasonic sensor unit and a multi-spectral camera unit; wherein: LiDAR unit: used to emit laser beams under various lighting conditions and receive reflected signals, and to obtain terrain data by calculating the flight time of the laser beam to measure the height and slope of the terrain; Infrared sensor unit: used to detect thermal radiation signals in the working environment at night and in low light conditions. It collects thermal imaging data in real time through infrared sensors to identify crops and obstacles to supplement environmental information during the day and at night. Ultrasonic sensor unit: used to transmit ultrasonic signals and receive reflected echoes under various lighting conditions, measure the position and distance of close-range obstacles by calculating the propagation time of the echoes, and generate obstacle detection data in real time; Multispectral camera unit: used to capture spectral images of different bands under various lighting conditions, and to detect the growth and health of crops in real time by analyzing the reflected light information in the multispectral images, and to generate data on the growth and health of crops.

3. The environment-aware driven highland barley harvester automatic navigation system according to claim 1 is characterized in that: The optimal path planning module includes a data receiving unit, a path calculation unit, a path optimization unit and a path generation unit; wherein: Data receiving unit: used to receive environmental feature data and night operation maps generated by the dynamic environment adaptation module and the night intelligent navigation module; Path calculation unit: After receiving environmental feature data and night operation map, it uses Dijkstra algorithm or A* algorithm to perform preliminary path calculation on the operation area and generate multiple candidate paths. The path calculation is based on the following formula: Among them, d(s, t) represents the path distance from the starting point s to the end point t, w(e i ) is the weight of the i-th edge on the path, and n is the number of edges on the path; Path optimization unit: Optimize the candidate paths generated by the path calculation unit, taking into account the factors of path smoothness, crop density and health, and terrain slope. Path optimization is achieved through an optimization method based on genetic algorithm. The specific optimization objective function is: Among them, f(P) is the optimization target value of path P, d i is the distance on the path, c j is the density and health of the crop, s k is the slope of the terrain, α, β, γ are weight coefficients, n, m, p are the distance, number of crops and slope on the path respectively; Path generation unit: processes the optimized path to generate the optimal navigation path. The path generation unit uses the Bezier curve to smooth the path. The Bezier curve is generated by the following formula: Where B(t) represents the Bezier curve, t is the parameter, P i is the control point, is the binomial coefficient.

4. The environment-aware driven highland barley harvester automatic navigation system according to claim 1 is characterized in that: The dynamic environment adaptation module includes a data receiving unit, an environment evaluation unit, a speed calculation unit, a path adjustment unit and an instruction generation unit; wherein: Data receiving unit: used to receive navigation path data from the optimal path planning module and real-time environmental feature data, including meteorological data, terrain features and obstacle locations; Environmental assessment unit: real-time assessment of received environmental characteristic data, specifically using the following formula to assess the impact of meteorological changes on operations: E m =f(T, H, P), where E m represents the meteorological impact assessment value, T is temperature, H is humidity, P is air pressure, and f is the meteorological impact assessment function; Speed ​​calculation unit: Based on the evaluated environmental feature data and navigation path, calculate the optimal operating speed of the highland barley harvester. Let the operating speed be V and the position point on the path be P. i , the calculation formula is: Among them, d(P i-1 , P i ) is the distance between adjacent points on the path, and t is the time interval; Path adjustment unit: Combines environmental assessment results and optimal operating speed to adjust the navigation path in real time. The adjusted path is represented by P i ′, the adjustment formula is: P i ′=P i +ΔP, where ΔP is the path adjustment amount calculated based on the environmental characteristic data; Instruction generation unit: According to the adjusted path P i ′ and the optimal operating speed V, generate a real-time adjustment instruction I, the instruction includes a new driving direction and speed, the instruction is expressed as: I = (D′, V), where I is the real-time adjustment instruction and D′ is the adjusted driving direction.

5. The environment-aware driven highland barley harvester automatic navigation system according to claim 1 is characterized in that: The execution control module includes a path receiving unit, a speed control unit, a direction control unit and a response unit; wherein: A path receiving unit: used for receiving the navigation path generated by the optimal path planning module and the real-time adjustment instruction I generated by the dynamic environment adaptation module; Speed ​​control unit: Calculates the driving speed V of the barley harvester based on the received navigation path cmd , each path point P is calculated by the following formula i Target speed: Among them, d(P i-1 , P i ) is the path point P i-1 and P i The distance between them, t is the set time interval; Direction control unit: Calculates the driving direction D of the barley harvester based on the received navigation path cmd , each path point P is calculated by the following formula i Target direction: Among them, x i ,y i The path points P i The coordinates of , arctan represents the inverse tangent function; Response unit: Use the real-time adjustment command I generated by the dynamic environment adaptation module to timely adjust the driving speed and direction of the highland barley harvester. Suppose the real-time adjustment command includes the new speed V adj and direction D adj , the final driving speed and direction are determined by the following formula: V final =V cmd +V adj ;D final =D cmd +D adj ; Among them, V final is the adjusted driving speed, D final The adjusted driving direction.

Images