Computer vision based harvester work path dynamic planning method and system

Abstract

The application provides a computer vision-based harvester operation path dynamic planning method and system, and belongs to the field of path planning. The method comprises the following steps: in the growth process of crops, abnormal meteorological data and growth stages are monitored to predict crop lodging, and predicted lodging information distribution is obtained; in the process of harvesting crops, a computer vision is used to identify regional images to obtain identified lodging information distribution; lodging credibility analysis is performed according to the predicted and identified lodging information distribution; the harvester operation path is planned in combination with the lodging credibility distribution, the lodging harvesting fitness is calculated, and the optimal operation path is dynamically optimized. The application solves the technical problems that the existing harvester operation path planning is unreasonable, the cutting table pushes the lodged crops, and the harvesting quality is poor, and achieves the technical effects that the harvester operation path is dynamically planned, the cutting table is prevented from pushing the lodged crops, and the harvesting quality of the lodged crops is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of path planning, in particular to a computer vision-based harvester operation path dynamic planning method and system. BACKGROUND

[0002] Crop harvesting is an important link in agricultural production, with the continuous improvement of agricultural mechanization level, the harvester plays an increasingly important role in crop harvesting. However, in actual agricultural production, affected by adverse weather conditions such as strong wind, heavy rain, etc., crop lodging often occurs, which brings serious challenges to mechanized harvesting.

[0003] The current harvester mainly relies on the preset fixed path for operation when harvesting the lodged crop, and this path planning method often cannot adapt to the actual distribution state of the lodged crop. Due to the lack of effective identification of the lodged state and dynamic adjustment of the path, the harvester is prone to push the lodged crop during operation, which not only affects the harvesting efficiency, but also causes the increase of grain loss, the decrease of harvesting quality and other adverse consequences, resulting in poor harvesting quality. SUMMARY

[0004] The present application provides a computer vision-based harvester operation path dynamic planning method and system to solve the technical problems of unreasonable harvester operation path planning in the prior art, which leads to the problem of pushing the lodged crop by the header and poor harvesting quality.

[0005] The technical solution of the present application to solve the above technical problems is as follows:

[0006] In a first aspect, the present application provides a computer vision-based harvester operation path dynamic planning method, comprising: during crop growth, monitoring and obtaining abnormal meteorological data and growth stages in the growth area, obtaining abnormal meteorological data sequence and growth stage information sequence, predicting crop lodging, and obtaining predicted lodging information distribution; during crop harvesting, collecting regional images in the growth area, identifying using computer vision, and obtaining identified lodging information distribution; performing lodging credibility analysis according to the predicted lodging information distribution and the identified lodging information distribution, and obtaining a lodging credibility distribution; planning a harvester operation path in the growth area, combining the lodging credibility distribution, calculating the lodging harvesting fitness of the operation path, performing path dynamic optimization, obtaining an optimal operation path as a path planning result, wherein the lodging harvesting fitness is calculated according to the operation direction in the operation path and the lodging direction in the lodging information.

[0007] In a second aspect, the present application provides a computer vision-based harvester operation path dynamic planning system, comprising: a lodging prediction module, configured to monitor and obtain abnormal weather data and growth stages in a growth area during crop growth, obtain an abnormal weather data sequence and a growth stage information sequence, perform crop lodging prediction, and obtain a predicted lodging information distribution; a visual recognition module, configured to collect regional images in the growth area during crop harvesting, perform recognition using computer vision, and obtain a recognized lodging information distribution; a credibility analysis module, configured to perform lodging credibility analysis based on the predicted lodging information distribution and the recognized lodging information distribution, and obtain a lodging credibility distribution; and a path planning module, configured to plan a harvester operation path in the growth area, combine the lodging credibility distribution, calculate a lodging harvesting fitness of the operation path, perform path dynamic optimization, obtain an optimal operation path as a path planning result, and wherein the lodging harvesting fitness is calculated based on an operation direction in the operation path and a lodging direction in the lodging information.

[0008] The present application has the following beneficial effects:

[0009] During crop growth, abnormal weather data and growth stages in a growth area are monitored and obtained, an abnormal weather data sequence and a growth stage information sequence are obtained, crop lodging prediction is performed, and a predicted lodging information distribution is obtained, so as to predict in advance the area, lodging proportion, and lodging direction that may be lodged before harvesting, thereby providing basic data support for subsequent path planning. During crop harvesting, regional images in the growth area are collected, computer vision is used for recognition, and a recognized lodging information distribution is obtained, so as to obtain the actual lodging state of the crops in real time, make up for possible deviations in weather prediction, and provide more accurate lodging information. Based on the predicted lodging information distribution and the recognized lodging information distribution, lodging credibility analysis is performed, and a lodging credibility distribution is obtained, so as to fuse and analyze the predicted information and the real-time recognized information, eliminate errors of a single information source, and improve the accuracy and reliability of the lodging information. In the growth area, a harvester operation path is planned, the lodging credibility distribution is combined, the lodging harvesting fitness of the operation path is calculated, path dynamic optimization is performed, an optimal operation path is obtained as a path planning result, and wherein the lodging harvesting fitness is calculated based on an operation direction in the operation path and a lodging direction in the lodging information. Through intelligent optimization of the path, the operation direction of the harvester and the lodging direction are perpendicular as much as possible, the header pushes the lodged crops, and the harvesting quality is improved.

[0010] By the technical scheme, dynamic planning and intelligent optimization of the harvesting path of the lodged crop are achieved. Compared with the fixed path planning manner in the prior art, the technical scheme can flexibly adjust the working path and direction according to the lodging conditions of different areas, maximally realizes perpendicularization of the harvesting direction and the lodging direction, and thus effectively avoids the pushing phenomenon of the cutter header to the lodged crop and improves the harvesting quality. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 A flowchart of a computer vision-based dynamic planning method of a harvester working path provided by the present application is shown in the figure.

[0012] Figure 2 A structure diagram of a computer vision-based dynamic planning system of a harvester working path provided by the present application is shown in the figure.

[0013] In the drawings, the components represented by the numbers are as follows:

[0014] The lodging prediction module 11, the visual recognition module 12, the credibility analysis module 13, and the path planning module 14. DETAILED DESCRIPTION

[0015] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0016] In the description of the present application, the terms “first” and “second” are only used for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with “first” and “second” can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of “a plurality of” is two or more, unless otherwise specifically limited.

[0017] In the description of the present application, the term "for example" is used to mean "serving as an example, instance, or illustration." Any embodiment described as "for example" in the present application is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is presented to enable any person skilled in the art to make and use the application. In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present application. It will be apparent, however, to one skilled in the art that the present application can be practiced without using these specific details. In other instances, well-known structures and processes are not elaborated upon in order to avoid unnecessary detail, which can obscure the description of the present application. Thus, the present application is not intended to be limited by the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

[0018] As shown in Embodiment I, Figure 1 The computer vision-based harvester operation path dynamic planning method provided by the embodiments of the present application comprises the following steps:

[0019] S1, during the growth of crops, abnormal weather data and growth stages in the growth area are monitored and acquired to obtain an abnormal weather data sequence and a growth stage information sequence, crop lodging is predicted, and a predicted lodging information distribution is obtained.

[0020] Specifically, by deploying weather monitoring equipment in the growth area, the weather data of the area is monitored and acquired in real time. When the monitored weather data meets the preset abnormal weather condition, the corresponding abnormal weather data is automatically recorded, and the growth stage information of the crops at that time is acquired. The abnormal weather data includes, but is not limited to, wind speed, wind direction, rainfall, temperature change, and other weather factors that may cause crop lodging; the abnormal weather condition can be set as a judgment standard that the wind speed exceeds a preset threshold value and the duration reaches a specific requirement. The growth stage information of the crops includes identification information of different development stages such as sowing period, tillering period, jointing period, heading period, grain filling period, and mature period.

[0021] By continuous monitoring, an abnormal weather data sequence and a corresponding growth stage information sequence arranged in time sequence are obtained. These sequence data reflect the historical record of the crops suffering from abnormal weather conditions and the growth state at the time of occurrence in the entire growth cycle. Subsequently, based on the obtained abnormal weather data sequence and growth stage information sequence, a pre-trained lodging prediction network is used for crop lodging prediction analysis, and factors such as the intensity of meteorological factors and the difference in lodging resistance of crop growth stages are considered to output a predicted lodging information distribution containing lodging direction and lodging proportion. The predicted lodging information distribution refers to the spatial distribution of the predicted lodging information corresponding to each grid in the entire region after the growth area is divided into multiple regional grids, which provides a lodging risk assessment basis for subsequent path planning.

[0022] S2, when harvesting crops, collect regional images in the growth area, identify using computer vision, and obtain identification lodging information distribution.

[0023] Specifically, when performing crop harvesting operations, first, real-time image acquisition of the growth area is performed by an image acquisition device to obtain regional images. The image acquisition device is preferably a high-resolution camera mounted on a drone, satellite, or other aircraft, which can obtain regional images covering the entire growth area from an aerial overhead perspective. The collected regional images contain actual growth state information of the crops, especially the real situation of crop lodging. To facilitate subsequent processing and analysis, the acquired regional images are divided into a plurality of grid images corresponding to the regional grid according to a predetermined grid division rule, and each grid image represents the crop state in a specific regional range.

[0024] Subsequently, computer vision technology based on deep learning is used to intelligently identify and process the grid images. Specifically, a lodging identification network is pre-constructed and trained, which uses a deep learning architecture such as a convolutional neural network (CNN) and is trained through a large number of sample grid images containing different lodging states. The lodging identification network can automatically extract crop features in the image, including the inclination angle of the crop stem, the distribution form of the leaf, the shadow feature, and other visual features, and determine the lodging state of the crop based on these features. For each grid image, the lodging identification network outputs corresponding identification lodging information, which includes two key parameters: lodging direction and lodging proportion. The lodging direction indicates the main direction of the crop's inclination, usually represented by an angle value; the lodging proportion represents the percentage of the crop in the grid area that has lodged.

[0025] By identifying and processing all grid images, a plurality of regional grid corresponding identification lodging information is obtained, and an identification lodging information distribution is formed, reflecting the actual lodging condition of the crops in the entire growth area at the time of harvesting.

[0026] S3, according to the predicted lodging information distribution and the identification lodging information distribution, performing lodging credibility analysis to obtain a lodging credibility distribution.

[0027] Specifically, since meteorological prediction and image recognition both have certain errors and uncertainties, by fusing the predicted lodging information distribution and the identification lodging information distribution, lodging credibility analysis is performed to improve the accuracy and reliability of the lodging information.

[0028] Firstly, the predicted lodging information and the identified lodging information corresponding to each grid of each region in the growth region are extracted to form a plurality of sets of lodging information pairs to be analyzed. Each set of lodging information pairs contains the predicted lodging information (from meteorological prediction) and the identified lodging information (from image recognition) of the same grid position. For each set of lodging information pairs, the similarity between the predicted lodging information and the identified lodging information is calculated as the lodging credibility of the grid position. The similarity calculation involves two dimensions, namely the lodging direction similarity and the lodging proportion similarity. The lodging direction similarity is determined by comparing the angle difference between the predicted lodging direction and the identified lodging direction. The smaller the angle difference, the higher the lodging direction similarity. The lodging proportion similarity is calculated by comparing the numerical difference between the predicted lodging proportion and the identified lodging proportion. The smaller the numerical difference, the higher the lodging proportion similarity. Subsequently, a weighted fusion method is used to comprehensively calculate the lodging direction similarity and the lodging proportion similarity to obtain the comprehensive lodging credibility of the grid. The higher the value of the lodging credibility, the more reliable the lodging information of the grid position; the lower the value of the lodging credibility, the greater the prediction deviation or identification error.

[0029] By performing the above lodging credibility calculation on all regional grids in the growth region, the lodging credibility corresponding to each grid is obtained, and a lodging credibility distribution is formed. The lodging credibility distribution provides a confidence assessment of the lodging information for subsequent path planning, enabling reasonable allocation of planning weights for each region in the path optimization process, giving priority to the lodging conditions of high credibility regions, thereby improving the accuracy and practicality of path planning.

[0030] S4, performing harvester operation path planning in the growth region, combining the lodging credibility distribution to calculate the lodging harvesting fitness of the operation path, performing path dynamic optimization to obtain an optimal operation path as the path planning result, wherein the lodging harvesting fitness is calculated according to the operation direction of the operation path and the lodging direction in the lodging information.

[0031] Specifically, after obtaining the lodging credibility distribution, the harvester operation path planning is performed in the growth region, and the operation path of the harvester is dynamically optimized in combination with the lodging credibility distribution, so that it forms an optimal geometric relationship with the lodging direction, thereby achieving efficient harvesting of the lodging crops.

[0032] Firstly, initial harvester operation path planning is performed in the growth area to generate a first operation path as an optimization starting point. The first operation path can be generated using traditional path planning algorithms, such as grid method, A* algorithm, etc., mainly considering basic constraint conditions such as area boundary, obstacle distribution, etc. Then, the lodging proportion and lodging direction of each area grid in the identified lodging information distribution are extracted to form the identified lodging proportion distribution and the identified lodging direction distribution, respectively. In combination with the aforementioned lodging reliability distribution, each area grid is assigned a corresponding planning weight. Specifically, a higher lodging proportion area is assigned a larger lodging proportion weight, and a higher lodging reliability area is assigned a larger reliability weight. The planning weight of each grid is calculated by combining the two, and a planning weight distribution is formed.

[0033] When calculating the lodging harvesting fitness of the operation path, the geometric relationship between the operation direction and the lodging direction is mainly considered. According to the principles of agricultural machinery, when the operation direction of the harvester forms a 90° angle with the lodging direction of the crops, the "pushing" phenomenon of the header to the lodging crops can be effectively avoided, thereby improving the harvesting quality and efficiency. Specifically, the operation direction of the harvester when passing through each area grid in the operation path is obtained, and the angle between the operation direction and the identified lodging direction in the corresponding grid is calculated. By comparing the similarity of the angle with the preset perpendicular angle (90°), the lodging harvesting angle similarity of each grid is obtained. Subsequently, the angle similarity of each grid is weighted calculated according to the planning weight distribution to obtain the overall harvesting angle fitness. At the same time, the ratio of the operation time corresponding to the operation path to the preset operation time is calculated to obtain the harvesting time fitness. Finally, the lodging harvesting fitness of the operation path is calculated by combining the harvesting angle fitness and the harvesting time fitness.

[0034] Based on the above fitness evaluation mechanism, a heuristic optimization algorithm (such as genetic algorithm, particle swarm algorithm, etc.) is used for dynamic optimization of the path. By continuously generating new candidate paths and calculating their lodging harvesting fitness, the path scheme is gradually improved in the optimization process. When the optimization algorithm converges, the operation path with the maximum lodging harvesting fitness is retained as the optimal operation path, which is output as the final path planning result. The optimal operation path can maximize the perpendicularity of the operation direction of the harvester and the lodging direction, while also considering the operation efficiency, providing path guidance for the mechanized harvesting of lodging crops and improving the harvesting quality.

[0035] Further, during the growth of the crops, abnormal weather data and growth stages in the growth area are monitored and obtained to obtain abnormal weather data sequence and growth stage information sequence, and the crops are predicted to obtain predicted lodging information distribution, including:

[0036] S11, during the growth of the crops, monitoring and obtaining weather data of multiple area grids in the growth area;

[0037] S12, record the abnormal weather data and the growth stage of the crop at the corresponding time when the monitored weather data meets the abnormal weather condition, obtain the abnormal weather data sequence and the growth stage information sequence of the plurality of regional grids;

[0038] S13, input the abnormal weather data sequence and the growth stage information sequence of the plurality of regional grids into the lodging prediction network respectively, output a plurality of predicted lodging information of the plurality of regional grids, and obtain a predicted lodging information distribution, wherein each predicted lodging information includes a lodging direction and a lodging proportion.

[0039] In a feasible implementation, first, the growth area is spatially gridded according to a preset grid division standard. The grid division standard is determined according to the crop planting density, the terrain characteristics and the monitoring accuracy requirement, for example, the growth area is divided into square grids with a side length of 10 m x 10 m to 50 m x 50 m, or irregular grid division is adopted according to the actual land shape. Based on the grid division result, a distributed weather monitoring network is constructed in the growth area. The distributed weather monitoring network is composed of a plurality of weather monitoring nodes, each weather monitoring node is responsible for covering a regional grid, and each regional grid is configured with an independent weather monitoring device. If the weather monitoring device resources are limited, a sparse deployment mode can be adopted, and the weather data of the grid not directly monitored is calculated by a spatial interpolation algorithm (such as Kriging interpolation, inverse distance weighted interpolation, etc.). The weather monitoring device of each weather monitoring node collects multi-dimensional weather parameters in the covered area in real time, including but not limited to wind speed, wind direction, air temperature, humidity, air pressure, rainfall, solar radiation intensity and other key weather factors affecting crop lodging. In order to ensure the timeliness and integrity of the data, the monitoring frequency can be dynamically adjusted according to the lodging sensitivity of the crop growth stage, for example, in the seedling stage and the mature stage where the crop has strong lodging resistance, a lower monitoring frequency (such as collecting once every hour) is adopted; in the key growth period (such as the rapid elongation stage of stem in the jointing stage and the stage of heavy head and light foot in the heading stage) where the crop is prone to lodging, the monitoring frequency is increased to collect once every 10-15 minutes, so as to ensure the timely capture of abnormal weather events.

[0040] In the process of meteorological continuous monitoring, multi-level abnormal weather condition determination standards are established. The standards comprehensively consider single meteorological parameter threshold and multi-parameter combination conditions, such as: wind speed exceeding 8 m / s and duration exceeding 30 minutes, or wind direction changing amplitude exceeding 90° within 2 hours, or short-time rainfall exceeding 20 mm, etc. When the monitoring data of any regional grid meets the above abnormal weather conditions, an abnormal event record is automatically triggered, recording the specific meteorological parameter values (including peak value, duration, change rate, etc.) of the triggering abnormal condition, and obtaining the growth stage information of the crops in the grid at the time of the abnormal event. Among them, the growth stage information of the crops is calculated in real time through the pre-established crop growth model, which can accurately divide the crop growth process into 11 main stages such as sowing period, seedling period, tillering period, jointing period, booting period, heading period, flowering period, grain filling period, milk stage, dough stage, and full maturity stage, each stage corresponding to different lodging resistance characteristic parameters. Through continuous monitoring and recording throughout the growth cycle, a time series of abnormal weather data sequence and corresponding growth stage information sequence are constructed for each regional grid, forming a complete meteorological-growth stage correlation database.

[0041] Subsequently, the pre-trained lodging prediction network is used to analyze the time series data of each regional grid. The lodging prediction network is based on a deep learning architecture, preferably using a time series neural network structure such as Long Short-Term Memory (LSTM) or Gated Recurrent Unit (GRU), which can effectively learn the time dependence of abnormal weather data and the influence of growth stage changes on lodging risk. For each regional grid, the abnormal weather data sequence and growth stage information sequence are processed by feature engineering and input into the lodging prediction network as feature vectors. The lodging prediction network analyzes the intensity of meteorological factors, duration of action, and current lodging resistance of crops through nonlinear transformation of multiple neurons, and outputs the predicted lodging information of the grid, including the lodging direction and the lodging proportion. Among them, the lodging direction is represented by an angle value (0°-360°), which is closely related to the historical dominant wind direction; the lodging proportion is represented by a percentage (0%-100%), reflecting the proportion of the area of crops expected to be lodged in the grid. The above prediction process is performed on all regional grids in the growth area to obtain the predicted lodging information corresponding to each grid, and finally the predicted lodging information distribution covering the entire growth area is obtained, providing lodging analysis based on historical meteorological data for subsequent harvesting path planning.

[0042] Further, the lodging prediction network is obtained based on machine learning training, and the training steps include:

[0043] S131, according to the crop lodging monitoring data, collect a sample abnormal weather data sequence set and a sample growth stage information sequence set, and collect the lodging direction and the lodging proportion of the crop growth under different sample abnormal weather data sequences and sample growth stage information sequences, and label to obtain a sample lodging information set;

[0044] S132, construct a lodging prediction network based on machine learning;

[0045] S133, use the sample abnormal weather data sequence set, the sample growth stage information sequence set and the sample lodging information set to supervise the training and test the accuracy of the lodging prediction network, and complete the training after meeting the requirements.

[0046] In a preferred embodiment, the lodging prediction network is obtained based on machine learning training. First, a training data set of the lodging prediction network is established through large-scale field tests and historical data collection. Specifically, first, historical records are extracted from crop lodging monitoring databases in different regions, different crop varieties and different planting years. These monitoring data cover various combinations of weather conditions and crop growth stages. Then, based on the above monitoring data, a sample abnormal weather data sequence set and a sample growth stage information sequence set are collected. The sample abnormal weather data sequence set contains records of abnormal weather events of different intensities, different durations and different frequencies, such as detailed parameters of extreme weather such as strong wind, heavy rain and hail; the sample growth stage information sequence set records the specific growth stage of the crop when the abnormal weather event occurs and its lodging resistance characteristic parameters. In order to establish the corresponding relationship between input and output, the actual lodging results of the crop under different sample conditions can be collected in various ways. For example, through field investigation, unmanned aerial vehicle aerial photography, satellite remote sensing and other means, the actual lodging direction and lodging proportion of the crop under the action of specific abnormal weather data sequences and growth stage information sequences are accurately measured and recorded. Among them, the lodging direction is determined by compass measurement or image analysis, accurate to degrees; the lodging proportion is obtained by statistics of the percentage of the number of lodging plants in the total number of plants in the quadrat. Through the above data collection process, the output label corresponding to the specific sample abnormal weather data sequence and sample growth stage information sequence, i.e. the sample lodging information (including the lodging direction and the lodging proportion), is established, forming a sample lodging information set, which provides sufficient supervised learning data for network training.

[0047] Subsequently, a lodging prediction network based on machine learning is constructed. The lodging prediction network adopts a multi-input and multi-output design idea, which can simultaneously process two types of time series inputs of abnormal meteorological data sequences and growth stage information sequences, and output two prediction results of lodging direction and lodging proportion. Among them, the network main structure preferably adopts a recurrent neural network architecture such as long short-term memory network (LSTM) or gated recurrent unit (GRU) to effectively capture the long-term dependence relationship in the time series data. Specifically, the lodging prediction network includes four main parts: feature extraction layer, time series modeling layer, feature fusion layer and output layer. Among them, the feature extraction layer is responsible for preprocessing and preliminary feature extraction of the original input data; the time series modeling layer learns the time evolution law of abnormal meteorological events and the growth stage change pattern through multiple layers of LSTM or GRU units; the feature fusion layer fuses the meteorological features and the growth stage features to form a comprehensive lodging risk feature representation; the output layer maps the features to the predicted values of the lodging direction (regression task) and the lodging proportion (regression task) through the full connection layer and the activation function. At the same time, the lodging prediction network can also introduce an attention mechanism, so that the lodging prediction network can automatically identify the most critical time points and feature dimensions for lodging prediction, and improve the prediction accuracy.

[0048] Subsequently, the constructed lodging prediction network is trained using a supervised learning method. The collected sample abnormal meteorological data sequence set and the sample growth stage information sequence set are used as training inputs, and the lodging direction and the lodging proportion in the sample lodging information set are used as training labels. To ensure the training effect, the total samples can be randomly divided into a training set, a validation set and a test set in a ratio of 7:2:1. The lodging prediction network is trained through the training set, and the training process adopts a gradient descent optimization algorithm. The network parameters are updated through back propagation to minimize the loss function between the predicted output and the true label. The loss function is designed as a weighted combination of the lodging direction prediction error and the lodging proportion prediction error to balance the importance of the two output tasks. At the same time, the performance of the lodging prediction network on the validation set is continuously monitored, including the mean absolute error (MAE) of the lodging direction prediction, the root mean square error (RMSE) of the lodging proportion prediction and other evaluation indicators. When the network performance indicators have not significantly improved for several consecutive training periods, the early stopping mechanism is triggered to prevent overfitting. After training is completed, the generalization ability of the network is evaluated on an independent test set. The preset performance requirements include: the lodging direction prediction error is less than 15°, the lodging proportion prediction error is less than 10%, and the comprehensive prediction accuracy is more than 85%. When the performance indicators of the network on the test set all meet the above requirements, it is confirmed that the training is completed, and the lodging prediction network that can be used for actual lodging prediction is obtained.

[0049] Further, when the crop is harvested, the regional image in the growth area is collected, computer vision is used for recognition, and the identified lodging information distribution is obtained, including:

[0050] S21, when crop harvesting is performed, collecting regional images in the growth area, and dividing into a plurality of grid images of a plurality of regional grids;

[0051] S22, respectively inputting the plurality of grid images into the lodging recognition network, identifying to obtain a plurality of identification lodging information as identification lodging information distribution, wherein the lodging recognition network is constructed based on computer vision and trained by using sample grid images and labeled sample identification lodging information.

[0052] In a preferred embodiment, when crop harvesting is performed, the image acquisition device is used to obtain comprehensive image data of the entire growth area, and the regional images in the growth area are collected. The image acquisition device preferably uses a UAV platform equipped with a high-resolution digital camera to perform aerial photography at a fixed height (usually 50-100 meters), ensuring that the obtained regional images have consistent spatial resolution and clarity. To ensure the integrity and overlap of the image coverage, the UAV follows a pre-set flight path for automatic aerial photography, maintaining an overlap rate of 60%-80% between adjacent images.

[0053] Meanwhile, satellite remote sensing, high-altitude balloons or fixed monitoring towers and other image acquisition methods can also be used as supplementary means according to actual needs. After obtaining the regional images, the large-scale images covering the entire growth area are accurately divided into a plurality of grid images corresponding to the regional grids according to the same grid division standard as in step S11. Each grid image represents the crop state in a specific geographical range, and the boundaries of the grid images strictly correspond to the boundaries of the regional grids monitored by the weather, ensuring the spatial consistency of subsequent data fusion.

[0054] Subsequently, the pre-trained lodging recognition network is used to analyze and identify each grid image. The plurality of grid images obtained by segmentation are sequentially input into the lodging recognition network to automatically extract and analyze the crop lodging feature information in the images. The lodging recognition network is constructed based on a convolutional neural network (CNN) architecture and is specifically designed for crop lodging recognition tasks. The network structure includes multiple convolutional layers, pooling layers and fully connected layers, which can automatically learn and extract visual features of crops from grid images, including stem inclination angle, leaf arrangement direction, shadow distribution pattern, texture variation characteristics and other image features closely related to lodging state.

[0055] For the special requirements of lodging recognition, the network is designed as a multi-task learning architecture, including a lodging direction recognition subnetwork and a lodging proportion estimation subnetwork, which simultaneously outputs two prediction results of lodging direction and lodging proportion. Among them, the lodging direction recognition subnetwork outputs the lodging direction represented by an angle value by analyzing the main direction of crop lodging in the image; the lodging proportion estimation subnetwork outputs the lodging proportion represented by a percentage by statistical analysis of the area proportion of the lodging area and the normal upright area in the image. In order to ensure the accuracy and reliability of the recognition network, the network uses a large number of sample grid images containing different lodging degrees, different lodging directions, different light conditions and different crop varieties for supervised learning training. The labeling information (sample lodging recognition information) of the training samples is obtained through artificial measurement, field investigation and other methods, ensuring the accuracy and consistency of the labeled data. Through the recognition processing of all grid images, multiple recognition lodging information corresponding to multiple regional grids is obtained. Each recognition lodging information contains the lodging direction and lodging proportion parameters of the grid, and these information is summarized to form the recognition lodging information distribution, which is used as the data basis reflecting the actual lodging condition of the crop at the harvesting moment. The recognition lodging information distribution and the aforementioned prediction lodging information distribution based on meteorological prediction complement each other, providing accurate lodging state information based on real-time images for subsequent lodging credibility analysis and path optimization.

[0056] Further, according to the prediction lodging information distribution and the recognition lodging information distribution, lodging credibility analysis is performed to obtain a lodging credibility distribution, including:

[0057] S31, extract the prediction lodging information and the recognition lodging information of each regional grid in the growth area, and obtain multiple groups of lodging information;

[0058] S32, calculate the similarity of the prediction lodging information and the recognition lodging information in each group of lodging information to obtain a lodging credibility, wherein the similarity of the lodging direction and the lodging proportion in the prediction lodging information and the recognition lodging information is calculated, and the lodging credibility is obtained;

[0059] S33, combine the lodging credibilities in multiple regional grids to obtain a lodging credibility distribution.

[0060] In a preferred embodiment, firstly, a spatial correspondence between the predicted lodging information distribution and the identified lodging information distribution is established. Based on the unified grid division system established in the foregoing step, the two types of lodging information corresponding to each regional grid in the growth area are extracted one by one. Specifically, for each regional grid, the predicted lodging information (including the predicted lodging direction and the predicted lodging proportion) of the grid is extracted from the predicted lodging information distribution, and the identified lodging information (including the identified lodging direction and the identified lodging proportion) of the grid is extracted from the identified lodging information distribution. Through the above extraction process, a plurality of groups of lodging information are obtained, each group of lodging information containing the predicted lodging information and the identified lodging information of the same grid position. These information groups constitute the basic data set for subsequent similarity calculation and credibility evaluation, ensuring one-to-one correspondence of spatial positions.

[0061] Subsequently, a multi-dimensional similarity calculation method is used to evaluate the consistency degree of the predicted results and the identified results in each group of lodging information. The calculation process includes lodging direction similarity calculation and lodging proportion similarity calculation. For the lodging direction similarity, firstly, the angle difference between the predicted lodging direction and the identified lodging direction is calculated; considering the periodicity of the angle, the minimum included angle between the two angles is selected as the difference value. For example, when the predicted lodging direction is 10° and the identified lodging direction is 350°, the actual angle difference is 20° instead of 340°; then, the angle difference is converted into the direction similarity: the smaller the angle difference, the closer the direction similarity to 1; when the angle difference reaches 180°, the direction similarity approaches 0.

[0062] For the lodging proportion similarity calculation, a normalized absolute error method is used to evaluate the consistency of the predicted proportion and the identified proportion; when the two proportion values are exactly equal, the proportion similarity is 1; when the difference gradually increases, the similarity decreases accordingly. Based on the similarity calculation results of the two dimensions, a weighted fusion method is used to calculate the comprehensive lodging credibility. This fusion process is realized by setting the weight coefficients of the direction similarity and the proportion similarity, and the sum of the weight coefficients is equal to 1. Under normal circumstances, since the lodging direction has a more critical impact on the harvesting path planning, a slightly higher weight is given to the lodging direction similarity. For example, under the standard configuration, the weight coefficient of the lodging direction similarity is set to 0.6, and the weight coefficient of the lodging proportion similarity is set to 0.4; in the application scenario where the accuracy of the lodging direction is extremely high, the weight ratio can be adjusted to 0.7 and 0.3; and in the application that emphasizes the evaluation of the lodging area, the balanced weight configuration of 0.5 and 0.5 can be used.

[0063] By executing the above credibility calculation process on all regional grids in the growth area, the lodging credibility corresponding to each grid is obtained. These discrete lodging credibilities are orderly organized according to the spatial positions of the grids to form a complete lodging credibility distribution. The formed lodging credibility distribution quantitatively evaluates the reliability of the lodging information of each regional grid, and provides a basis for subsequent operation path planning, ensuring that the path optimization process can fully consider the confidence level difference of the lodging information of different regions.

[0064] Further, the harvester operation path planning is performed in the growth area, the lodging harvesting fitness of the operation path is calculated in combination with the lodging credibility distribution, the path dynamic optimization is performed, and the optimal operation path is obtained, including:

[0065] S41, performing harvester operation path planning in the growth area to obtain a first operation path;

[0066] S42, extracting the lodging proportion and the lodging direction in the identified lodging information distribution to obtain an identified lodging proportion distribution and an identified lodging direction distribution, assigning a planning weight of a plurality of regional grids in combination with the lodging credibility distribution to obtain a planning weight distribution;

[0067] S43, calculating a first lodging harvesting fitness in combination with the identified lodging direction distribution and the planning weight distribution according to the operation direction of the harvester passing through a plurality of regional grids in the first operation path;

[0068] S44, continuing to randomly plan the operation path, performing path dynamic optimization, and retaining the operation path with the largest lodging harvesting fitness after optimization convergence to obtain the optimal operation path.

[0069] In a preferred embodiment, first, the harvester operation path planning is performed in the growth area to generate a first operation path as an optimization starting point. The first operation path planning process comprehensively considers basic constraint conditions such as field boundary, obstacle distribution, harvester turning radius, etc., and generates a feasible operation trajectory by using a traditional path planning algorithm. The first operation path can adopt various strategies, including a parallel straight line back-and-forth mode, a spiral centripetal mode, or a partition block operation mode, etc. The parallel straight line back-and-forth mode is preferably adopted, that is, the harvester performs back-and-forth operation along parallel lines, which has the advantages of high operation efficiency and few turning times. The path spacing is determined according to the working width of the harvester to ensure the completeness and non-overlapping of operation coverage. The generated first operation path is stored in the form of a coordinate sequence, recording the position coordinates and the traveling direction of the harvester at different times, and providing basic data for subsequent fitness calculation.

[0070] To reasonably reflect the importance of different regions in the path optimization process, a planning weight distribution mechanism based on multi-factor fusion is established. This mechanism considers three key factors: lodging proportion, lodging direction consistency, and lodging reliability. First, the lodging proportion and lodging direction of each region grid are extracted from the identified lodging information distribution to form the identified lodging proportion distribution and the identified lodging direction distribution. The identified lodging proportion distribution reflects the area proportion of the lodged crops in each grid, and the higher the lodging proportion, the greater the impact on path planning. The identified lodging direction distribution provides the dominant direction information of the lodged crops in each grid. Then, based on the identified lodging proportion distribution, the lodging proportion weight of each region grid is calculated. The grid with a higher lodging proportion is given a larger weight value, indicating that this region is more important in path planning. The grid with a lower lodging proportion or no lodging is given a smaller weight value. At the same time, the reliable weight of each region grid is assigned according to the lodging reliability distribution. The grid with a higher lodging reliability obtains a larger reliable weight, indicating that the lodging information of this region is more accurate and reliable. The grid with a lower lodging reliability obtains a smaller reliable weight, avoiding the negative impact of inaccurate information on path planning. Subsequently, the comprehensive planning weight of each region grid is calculated by fusing the lodging proportion weight and the reliable weight. This fusion process is realized by multiplication or weighted sum, ensuring that both the impact of lodging degree and the reliability of information are considered. The planning weights of all grids are organized according to the spatial location to form the planning weight distribution.

[0071] Subsequently, based on the specific trajectory of the first work path, in combination with the identified lodging direction distribution and the planned weight distribution, the lodging harvesting fitness of the path is calculated, aiming to quantitatively evaluate the matching degree of the work path and the lodging condition. Specifically, first, the work direction information of the harvester when passing through each area grid in the first work path is extracted. The work direction is determined by analyzing the travel trajectory of the harvester in the grid, and is expressed in the form of an angle value representing the forward direction of the harvester. For each passed grid, the included angle between the harvester work direction and the identified lodging direction of the grid is calculated. According to the principles of agricultural machinery, when the work direction and the lodging direction form a 90-degree angle, the harvesting effect is best, which can avoid the pushing phenomenon of the header to the lodged crops. By comparing the closeness of the actual included angle and the ideal perpendicular angle, the lodging harvesting angle similarity of each grid is calculated. Then, according to the planned weight distribution, the lodging harvesting angle similarity of each grid is weighted to obtain the overall harvesting angle fitness. The grid with larger weight contributes more to the overall fitness, ensuring that the lodging treatment effect of important areas is fully valued. In addition to the harvesting angle fitness, the total work time corresponding to the first work path is calculated, and compared with the preset ideal work time to obtain the harvesting time fitness. The time fitness reflects the work efficiency of the path, avoiding excessive sacrifice of work efficiency in pursuit of perfect angle matching. Then, the first lodging harvesting fitness of the first work path is calculated by comprehensively considering the harvesting angle fitness and the harvesting time fitness, which serves as the comprehensive evaluation index of the path scheme.

[0072] Next, based on the calculation result of the first lodging harvesting fitness, the path dynamic optimization process is started, and a work path scheme with higher lodging harvesting fitness is searched through iterative search. For example, heuristic optimization algorithms such as genetic algorithm, particle swarm optimization algorithm, and simulated annealing algorithm are used for path search. During the optimization process, new candidate work paths are continuously generated, which are generated from existing paths through random disturbance, crossover and mutation operations, or generated in a completely random manner. For each newly generated candidate path, the calculation process of step S43 is repeated to obtain the corresponding lodging harvesting fitness. By comparing the fitness performance of different paths, the path schemes with higher fitness are gradually selected and retained, and the candidate paths with poor performance are eliminated. The optimization process uses a convergence judgment mechanism, and when the optimal fitness value does not improve significantly for consecutive iterations, or the maximum number of iterations is reached, the optimization process is considered to have converged, and further path search is stopped. After the optimization converges, the work path with the maximum lodging harvesting fitness is selected from all evaluated candidate paths, and is determined as the optimal work path. This optimal path can effectively handle lodged crops (the work direction is as perpendicular to the lodging direction as possible) and maintain reasonable work efficiency, providing a path guidance scheme for actual harvesting operations.

[0073] Further, the lodging proportion and the lodging direction in the identified lodging information distribution are extracted to obtain an identified lodging proportion distribution and an identified lodging direction distribution, and the planning weight of the multiple region grids is distributed in combination with the lodging reliability distribution to obtain a planning weight distribution, including:

[0074] S421, the lodging proportion and the lodging direction of the multiple region grids in the identified lodging information distribution are extracted to obtain an identified lodging proportion distribution and an identified lodging direction distribution;

[0075] S422, the multiple lodging proportion weights of the multiple region grids are calculated and obtained according to the identified lodging proportion distribution;

[0076] S423, the multiple reliability weights of the multiple region grids are calculated and obtained according to the lodging reliability distribution;

[0077] S424, the multiple planning weights of the multiple region grids are calculated and obtained according to the multiple lodging proportion weights and the multiple reliability weights.

[0078] In a preferred embodiment, first, the identified lodging information distribution is structurally analyzed to extract two information components contained therein respectively. Specifically, the identified lodging information corresponding to each region grid in the growth region is accessed one by one, and the lodging proportion and the lodging direction are separated therefrom. The lodging proportion is expressed in percentage form as the proportion of the crop area in the region grid that has occurred lodging to the total crop area of the region grid, and the numerical range is 0% to 100%. The lodging proportion data of all the grids are organized and arranged according to the spatial position to form an identified lodging proportion distribution, which directly reflects the spatial variation of the lodging degree in the entire growth region. The lodging direction is expressed in angle value form as the dominant direction of the crop lodging in each grid, and the angle range is 0° to 360°, with the north direction as the 0° reference. The lodging direction data of all the grids are organized and arranged according to the corresponding spatial position to form an identified lodging direction distribution, which clearly shows the distribution rule and the change trend of the lodging direction in space.

[0079] Subsequently, based on the identified distribution of lodging proportion, a differentiated weight allocation strategy for lodging severity is adopted to calculate the corresponding lodging proportion weight for each regional grid. The core principle of this weight allocation process is that the more severe the lodging degree, the higher the planning importance. Specifically, first, analyze the numerical range and distribution characteristics in the identified distribution of lodging proportion, and determine statistical parameters such as the maximum, minimum and average values of the lodging proportion. Subsequently, a non-linear mapping function is used to convert the lodging proportion value into the corresponding weight value. For example, for grids with high lodging proportion (e.g. lodging proportion exceeding 70%), a larger lodging proportion weight is assigned, indicating that these areas have serious lodging problems and should be given priority in path planning to ensure that the harvesting direction can effectively handle the lodged crops; for grids with medium lodging proportion (e.g. lodging proportion between 30%-70%), a medium weight value is assigned; for grids with low or no lodging proportion (e.g. lodging proportion less than 30%), a smaller weight value is assigned, indicating that these areas have relatively weak constraints on path planning. Through weight allocation, multiple lodging proportion weights corresponding to each regional grid are obtained, which quantitatively reflect the importance differences of different regions in lodging treatment.

[0080] At the same time, based on the aforementioned obtained lodging reliability distribution, a corresponding reliability weight is assigned to each regional grid to ensure sufficient consideration of information reliability in the path planning process. The core principle of this weight allocation process is that the higher the lodging reliability, the higher the planning dependence. Specifically, first, analyze the reliability value distribution of each grid in the lodging reliability distribution, and identify the spatial distribution patterns of high reliability areas, medium reliability areas and low reliability areas. Subsequently, a piecewise linear or smooth curve mapping method is used to convert the reliability value into the corresponding reliability weight. For example, for grids with high reliability (e.g. reliability exceeding 0.8), a larger reliability weight is assigned, indicating that the lodging information in this area is highly reliable and should be fully trusted and valued in path planning; for grids with medium reliability (e.g. reliability between 0.4-0.8), a medium reliability weight is assigned; for grids with low reliability (e.g. reliability less than 0.4), a smaller reliability weight is assigned, indicating that the lodging information in this area has a high degree of uncertainty and should adopt a conservative strategy in path planning to avoid excessive reliance on possibly inaccurate information. Through this weight allocation mechanism, multiple reliability weights corresponding to each regional grid are obtained, which reflect the reliability differences of lodging information in different regions and provide a basis for information quality evaluation for subsequent comprehensive weight calculation.

[0081] After that, based on the plurality of lodging proportion weights and the plurality of reliability weights obtained in the preceding steps, a multi-factor fusion method is used to calculate the final planning weight of each regional grid. The fusion process aims to balance the influence of the two key factors of lodging severity and information reliability. Specifically, for each regional grid, its corresponding lodging proportion weight and reliability weight are subjected to mathematical fusion operation. The fusion method can adopt a product mode or a weighted sum mode: the product mode obtains a comprehensive weight by multiplying the two types of weights, which emphasizes the synergistic effect of the two factors, and only when the lodging proportion and reliability are both high, the comprehensive weight will reach a large value; the weighted sum mode linearly combines the two types of weights by setting a fusion coefficient, which allows the advantage of a single factor to compensate for the deficiency of another factor. Preferably, special case handling strategies can also be considered: for grids with high lodging proportion but low reliability, a conservative weight assignment is adopted to avoid making incorrect path decisions based on unreliable information; for grids with low lodging proportion but high reliability, a moderate weight is assigned to ensure that reliable information is reasonably utilized. Through comprehensive calculation, a plurality of planning weights corresponding to each regional grid are obtained. These planning weights are organized and arranged according to the spatial position of the grid, forming a complete planning weight distribution that comprehensively reflects the importance of each region in path planning, taking into account both the urgency of lodging treatment and the reliability of information, providing a weight basis for subsequent path fitness calculation and optimization decision-making.

[0082] Further, according to the working direction of the harvester through the plurality of regional grids in the first working path, in combination with the identified lodging direction distribution and the planning weight distribution, a first lodging harvesting fitness is calculated and obtained, including:

[0083] S431, obtaining the working direction of the harvester through the plurality of regional grids in the first working path, obtaining a plurality of first working directions;

[0084] S432, according to the identified lodging direction of the plurality of regional grids in the identified lodging direction distribution, obtaining the included angle between the plurality of first working directions, obtaining a plurality of first working included angles, and calculating the similarity with the preset perpendicular angle, obtaining a plurality of first lodging harvesting angle similarities;

[0085] S433, according to the planning weight distribution, performing weighted calculation on the plurality of first lodging harvesting angle similarities, obtaining a first harvesting angle fitness;

[0086] S434, obtaining the first working time of the first working path, calculating the ratio of the preset working time and the first working time, obtaining a first harvesting time fitness;

[0087] S435, according to the first harvesting angle fitness and the first harvesting time fitness, calculating and obtaining a first lodging harvesting fitness.

[0088] In a preferred implementation, firstly, the first working path is analyzed in detail, and the specific working direction information of the harvester when passing through each regional grid is extracted. Since the harvester will pass through multiple regional grids when traveling along the path, it is necessary to determine the local working direction of the harvester in each grid. Specifically, by analyzing the trajectory segment of the harvester in each grid, the dominant travel direction of the segment is calculated. For the grids that the harvester completely passes through, the working direction is determined by the average direction of the trajectory in the grid; for the grids that the harvester only partially passes through, the working direction is calculated based on the trajectory direction of the passing part. The working direction is represented by an angle value, using the same angle reference system as the lodging direction, i.e. taking the north direction as the 0° reference. Through the above extraction process, a plurality of first working directions corresponding to the regional grids passed through by the harvester are obtained.

[0089] Then, based on the identified lodging direction distribution, the identified lodging direction corresponding to each regional grid passed through by the harvester is obtained, and an angle matching analysis is performed with the corresponding first working direction. The analysis process aims to evaluate the rationality of the geometric relationship between the working direction of the harvester and the lodging direction. For each grid passed through by the harvester, the included angle between the first working direction and the identified lodging direction in the grid is calculated. Considering the periodicity of the angle, the smallest included angle between the two directions is selected as the actual included angle value, ensuring that the included angle range is between 0° and 180°. Through the above calculation, a plurality of corresponding first working included angles are obtained. Subsequently, based on the principles of agricultural machinery, the ideal working angle reference of 90° perpendicular angle is calculated, and the similarity of the actual included angle of each grid to the preset perpendicular angle is calculated. The similarity calculation uses a normalization processing method: when the actual included angle is equal to 90°, the similarity is 100%; when the actual included angle deviates from 90°, the similarity decreases accordingly. The specific calculation method is: subtract 1 from the ratio of the difference between the actual included angle and the ideal perpendicular angle to the maximum possible deviation angle. For example, when the included angle between the working direction and the lodging direction is 80°, the deviation from the 90° perpendicular angle is 10°, and the similarity calculation is 1 minus 10° divided by 90°, resulting in about 89%. By executing the above calculation process for all grids passed through by the harvester, a plurality of corresponding first lodging harvesting angle similarities are obtained, which quantitatively reflect the matching degree of the working direction and the lodging direction in each grid.

[0090] Considering that the importance of different regional grids in path planning varies, the lodging harvesting angle similarity of each grid is differentiated based on the planning weight distribution, and the weighted comprehensive harvesting angle fitness is calculated. The planning weight corresponding to each grid is extracted from the planning weight distribution, which comprehensively reflects the lodging severity and information reliability of the grid. Then, the first lodging harvesting angle similarity of each grid is multiplied by the corresponding planning weight to obtain the weighted angle contribution value of each grid. After normalization processing of the weighted angle contribution value of all grids, the first harvesting angle fitness of the entire first operation path is obtained by summation. The first harvesting angle fitness comprehensively reflects the overall effect of the first operation path in handling lodging crops, and the important grid with larger weight has a more significant impact on the fitness.

[0091] In addition to the angle matching effect, the time efficiency performance of the first operation path is evaluated. First, the first operation time corresponding to the first operation path is determined based on the path, which comprehensively considers the actual operation factors such as path length, harvester operation speed, number of turns, and turn time. At the same time, a preset operation time is established as the benchmark for time efficiency evaluation. The preset operation time represents the ideal time standard under the current operation conditions, which is usually calculated based on the theoretical shortest operation time under the efficient path mode such as straight-line round trip, reflecting the time efficiency expectation value of the operation area under ideal path planning. The preset operation time and the first operation time are calculated by ratio, and the first harvesting time fitness is obtained. Specifically, the first harvesting time fitness is equal to the quotient value of the preset operation time divided by the first operation time. When the first operation time is equal to the preset operation time, the ratio is 1, indicating that the time fitness is optimal, and the time efficiency of the path reaches the ideal standard; when the first operation time exceeds the preset operation time, the ratio is less than 1, the time fitness decreases accordingly, indicating that the path needs longer operation time, and the time efficiency is lower than the ideal level; when the first operation time is less than the preset operation time, the ratio is greater than 1, indicating that the time efficiency of the path exceeds the ideal expectation. Through the above first harvesting time fitness calculation, the performance of the first operation path in operation efficiency is quantitatively evaluated, providing an evaluation basis for time dimension for comprehensive fitness evaluation.

[0092] Then, the first harvesting angle fitness and the first harvesting time fitness are fused to calculate a first lodging harvesting fitness as a comprehensive evaluation index of the first operation path. In the fusion process, a weighted combination method is adopted to balance the two factors by setting the weight coefficients of the angle fitness and the time fitness. Generally, since the lodging treatment effect is the main target of path planning, the angle fitness is given a higher weight (such as 0.7), and the time fitness is given a relatively lower weight (such as 0.3). At the same time, the weight configuration can also be adjusted according to actual operation requirements: in the scene where the lodging is serious and the treatment quality requirement is extremely high, the angle fitness weight can be increased to 0.8 or higher; in the scene where the operation time constraint is relatively strict, the time fitness weight can be appropriately increased to balance the two targets. Through fusion calculation, the first lodging harvesting fitness is obtained, which comprehensively reflects the overall performance of the first operation path in terms of lodging treatment effect and operation efficiency, and provides a quantitative evaluation benchmark for subsequent path dynamic optimization.

[0093] In the embodiment two, based on the same inventive concept of the computer vision-based harvester operation path dynamic planning method provided in the embodiment one, the present embodiment also provides a computer vision-based harvester operation path dynamic planning system, which comprises: Figure 2

[0094] A lodging prediction module 11 is configured to monitor and obtain abnormal weather data and growth stages in a growth area during crop growth, to obtain a sequence of abnormal weather data and a sequence of growth stage information, to perform crop lodging prediction, and to obtain a predicted lodging information distribution.

[0095] A visual recognition module 12 is configured to collect regional images in the growth area during crop harvesting, to perform recognition using computer vision, and to obtain a recognized lodging information distribution.

[0096] A credibility analysis module 13 is configured to perform lodging credibility analysis according to the predicted lodging information distribution and the recognized lodging information distribution, and to obtain a lodging credibility distribution.

[0097] A path planning module 14 is configured to plan a harvester operation path in the growth area, to calculate a lodging harvesting fitness of the operation path in combination with the lodging credibility distribution, to perform path dynamic optimization, and to obtain an optimal operation path as a path planning result, wherein the lodging harvesting fitness is calculated according to an operation direction in the operation path and a lodging direction in the lodging information.

[0098] Further, the lodging prediction module 11 comprises the following execution steps:

[0099] During crop growth, the weather data of a plurality of regional grids in the growth area is monitored and obtained. ​

[0100] When the monitored meteorological data meets the abnormal meteorological condition, record the abnormal meteorological data and the growth stage of the crop at the corresponding time, obtain the abnormal meteorological data sequence and the growth stage information sequence of the multiple area grids;

[0101] Input the abnormal meteorological data sequence and the growth stage information sequence of the multiple area grids into the lodging prediction network respectively, output multiple predicted lodging information of the multiple area grids, and obtain the predicted lodging information distribution, wherein each predicted lodging information includes a lodging direction and a lodging proportion.

[0102] Further, the lodging prediction network is obtained based on machine learning training, and the training steps include:

[0103] According to the crop lodging monitoring data, a sample abnormal meteorological data sequence set and a sample growth stage information sequence set are collected, and the lodging direction and the lodging proportion of the crop growth under different sample abnormal meteorological data sequences and sample growth stage information sequences are collected, and a sample lodging information set is labeled;

[0104] A lodging prediction network based on machine learning is constructed;

[0105] The sample abnormal meteorological data sequence set, the sample growth stage information sequence set and the sample lodging information set are used to supervise the training and test the accuracy of the lodging prediction network, and the training is completed after meeting the requirements.

[0106] Further, the visual recognition module 12 includes the following execution steps:

[0107] When the crop is harvested, the area image in the growth area is collected and divided into multiple grid images of multiple area grids;

[0108] The multiple grid images are input into the lodging recognition network respectively, and multiple identified lodging information is obtained as the identified lodging information distribution, wherein the lodging recognition network is constructed based on computer vision and trained using sample grid images and labeled sample identified lodging information.

[0109] Further, the credibility analysis module 13 includes the following execution steps:

[0110] The predicted lodging information and the identified lodging information of each area grid in the growth area are extracted, and multiple sets of lodging information are obtained;

[0111] The similarity of the predicted lodging information and the identified lodging information in each set of lodging information is calculated, and the lodging credibility is obtained, wherein the similarity of the lodging direction and the lodging proportion in the predicted lodging information and the identified lodging information is calculated, and the lodging credibility is obtained;

[0112] The lodging credibility in the plurality of region grids is combined to obtain a lodging credibility distribution.

[0113] Further, the path planning module 14 comprises the following execution steps:

[0114] The harvester operation path planning is performed in the growth area to obtain a first operation path;

[0115] The lodging proportion and the lodging direction in the identified lodging information distribution are extracted to obtain an identified lodging proportion distribution and an identified lodging direction distribution, and the planning weight of the plurality of region grids is distributed in combination with the lodging credibility distribution to obtain a planning weight distribution;

[0116] According to the operation direction of the harvester passing through the plurality of region grids in the first operation path, the first lodging harvesting fitness is calculated in combination with the identified lodging direction distribution and the planning weight distribution;

[0117] The random operation path planning is continuously performed, the path dynamic optimization is performed, and after the optimization converges, the operation path with the largest lodging harvesting fitness is reserved to obtain an optimal operation path.

[0118] Further, the path planning module 14 further comprises the following execution steps:

[0119] The lodging proportion and the lodging direction of the plurality of region grids in the identified lodging information distribution are extracted to obtain an identified lodging proportion distribution and an identified lodging direction distribution;

[0120] According to the identified lodging proportion distribution, a plurality of lodging proportion weights of the plurality of region grids are distributed and calculated;

[0121] According to the lodging credibility distribution, a plurality of credibility weights of the plurality of region grids are distributed and calculated;

[0122] According to the plurality of lodging proportion weights and the plurality of credibility weights, a plurality of planning weights of the plurality of region grids are distributed and calculated.

[0123] Further, the path planning module 14 further comprises the following execution steps:

[0124] The operation direction of the harvester passing through the plurality of region grids in the first operation path is obtained to obtain a plurality of first operation directions;

[0125] According to the identified lodging direction of the plurality of region grids in the identified lodging direction distribution, the included angle with the plurality of first operation directions is obtained to obtain a plurality of first operation included angles, and the similarity with a preset vertical angle is calculated to obtain a plurality of first lodging harvesting angle similarities;

[0126] According to the planning weight distribution, the first lodging harvesting angle similarities are weighted to obtain a first harvesting angle fitness;

[0127] A first working time of the first working path is obtained, and a ratio of a preset working time and the first working time is calculated to obtain a first harvesting time fitness;

[0128] According to the first harvesting angle fitness and the first harvesting time fitness, a first lodging harvesting fitness is calculated.

[0129] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0130] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0131] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system), and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing devices to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices produce a device that implements the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that implements the functions specified in the flow(s) or block(s).

[0132] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing devices to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction apparatus, which implements the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that implements the functions specified in the flow(s) or block(s).

[0133] These computer program instructions can also be loaded into a computer or other programmable data processing devices, so that a series of operational steps are performed on the computer or other programmable devices to generate computer-implemented processes, thus the instructions executed on the computer or other programmable devices provide the function for implementing the processes specified in the flowchart Figure 1 one flow or multiple flows and / or the functions specified in the flowchart Figure 1 one flow or multiple flows and / or the functions specified in the flowchart

[0134] Although the preferred embodiments of the application have been described, those skilled in the art will be able to make additional modifications and variations to these embodiments without departing from the spirit and scope of the application.

[0135] Obviously, various modifications and changes can be made to the present application by those skilled in the art without departing from the spirit and scope of the present application. Accordingly, it is intended that the present application embrace all such modifications and changes as fall within the scope of the present application and its equivalents.

Claims

1. A dynamic planning method for harvester operation paths based on computer vision, characterized in that, The method includes: During crop growth, abnormal meteorological data and growth stages within the growth area are monitored and acquired to obtain abnormal meteorological data sequences and growth stage information sequences, thereby enabling crop lodging prediction and obtaining the distribution of predicted lodging information. During crop harvesting, regional images of the growing area are collected, and computer vision is used to identify the distribution of lodging information. Based on the predicted lodging information distribution and the identified lodging information distribution, a lodging confidence analysis is performed to obtain the lodging confidence distribution; Harvester operation path planning is performed within the growth area. Based on the lodging confidence distribution, the lodging harvesting adaptability of the operation path is calculated. The path is dynamically optimized to obtain the optimal operation path as the path planning result. The lodging harvesting adaptability is calculated based on the operation direction within the operation path and the lodging direction within the lodging information. During crop growth, abnormal meteorological data and growth stages within the growth area are monitored and acquired to obtain abnormal meteorological data sequences and growth stage information sequences. This data is then used to predict crop lodging and obtain the distribution of predicted lodging information, including: During crop growth, meteorological data from multiple regional grids within the growth area are monitored and acquired. When the monitored meteorological data meets the abnormal meteorological conditions, the abnormal meteorological data and the corresponding crop growth stage at the time are recorded and obtained, and the abnormal meteorological data sequence and growth stage information sequence of multiple regional grids are obtained. The abnormal meteorological data sequences and growth stage information sequences of multiple regional grids are input into the lodging prediction network, and multiple predicted lodging information of multiple regional grids are predicted and output to obtain the distribution of predicted lodging information. Each predicted lodging information includes the lodging direction and lodging ratio. The lodging prediction network is obtained through machine learning training, and the training steps include: Based on crop lodging monitoring data, a set of abnormal meteorological data sequences and a set of sample growth stage information sequences were collected. The lodging direction and lodging ratio of crops under different abnormal meteorological data sequences and sample growth stage information sequences were collected and labeled to obtain a set of sample lodging information. Construct a lodging prediction network based on machine learning; The accuracy of the lodging prediction network is supervised and tested using the sample abnormal meteorological data sequence set, sample growth stage information sequence set, and sample lodging information set, and training is completed after meeting the requirements. Harvester operation path planning is performed within the growth area. Based on the lodging confidence distribution, the lodging harvesting adaptability of the operation path is calculated. Dynamic path optimization is then performed to obtain the optimal operation path, including: Within the growth area, a harvester operation path is planned to obtain a first operation path; Extract the lodging ratio and lodging direction within the lodging information distribution to obtain the lodging ratio distribution and the lodging direction distribution. Combine the lodging confidence distribution to assign planning weights to multiple regional grids to obtain the planning weight distribution. Based on the working direction of the harvester passing through multiple grid areas within the first working path, and combined with the identified lodging direction distribution and the planning weight distribution, the first lodging harvesting fitness is calculated. Continue to randomly plan the operation path and perform dynamic path optimization. After the optimization converges, retain the operation path with the highest adaptability to lodging harvesting to obtain the optimal operation path.

2. The harvester operation path dynamic planning method based on computer vision according to claim 1, characterized in that, During crop harvesting, images of the growing area are collected, and computer vision is used to identify lodging information, including: During crop harvesting, regional images of the growing area are acquired and divided into multiple raster images of multiple regions. Multiple raster images are input into the landslide recognition network to identify multiple landslide information, which are used as the landslide information distribution. The landslide recognition network is constructed based on computer vision and is trained using sample raster images and labeled sample landslide information.

3. The harvester operation path dynamic planning method based on computer vision according to claim 1, characterized in that, Based on the predicted lodging information distribution and the identified lodging information distribution, a lodging confidence analysis is performed to obtain the lodging confidence distribution, including: Extract the predicted lodging information and identified lodging information of each grid cell within the growth area to obtain multiple sets of lodging information; Calculate the similarity between predicted and identified lodging information within each group of lodging information to obtain the lodging confidence score. Specifically, calculate the similarity between the lodging direction and the lodging ratio within the predicted and identified lodging information, and obtain the lodging confidence score. By combining the lodging confidence scores within multiple regional grids, the lodging confidence score distribution is obtained.

4. The harvester operation path dynamic planning method based on computer vision according to claim 1, characterized in that, Extract the lodging ratio and lodging direction within the lodging information distribution to obtain the lodging ratio distribution and the lodging direction distribution. Combined with the lodging confidence distribution, assign planning weights to multiple regional grids to obtain the planning weight distribution, including: Extract the lodging ratio and lodging direction of multiple grid areas within the lodging information distribution to obtain the lodging ratio distribution and the lodging direction distribution. Based on the identified lodging ratio distribution, multiple lodging ratio weights are calculated for multiple regional grids; Based on the collapse confidence distribution, multiple confidence weights are calculated for multiple regional grids; Based on the multiple collapse ratio weights and multiple reliable weights, multiple planning weights for multiple regional grids are calculated and allocated.

5. The harvester operation path dynamic planning method based on computer vision according to claim 1, characterized in that, Based on the harvester's operating direction through multiple grid areas within the first operating path, and combined with the identified lodging direction distribution and planning weight distribution, the first lodging harvesting fitness is calculated, including: Obtain the working direction of the harvester passing through multiple grid areas within the first working path to obtain multiple first working directions; Based on the identified lodging direction of multiple grid areas within the lodging direction distribution, the angle between the lodging direction and the multiple first working directions is obtained, multiple first working angles are obtained, and the similarity with the preset vertical angle is calculated to obtain multiple first lodging harvesting angle similarity. Based on the planning weight distribution, the similarity of the multiple first lodging harvesting angles is weighted and calculated to obtain the fitness of the first harvesting angle; Obtain the first operation time of the first operation path, calculate the ratio of the preset operation time to the first operation time, and obtain the first harvesting time adaptability; The first lodging harvesting fitness is calculated based on the first harvesting angle fitness and the first harvesting time fitness.

6. A harvester operation path dynamic planning system based on computer vision, characterized in that, A method for implementing the computer vision-based dynamic planning method for harvester operation paths as described in any one of claims 1 to 5, the method comprising: The lodging prediction module is used to monitor and acquire abnormal meteorological data and growth stages within the crop growth area during the crop growth process, obtain abnormal meteorological data sequences and growth stage information sequences, perform crop lodging prediction, and obtain the predicted lodging information distribution. The visual recognition module is used to collect regional images of the growing area during crop harvesting, and to use computer vision to identify the distribution of lodging information. The credibility analysis module is used to perform lodging credibility analysis based on the predicted lodging information distribution and the identified lodging information distribution, and to obtain the lodging credibility distribution. The path planning module is used to plan the harvester's operating path within the growth area. Combining the lodging confidence distribution, it calculates the lodging harvesting fitness of the operating path, performs dynamic path optimization, and obtains the optimal operating path as the path planning result. The lodging harvesting fitness is calculated based on the operating direction within the operating path and the lodging direction within the lodging information.

Images