Harvester operation path dynamic planning method and system based on computer vision

Abstract

The invention provides a harvester operation path dynamic planning method and system based on computer vision, and belongs to the field of path planning. The method comprises the steps that in the crop growth process, crop lodging prediction is carried out by monitoring abnormal meteorological data and growth stages, and predicted lodging information distribution is obtained; when crops are harvested, computer vision is adopted to identify the area image, and lodging identification information distribution is obtained; performing lodging credibility analysis according to the predicted and identified lodging information distribution; and performing harvester operation path planning in combination with lodging credibility distribution, calculating lodging harvesting fitness, and performing dynamic optimization to obtain an optimal operation path. The technical problems that in the prior art, due to unreasonable operation path planning of the harvester, the header pushes lodging crops, and the harvesting quality is poor are solved, and the technical effects that by dynamically planning the operation path of the harvester, the header is prevented from pushing the lodging crops, and the harvesting quality of the lodging crops is improved are achieved.

Description

Technical Field

[0001] This invention relates to the field of path planning, and more particularly to a method and system for dynamic planning of harvester operation paths based on computer vision. Background Technology

[0002] Crop harvesting is a crucial part of agricultural production, and with the continuous improvement of agricultural mechanization, harvesters are playing an increasingly important role in crop harvesting. However, in actual agricultural production, crops often lodge due to adverse weather conditions such as strong winds and heavy rains, posing a serious challenge to mechanized harvesting.

[0003] Currently, combine harvesters rely primarily on pre-set fixed paths when harvesting lodged crops. This path planning method often fails to adapt to the actual distribution of lodged crops. Due to the lack of effective identification of lodging conditions and dynamic path adjustment, combine harvesters are prone to pushing and shoving lodged crops during operation. This not only affects harvesting efficiency but also leads to increased grain loss, decreased harvesting quality, and other adverse consequences, resulting in poor harvesting quality. Summary of the Invention

[0004] This invention addresses the technical problem in the prior art where unreasonable harvester path planning leads to the header pushing and lodging crops, resulting in poor harvesting quality. It provides a method and system for dynamic planning of harvester paths based on computer vision to solve this problem.

[0005] The technical solution of the present invention to solve the above-mentioned technical problems is as follows:

[0006] In a first aspect, the present invention provides a dynamic planning method for harvester operation paths based on computer vision, comprising: during crop growth, monitoring and acquiring abnormal meteorological data and growth stages within the growth area, obtaining abnormal meteorological data sequences and growth stage information sequences, predicting crop lodging, and obtaining the predicted lodging information distribution; during crop harvesting, acquiring regional images within the growth area, using computer vision for identification, and obtaining the identified lodging information distribution; based on the predicted lodging information distribution and the identified lodging information distribution, performing lodging confidence analysis, and obtaining the lodging confidence distribution; planning the harvester operation path within the growth area, calculating the lodging harvesting fitness of the operation path in conjunction with the lodging confidence distribution, performing dynamic path optimization, and obtaining the optimal operation path as the path planning result, wherein the lodging harvesting fitness is calculated based on the operation direction within the operation path and the lodging direction within the lodging information.

[0007] Secondly, this invention provides a harvester operation path dynamic planning system based on computer vision, comprising: a lodging prediction module, used to monitor and acquire abnormal meteorological data and growth stages within the growth area during crop growth, obtain abnormal meteorological data sequences and growth stage information sequences, perform crop lodging prediction, and obtain the predicted lodging information distribution; a visual recognition module, used to collect regional images within the growth area during crop harvesting, and use computer vision for recognition to obtain the identified lodging information distribution; a credibility analysis module, used to perform lodging credibility analysis based on the predicted lodging information distribution and the identified lodging information distribution, and obtain the lodging credibility distribution; and a path planning module, used to plan the harvester operation path within the growth area, calculate the lodging harvesting fitness of the operation path based on the lodging credibility distribution, perform dynamic path optimization, and obtain the optimal operation path as the path planning result, wherein the lodging harvesting fitness is calculated based on the operation direction within the operation path and the lodging direction within the lodging information.

[0008] The beneficial effects of this invention are:

[0009] During crop growth, abnormal meteorological data and growth stages within the growing area are monitored and acquired to obtain abnormal meteorological data sequences and growth stage information sequences. This allows for lodging prediction, obtaining the predicted lodging information distribution. This enables the early prediction of potential lodging areas, lodging proportions, and lodging directions before harvest, providing fundamental data support for subsequent path planning. During harvesting, regional images of the growing area are collected, and computer vision is used for identification to obtain the lodging information distribution. This allows for real-time acquisition of the actual lodging status of the crop, compensating for potential biases in meteorological forecasts and providing more accurate lodging information. Based on the predicted and identified lodging information distributions, lodging reliability analysis is performed to obtain the lodging reliability distribution. This allows for the fusion analysis of predicted and real-time identified information, eliminating errors from single information sources and improving the accuracy and reliability of lodging information. Harvester operation path planning is performed within the growing area. Combining the lodging reliability distribution, the lodging harvesting adaptability of the operation path is calculated, and dynamic path optimization is performed to obtain the optimal operation path as the path planning result. The lodging harvesting adaptability is calculated based on the operation direction within the operation path and the lodging direction within the lodging information. By intelligently optimizing the path, the harvester's operating direction is made as perpendicular as possible to the direction of lodging, avoiding the header from pushing over lodged crops and improving harvesting quality.

[0010] The above technical solution enables dynamic planning and intelligent optimization of the harvesting path for lodged crops. Compared to the fixed path planning method in existing technologies, the technical solution of this application can flexibly adjust the operation path and direction according to the lodging situation in different areas, maximizing the perpendicularity between the harvesting direction and the lodging direction, thereby effectively avoiding the pushing of lodged crops by the header and improving the harvesting quality. Attached Figure Description

[0011] Figure 1 A flowchart illustrating the computer vision-based dynamic path planning method for harvesters provided by this invention;

[0012] Figure 2 This is a schematic diagram of the structure of the harvester operation path dynamic planning system based on computer vision provided by the present invention.

[0013] In the attached diagram, the components represented by each number are as follows:

[0014] The system includes a lodging prediction module 11, a visual recognition module 12, a credibility analysis module 13, and a path planning module 14. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0017] In the description of this invention, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed herein.

[0018] Example 1, as Figure 1 As shown, this embodiment of the invention provides a dynamic planning method for harvester operation paths based on computer vision, including:

[0019] S1. During crop growth, monitor and acquire abnormal meteorological data and growth stages within the growth area, obtain abnormal meteorological data sequences and growth stage information sequences, perform crop lodging prediction, and obtain the predicted lodging information distribution.

[0020] Specifically, meteorological monitoring equipment deployed within the growing area monitors and acquires meteorological data in real time. When the monitored meteorological data meets preset abnormal meteorological conditions, the corresponding abnormal meteorological data is automatically recorded, and the crop's current growth stage information is also acquired. Abnormal meteorological data includes, but is not limited to, meteorological factors that may cause crop lodging, such as strong wind speed, wind direction, rainfall, and temperature changes. Abnormal meteorological conditions can be set as criteria such as wind speed exceeding a preset threshold or duration meeting specific requirements. The crop's current growth stage information includes identifiers for different developmental stages, such as sowing, tillering, jointing, heading, grain-filling, and maturity.

[0021] Through continuous monitoring, a time-series sequence of abnormal meteorological data and corresponding growth stage information was obtained. These data sequences reflect the historical record of abnormal meteorological conditions experienced by crops throughout their growth cycle and their growth status at the time of occurrence. Subsequently, based on the obtained abnormal meteorological data sequences and growth stage information sequences, a pre-trained lodging prediction network was used to perform crop lodging prediction analysis. Taking into account factors such as the intensity of meteorological factors and differences in lodging resistance at different growth stages, the network outputs a predicted lodging information distribution including lodging direction and lodging ratio. This predicted lodging information distribution refers to the spatial distribution of predicted lodging information corresponding to each grid cell within the entire region after dividing the growth area into multiple grid cells, providing a basis for lodging risk assessment in subsequent path planning.

[0022] S2. During crop harvesting, regional images of the growing area are collected and computer vision is used for identification to obtain information on the distribution of lodging.

[0023] Specifically, during crop harvesting, real-time image acquisition of the growing area is first performed using image acquisition equipment to obtain regional images. The image acquisition equipment is preferably a high-resolution camera mounted on a drone, satellite, or other aircraft, capable of acquiring regional images covering the entire growing area from an aerial perspective. The acquired regional images contain information about the actual growth status of the crops, especially the actual situation of crop lodging. To facilitate subsequent processing and analysis, the acquired regional images are divided into multiple raster images corresponding to regional rasteres according to a preset raster division rule. Each raster image represents the crop status within a specific area.

[0024] Subsequently, deep learning-based computer vision technology was employed to perform intelligent recognition processing on the raster images. Specifically, a lodging recognition network was pre-built and trained using deep learning architectures such as convolutional neural networks (CNNs), and supervised learning training was conducted on a large number of sample raster images containing different lodging states. The lodging recognition network can automatically extract crop features from the images, including visual features such as the tilt angle of crop stems, leaf distribution morphology, and shadow features, and determine the lodging state of the crops based on these features. For each raster image, the lodging recognition network outputs corresponding lodging information, which includes two key parameters: lodging direction and lodging percentage. The lodging direction indicates the main direction in which the crop tilts, usually expressed as an angle value; the lodging percentage indicates the percentage of crops that have lodged in that raster area out of the total crop.

[0025] By performing recognition processing on all raster images, lodging information corresponding to multiple regional raster areas is obtained, thereby forming the lodging information distribution, which reflects the actual lodging status of the crop in the entire growing area at the time of harvest.

[0026] S3. Based on the predicted lodging information distribution and the identified lodging information distribution, perform lodging confidence analysis to obtain the lodging confidence distribution.

[0027] Specifically, since both weather forecasting and image recognition have certain errors and uncertainties, a lodging credibility analysis is conducted by fusing the predicted lodging information distribution and the identified lodging information distribution to improve the accuracy and reliability of lodging information.

[0028] First, predicted lodging information and identified lodging information are extracted from each raster area within the growth region, forming multiple pairs of lodging information to be analyzed. Each pair contains predicted lodging information (from weather forecasts) and identified lodging information (from image recognition) for the same raster location. For each pair, the similarity between the predicted and identified lodging information is calculated as the lodging confidence level for that raster location. The similarity calculation involves two dimensions: lodging direction similarity and lodging proportion similarity. Lodging direction similarity is determined by comparing the angular difference between the predicted and identified lodging directions; the smaller the angular difference, the higher the similarity. Lodging proportion similarity is calculated by comparing the numerical difference between the predicted and identified lodging proportions; the smaller the numerical difference, the higher the similarity. Subsequently, a weighted fusion method is used to comprehensively calculate the lodging direction similarity and lodging proportion similarity to obtain the overall lodging confidence level for that raster. A higher confidence level for landslides indicates that the landslide information at that grid location is more reliable; a lower confidence level indicates that there is a larger prediction bias or identification error.

[0029] By performing the aforementioned lodging confidence calculation on all grid cells within the growth area, the lodging confidence of each grid cell is obtained, thus forming a lodging confidence distribution. This lodging confidence distribution provides a confidence assessment of lodging information for subsequent path planning, enabling the reasonable allocation of planning weights for each region during path optimization, prioritizing the lodging status of high-confidence regions, thereby improving the accuracy and practicality of path planning.

[0030] S4. In the growth area, perform harvester operation path planning, combine the lodging confidence distribution, calculate the lodging harvesting adaptability of the operation path, perform dynamic path optimization, and obtain the optimal operation path as the path planning result. The lodging harvesting adaptability is calculated based on the operation direction within the operation path and the lodging direction within the lodging information.

[0031] Specifically, after obtaining the lodging confidence distribution, harvester operation path planning is carried out within the growing area. Based on the lodging confidence distribution, the harvester operation path is dynamically optimized to form the optimal geometric relationship with the lodging direction, so as to achieve efficient harvesting of lodged crops.

[0032] First, initial harvester path planning is performed within the growing area, generating a first working path as the starting point for optimization. This first working path can be generated using traditional path planning algorithms, such as the grid method or the A* algorithm, mainly considering basic constraints such as area boundaries and obstacle distribution. Next, the lodging ratio and lodging direction of each grid region in the lodging information distribution are extracted, forming lodging ratio distribution and lodging direction distribution respectively. Combined with the aforementioned lodging confidence distribution, a corresponding planning weight is assigned to each grid region. Specifically, regions with high lodging ratios are assigned larger lodging ratio weights, and regions with high lodging confidence are assigned larger confidence weights. The combined weights of both are used to calculate the planning weight distribution for each grid region.

[0033] When calculating the lodging harvesting fitness of a harvesting path, the geometric relationship between the harvesting direction and the lodging direction is the primary consideration. According to agricultural machinery principles, when the harvester's operating direction forms a 90° angle with the lodging direction, the "pushing" phenomenon of the header on the lodged crop can be effectively avoided, thereby improving harvesting quality and efficiency. Specifically, the operating direction of the harvester as it passes through each grid area within the harvesting path is obtained, and the angle between this operating direction and the identified lodging direction within the corresponding grid is calculated. By comparing the similarity of this angle with a preset vertical angle (90°), the lodging harvesting angle similarity of each grid is obtained. Subsequently, the angle similarity of each grid is weighted according to a planning weight distribution to obtain the overall harvesting angle fitness. Simultaneously, the operating time factor is also considered, and the ratio of the operating time corresponding to the harvesting path to the preset operating time is calculated to obtain the harvesting time fitness. Finally, by combining the harvesting angle fitness and the harvesting time fitness, the lodging harvesting fitness of the harvesting path is calculated.

[0034] Based on the aforementioned fitness evaluation mechanism, heuristic optimization algorithms (such as genetic algorithms and particle swarm optimization) are used for dynamic path optimization. By continuously generating new candidate paths and calculating their lodging harvesting fitness, the path scheme is gradually improved during the optimization process. When the optimization algorithm converges, the path with the highest lodging harvesting fitness is retained as the optimal path, and the output is the final path planning result. This optimal path maximizes the perpendicular alignment between the harvester's working direction and the lodging direction, while also considering operational efficiency, providing path guidance for the mechanized harvesting of lodged crops and improving harvesting quality.

[0035] Furthermore, during crop growth, abnormal meteorological data and growth stages within the growth area are monitored and acquired to obtain abnormal meteorological data sequences and growth stage information sequences. This information is then used to predict crop lodging and obtain the predicted lodging information distribution, including:

[0036] S11. During crop growth, monitor and acquire meteorological data from multiple regional grids within the growth area;

[0037] S12. When the monitored meteorological data meets the abnormal meteorological conditions, record the abnormal meteorological data and the corresponding crop growth stage at the time, and obtain the abnormal meteorological data sequence and growth stage information sequence of multiple regional grids.

[0038] S13. Input the abnormal meteorological data sequence and growth stage information sequence of multiple regional grids into the lodging prediction network respectively, predict and output multiple predicted lodging information of multiple regional grids, and obtain the distribution of predicted lodging information. Each predicted lodging information includes the lodging direction and lodging ratio.

[0039] In one feasible implementation, the growing area is first spatially gridded according to a preset grid division standard. The grid division standard is determined based on crop planting density, terrain features, and monitoring accuracy requirements. For example, the growing area may be divided into square grids with sides ranging from 10m x 10m to 50m x 50m, or an irregular grid division method may be used based on the actual plot shape. Based on the grid division results, a distributed meteorological monitoring network is constructed within the growing area. This distributed meteorological monitoring network consists of multiple meteorological monitoring nodes, each responsible for covering one area grid, and each area grid is equipped with independent meteorological monitoring equipment. If meteorological monitoring equipment resources are limited, a sparse deployment method can be adopted, using spatial interpolation algorithms (such as Kriging interpolation, inverse distance weighted interpolation, etc.) to calculate meteorological data for grids not directly monitored. The meteorological monitoring equipment at each meteorological monitoring node collects multi-dimensional meteorological parameters within the coverage area in real time, including but not limited to key meteorological factors affecting crop lodging such as wind speed, wind direction, temperature, humidity, air pressure, rainfall, and solar radiation intensity. To ensure the timeliness and completeness of data, the monitoring frequency can be dynamically adjusted according to the lodging sensitivity of crops at different growth stages. For example, during the seedling and maturity stages when crops have strong lodging resistance, a lower monitoring frequency (such as collecting data once per hour) can be used; during critical growth stages when crops are prone to lodging (such as the rapid stem elongation stage during the jointing stage and the top-heavy stage during the heading stage), the monitoring frequency can be increased to collecting data once every 10-15 minutes to ensure the timely capture of abnormal weather events.

[0040] During continuous meteorological monitoring, a multi-level standard for judging abnormal meteorological conditions is established. This standard comprehensively considers single meteorological parameter thresholds and combinations of multiple parameters, such as wind speed exceeding 8 m / s for more than 30 minutes, wind direction changing by more than 90° within 2 hours, or short-term rainfall exceeding 20 mm. When the monitoring data of any area grid meets the above abnormal meteorological conditions, an abnormal event is automatically triggered and the specific meteorological parameter values ​​that triggered the abnormal condition (including peak value, duration, rate of change, etc.) are recorded. At the same time, the growth stage information of the crop in that grid at the time of the abnormal event is obtained. Among them, the crop growth stage information is calculated in real time through a pre-established crop growth model. Based on parameters such as sowing time, accumulated temperature, and variety characteristics, the crop growth process can be accurately divided into 11 main stages: sowing period, seedling stage, tillering stage, jointing stage, booting stage, heading stage, flowering stage, grain filling stage, milk stage, waxy stage, and full maturity stage. Each stage corresponds to different lodging resistance parameters. Through continuous monitoring and recording throughout the entire growth cycle, a time-series sequence of abnormal meteorological data and a corresponding sequence of growth stage information are constructed for each regional raster, forming a complete meteorological-growth stage correlation database.

[0041] Subsequently, a pre-trained lodging prediction network was used to analyze the time-series data of each regional grid. The lodging prediction network was built on a deep learning architecture, preferably employing temporal neural network structures such as Long Short-Term Memory (LSTM) or Gated Recurrent Unit (GRU), which can effectively learn the temporal dependencies of abnormal meteorological data and the impact of growth stage changes on lodging risk. For each regional grid, its abnormal meteorological data sequence and growth stage information sequence were feature-engineered and used as feature vectors input to the lodging prediction network. The lodging prediction network, through nonlinear transformations of multiple layers of neurons, comprehensively analyzes factors such as the intensity and duration of meteorological factors and the current lodging resistance of the crop, outputting the predicted lodging information for that grid, including the lodging direction and lodging percentage. The lodging direction is represented by angle values ​​(0°-360°), closely related to the historical prevailing wind direction; the lodging percentage is represented as a percentage (0%-100%), reflecting the proportion of crop area expected to be lodged within that grid region. The above prediction process is executed on all grid areas within the growing region to obtain the predicted lodging information corresponding to each grid. Finally, the predicted lodging information distribution covering the entire growing region is summarized to provide lodging analysis based on historical meteorological data for subsequent harvesting path planning.

[0042] Furthermore, the lodging prediction network is obtained based on machine learning training, and the training steps include:

[0043] S131. Based on crop lodging monitoring data, collect a set of abnormal meteorological data sequences and a set of growth stage information sequences for samples, and collect the lodging direction and lodging ratio of crops under different abnormal meteorological data sequences and growth stage information sequences for samples, and label them to obtain a set of lodging information for samples.

[0044] S132. Construct a lodging prediction network based on machine learning;

[0045] S133. Using the sample abnormal meteorological data sequence set, sample growth stage information sequence set, and sample lodging information set, supervise the training and testing of the accuracy of the lodging prediction network, and complete the training after meeting the requirements.

[0046] In a preferred embodiment, the lodging prediction network is trained using machine learning. First, a training dataset for the lodging prediction network is established through large-scale field trials and historical data collection. Specifically, historical records are extracted from crop lodging monitoring databases covering various regions, crop varieties, and planting years. These monitoring data encompass combinations of meteorological conditions and crop growth stages. Then, based on the aforementioned monitoring data, a set of sample abnormal meteorological data sequences and a set of sample growth stage information sequences are collected. The sample abnormal meteorological data sequence set contains records of abnormal meteorological events of different intensities, durations, and frequencies, such as detailed parameters of extreme weather events like strong winds, heavy rain, and hail. The sample growth stage information sequence set records the specific growth stage of the crop and its lodging resistance parameters when the abnormal meteorological event occurred. To establish the correspondence between input and output, various methods can be used to collect the actual lodging results of crops under different sample conditions. For example, through field surveys, drone aerial photography, and satellite remote sensing, the actual lodging direction and lodging ratio of crops under the influence of specific abnormal meteorological data sequences and growth stage information sequences can be accurately measured and recorded. The lodging direction was determined by compass measurement or image analysis, accurate to the degree; the lodging ratio was obtained by statistically analyzing the percentage of lodged plants in the quadrat out of the total number of plants. Through the above data collection process, output labels corresponding to specific sample abnormal meteorological data sequences and sample growth stage information sequences were established, namely, sample lodging information (including lodging direction and lodging ratio), forming a sample lodging information set, providing sufficient supervised learning data for network training.

[0047] Subsequently, a lodging prediction network based on machine learning was constructed. This network employs a multi-input, multi-output design, capable of simultaneously processing two types of time-series inputs: abnormal meteorological data sequences and growth stage information sequences, and outputting two prediction results: lodging direction and lodging ratio. The main network structure preferably utilizes recurrent neural network architectures such as Long Short-Term Memory (LSTM) or Gated Recurrent Units (GRU) to effectively capture long-term dependencies in the time-series data. Specifically, the lodging prediction network comprises four main parts: a feature extraction layer, a time-series modeling layer, a feature fusion layer, and an output layer. The feature extraction layer is responsible for preprocessing and initial feature extraction of the raw input data; the time-series modeling layer learns the temporal evolution of abnormal meteorological events and the changing patterns of growth stages through multi-layer LSTM or GRU units; the feature fusion layer fuses meteorological features and growth stage features to form a comprehensive lodging risk feature representation; and the output layer maps the features to predicted values ​​of lodging direction (regression task) and lodging ratio (regression task) through fully connected layers and activation functions. Meanwhile, the lodging prediction network can also incorporate an attention mechanism, enabling it to automatically identify the most critical time points and feature dimensions for lodging prediction, thereby improving prediction accuracy.

[0048] Subsequently, a supervised learning method was used to train the constructed lodging prediction network. The collected sets of abnormal meteorological data sequences and growth stage information sequences were used as training inputs, while the lodging direction and lodging ratio from the lodging information set were used as training labels. To ensure training effectiveness, the total samples were randomly divided into training, validation, and test sets in a 7:2:1 ratio. The lodging prediction network was trained using the training set, employing a gradient descent optimization algorithm. Network parameters were updated via backpropagation to minimize the loss function between the predicted output and the true label. The loss function was designed as a weighted combination of the lodging direction prediction error and the lodging ratio prediction error to balance the importance of the two output tasks. Simultaneously, the performance of the lodging prediction network on the validation set was continuously monitored, including evaluation metrics such as the mean absolute error (MAE) of lodging direction prediction and the root mean square error (RMSE) of lodging ratio prediction. An early stopping mechanism was triggered to prevent overfitting if the network performance metrics did not significantly improve for several consecutive training cycles. After training, the network's generalization ability was evaluated on an independent test set. The preset performance requirements include: a lodging direction prediction error of less than 15°, a lodging proportion prediction error of less than 10%, and an overall prediction accuracy of over 85%. When the network's performance metrics on the test set meet all the above requirements, training is considered complete, and a lodging prediction network suitable for practical lodging prediction is obtained.

[0049] Furthermore, during crop harvesting, regional images of the growing area are collected, and computer vision is used for identification to obtain information on the distribution of lodged crops, including:

[0050] S21. When harvesting crops, collect regional images of the growing area and divide them into multiple raster images of multiple regions.

[0051] S22. Input multiple raster images into the landslide recognition network to identify multiple landslide information, which are used as the landslide information distribution. The landslide recognition network is constructed based on computer vision and is trained using sample raster images and labeled sample landslide information.

[0052] In a preferred embodiment, during crop harvesting, an image acquisition device is used to acquire comprehensive image data of the entire growing area, collecting regional images within the growing area. The image acquisition device preferably employs a drone platform equipped with a high-resolution digital camera, conducting aerial photography at a fixed altitude (typically 50-100 meters) to ensure that the acquired regional images have consistent spatial resolution and clarity. To guarantee the integrity and overlap of image coverage, the drone performs automated aerial photography according to a preset flight path, maintaining a 60%-80% overlap rate between adjacent images.

[0053] In addition, other image acquisition methods such as satellite remote sensing, high-altitude balloons, or fixed monitoring towers can be used as supplementary means as needed. After acquiring the regional image, the large image covering the entire growth area is precisely divided into multiple raster images corresponding to the regional raster according to the same raster division standard as in step S11. Each raster image represents the crop status within a specific geographical area, and the boundary of the raster image strictly corresponds to the boundary of the regional raster for meteorological monitoring, ensuring spatial consistency for subsequent data fusion.

[0054] Subsequently, a pre-trained lodging detection network was used to analyze and identify each raster image. Multiple segmented raster images were sequentially input into the lodging detection network to automatically extract and analyze crop lodging features. The lodging detection network is built on a convolutional neural network (CNN) architecture and is specifically optimized for crop lodging detection tasks. The network structure includes multiple convolutional layers, pooling layers, and fully connected layers, enabling it to automatically learn and extract visual features of the crop from the raster images, including image features closely related to lodging status such as stem tilt angle, leaf arrangement direction, shadow distribution patterns, and texture variations.

[0055] To address the specific requirements of lodging identification, the network is designed with a multi-task learning architecture, including a lodging direction identification subnetwork and a lodging ratio estimation subnetwork, simultaneously outputting two prediction results: lodging direction and lodging ratio. The lodging direction identification subnetwork analyzes the main direction of crop lodging in the image and outputs the lodging direction as an angle value. The lodging ratio estimation subnetwork statistically analyzes the area ratio between lodged and normally standing areas in the image and outputs the lodging ratio as a percentage. To ensure the accuracy and reliability of the identification network, it is trained using a large number of sample raster images containing different lodging degrees, lodging directions, lighting conditions, and crop varieties under supervised learning conditions. The annotation information of the training samples (sample lodging identification information) is obtained through manual measurement and field surveys to ensure the accuracy and consistency of the annotation data. Through the recognition processing of all raster images, multiple lodging identification information corresponding to multiple raster regions are obtained. Each lodging identification information includes the lodging direction and lodging ratio parameters of that raster. This information is aggregated to form the lodging identification information distribution, serving as the data basis reflecting the actual lodging status of the crop at harvest time. The identification of lodging information distribution complements the aforementioned predicted lodging information distribution based on meteorological forecasts, providing accurate lodging status information based on real-time images for subsequent lodging credibility analysis and path optimization.

[0056] Furthermore, based on the predicted lodging information distribution and the identified lodging information distribution, a lodging confidence analysis is performed to obtain the lodging confidence distribution, including:

[0057] S31. Extract the predicted lodging information and identified lodging information of each grid cell within the growth area to obtain multiple sets of lodging information;

[0058] S32. Calculate the similarity between predicted lodging information and identified lodging information within each group of lodging information to obtain the lodging confidence. Specifically, calculate the similarity between the lodging direction and the lodging ratio within the predicted lodging information and identified lodging information, and obtain the lodging confidence.

[0059] S33. Combine the lodging confidence scores within multiple regional grids to obtain the lodging confidence score distribution.

[0060] In a preferred embodiment, firstly, a spatial correspondence is established between the predicted lodging information distribution and the identified lodging information distribution. Based on the unified grid division system established in the preceding steps, the two types of lodging information corresponding to each grid area within the growth region are extracted one by one. Specifically, for each grid area, the predicted lodging information (including the predicted lodging direction and the predicted lodging ratio) of that grid area is extracted from the predicted lodging information distribution, and the identified lodging information (including the identified lodging direction and the identified lodging ratio) of that grid area is extracted from the identified lodging information distribution. Through the above extraction process, multiple sets of lodging information are obtained, each set containing the predicted lodging information and the identified lodging information for the same grid location. These information sets constitute the basic dataset for subsequent similarity calculation and credibility assessment, ensuring a one-to-one correspondence between spatial locations.

[0061] Subsequently, a multi-dimensional similarity calculation method was employed to evaluate the consistency between the predicted and identified results in each group of lodging information. This calculation process consisted of two parts: lodging direction similarity calculation and lodging proportion similarity calculation. For lodging direction similarity, the angular difference between the predicted and identified lodging directions was first calculated. Considering the periodicity of angles, the smallest angle between the two was chosen as the difference value. For example, when the predicted lodging direction was 10° and the identified lodging direction was 350°, the actual angular difference was 20° instead of 340°. Then, the angular difference was converted into directional similarity: the smaller the angular difference, the closer the directional similarity was to 1; when the angular difference reached 180°, the directional similarity approached 0.

[0062] For calculating the similarity of the lodging proportion, a normalized absolute error method is used to evaluate the consistency between the predicted and identified proportions. When the two proportions are exactly equal, the similarity is 1; as the difference increases, the similarity decreases accordingly. Based on the similarity calculation results of these two dimensions, a weighted fusion method is used to calculate the overall lodging confidence. This fusion process is achieved by setting weight coefficients for directional similarity and proportion similarity, with the sum of the weight coefficients equal to 1. Generally, since the lodging direction has a more critical impact on harvesting path planning, a slightly higher weight is assigned to the similarity of the lodging direction. For example, in the standard configuration, the weight coefficient for lodging direction similarity is set to 0.6, and the weight coefficient for lodging proportion similarity is set to 0.4. In applications requiring extremely high accuracy in the lodging direction, the weight ratios can be adjusted to 0.7 and 0.3; while in applications that emphasize lodging area assessment, a balanced weight configuration of 0.5 and 0.5 can be used.

[0063] By performing the aforementioned confidence calculation process on all grid areas within the growth region, the lodging confidence level for each grid area is obtained. These discrete lodging confidence levels are organized in an orderly manner according to the spatial location of the grid areas, forming a complete lodging confidence distribution. The resulting lodging confidence distribution quantitatively assesses the reliability of lodging information in each grid area, providing a basis for subsequent operation path planning and ensuring that the path optimization process can fully consider the differences in confidence levels of lodging information in different areas.

[0064] Furthermore, harvester operation path planning is performed within the growth area. Based on the lodging confidence distribution, the lodging harvesting adaptability of the operation path is calculated, and dynamic path optimization is performed to obtain the optimal operation path, including:

[0065] S41. Plan the harvester's operating path within the growth area to obtain a first operating path;

[0066] S42. Extract the lodging ratio and lodging direction within the lodging information distribution to obtain the lodging ratio distribution and the lodging direction distribution. Combine the lodging confidence distribution to allocate planning weights to multiple regional grids to obtain the planning weight distribution.

[0067] S43. Based on the working direction of the harvester passing through multiple grid areas within the first working path, and combined with the identified lodging direction distribution and the planning weight distribution, the first lodging harvesting fitness is calculated.

[0068] S44. Continue to randomly plan the operation path and perform dynamic path optimization. After the optimization converges, retain the operation path with the highest adaptability to lodging harvesting to obtain the optimal operation path.

[0069] In a preferred embodiment, firstly, harvester path planning is performed within the growing area to generate a first working path as the starting point for optimization. This first working path planning process comprehensively considers basic constraints such as plot boundaries, obstacle distribution, and harvester turning radius, and uses traditional path planning algorithms to generate feasible working trajectories. The first working path can employ various strategies, including a parallel straight-line reciprocating mode, a spiral centripetal mode, or a segmented working mode. The parallel straight-line reciprocating mode is preferred, where the harvester operates along a parallel line, offering advantages such as high efficiency and fewer turns. The path spacing is determined based on the harvester's working width to ensure complete coverage and no overlap. The generated first working path is stored as a coordinate sequence, recording the harvester's position coordinates and direction of travel at different times, providing basic data for subsequent fitness calculations.

[0070] To reasonably reflect the importance of different regional grids during path optimization, a planning weight allocation mechanism based on multi-factor fusion is established. This mechanism comprehensively considers three key factors: lodging ratio, lodging direction consistency, and lodging confidence. First, the lodging ratio and lodging direction of each regional grid are extracted from the lodging identification information distribution, forming lodging ratio distribution and lodging direction distribution. The lodging ratio distribution reflects the area proportion of lodged crops within each grid; regions with higher lodging ratios have a greater impact on path planning. The lodging direction distribution provides information on the dominant lodging direction within each grid. Then, based on the lodging ratio distribution, a lodging ratio weight is calculated for each regional grid. Grids with higher lodging ratios are assigned larger weight values, indicating higher importance in path planning; grids with lower lodging ratios or no lodging are assigned smaller weight values. Simultaneously, a confidence weight is assigned to each regional grid according to the lodging confidence distribution. Grids with higher lodging confidence levels receive larger confidence weights, indicating more accurate and reliable lodging information in that area; grids with lower lodging confidence levels receive smaller confidence weights to avoid inaccurate information negatively impacting path planning. Subsequently, the overall planning weight for each grid area is calculated by fusing the lodging proportion weight and the confidence weight. This fusion process uses a product or weighted sum method to ensure that both the impact of lodging degree and information reliability are considered. The planning weights of all grids are organized according to spatial location, forming a planning weight distribution.

[0071] Subsequently, based on the specific trajectory of the first operating path, combined with the distribution of lodging direction identification and the planning weight distribution, the lodging harvesting fitness of this path is calculated, aiming to quantitatively evaluate the degree of matching between the operating path and the lodging condition. Specifically, firstly, the operating direction information of the harvester when passing through each grid area in the first operating path is extracted. The operating direction is determined by analyzing the harvester's trajectory within the grid and is expressed as an angle value. For each grid passed, the angle between the harvester's operating direction and the lodging direction identified by that grid is calculated. According to the principles of agricultural machinery, the harvesting effect is optimal when the operating direction and the lodging direction form a 90-degree angle, avoiding the pushing of lodged crops by the header. By comparing the closeness of the actual angle to the ideal vertical angle, the similarity of the lodging harvesting angle for each grid is calculated. Then, the similarity of the lodging harvesting angle for each grid is weighted according to the planning weight distribution to obtain the overall harvesting angle fitness. Grids with larger weights contribute more to the overall fitness, ensuring that the lodging treatment effect in important areas is given sufficient attention. In addition to harvesting angle adaptability, the operation time factor is also considered. The total operation time corresponding to the first operation path is calculated and compared with the preset ideal operation time to obtain the harvesting time adaptability. The time adaptability reflects the operation efficiency of the path, avoiding excessive sacrifice of operation efficiency in pursuit of perfect angle matching. Then, by combining the harvesting angle adaptability and the harvesting time adaptability, the first lodging harvesting adaptability of the first operation path is calculated as a comprehensive evaluation index for the path scheme.

[0072] Next, based on the calculation results of the first lodging harvesting fitness, a dynamic path optimization process is initiated, iteratively searching for operational path solutions with higher lodging harvesting fitness. For example, heuristic optimization algorithms, including genetic algorithms, particle swarm optimization, and simulated annealing, are used for path search. During the optimization process, new candidate operational paths are continuously generated. These paths are derived from existing paths through random perturbation, crossover mutation, or other operations, or generated completely randomly. For each newly generated candidate path, the calculation process in step S43 is repeated to obtain the corresponding lodging harvesting fitness. By comparing the fitness performance of different paths, higher-fitting path solutions are gradually selected and retained, while poor-performing candidate paths are eliminated. The optimization process employs a convergence judgment mechanism. When the optimal fitness value does not significantly improve in multiple consecutive iterations, or when the preset maximum number of iterations is reached, the optimization process is considered converged, and further path search is stopped. After optimization convergence, the operational path with the highest lodging harvesting fitness is selected from all evaluated candidate paths and determined as the optimal operational path. This optimal path can effectively handle lodged crops (the working direction is as perpendicular as possible to the lodging direction) while maintaining reasonable operational efficiency, providing a path guidance scheme for actual harvesting operations.

[0073] Furthermore, the lodging ratio and lodging direction within the lodging information distribution are extracted to obtain the lodging ratio distribution and the lodging direction distribution. Combined with the lodging confidence distribution, planning weights are assigned to multiple regional grids to obtain the planning weight distribution, including:

[0074] S421. Extract the lodging ratio and lodging direction of multiple grid areas within the lodging information distribution to obtain the lodging ratio distribution and the lodging direction distribution.

[0075] S422. Based on the identified lodging ratio distribution, calculate and assign multiple lodging ratio weights to multiple regional grids;

[0076] S423. Based on the collapse confidence distribution, calculate and assign multiple confidence weights to multiple regional grids;

[0077] S424. Based on the multiple collapse ratio weights and multiple reliable weights, allocate and calculate multiple planning weights for multiple regional grids.

[0078] In a preferred embodiment, firstly, the lodging information distribution is structured and analyzed to extract the two information components contained therein. Specifically, the lodging information corresponding to each grid cell within the growing area is accessed one by one, separating the lodging ratio and lodging direction. The lodging ratio is expressed as a percentage, representing the proportion of lodged crop area within each grid cell to the total crop area of ​​that grid cell, with a value ranging from 0% to 100%. The lodging ratio data of all grid cells are organized and arranged according to spatial location to form a lodging ratio distribution, which intuitively reflects the spatial variation of lodging degree throughout the growing area. The lodging direction is expressed as an angle value, representing the dominant lodging direction of crops within each grid cell, with an angle range from 0° to 360°, with due north as the 0° reference. The lodging direction data of all grid cells are organized according to their corresponding spatial location to form a lodging direction distribution, which clearly shows the spatial distribution pattern and trend of lodging direction.

[0079] Subsequently, based on the identified lodging ratio distribution, a differentiated weighting strategy based on the severity of lodging was adopted to calculate the corresponding lodging ratio weight for each raster region. This weighting process adheres to the core principle that the more severe the lodging, the higher the planning importance. Specifically, firstly, the numerical range and distribution characteristics of the identified lodging ratio distribution were analyzed to determine statistical parameters such as the maximum, minimum, and average lodging ratios. Then, a nonlinear mapping function was used to convert the lodging ratio values ​​into corresponding weight values. For example, for raster regions with high lodging ratios (e.g., lodging exceeding 70%), a larger lodging ratio weight was assigned, indicating that these areas have severe lodging problems and should be given priority in path planning to ensure that the harvesting direction can effectively handle lodged crops; for raster regions with medium lodging ratios (e.g., lodging between 30% and 70%), a moderate weight value was assigned; and for raster regions with low or no lodging (e.g., lodging below 30%), a smaller weight value was assigned, indicating that these areas have relatively weak constraints on path planning. By assigning weights, multiple lodging ratio weights are obtained that correspond one-to-one with each region grid. These weight values ​​quantify the differences in the importance of lodging treatment in different regions.

[0080] Simultaneously, based on the aforementioned lodging confidence distribution, corresponding confidence weights are assigned to each regional grid to ensure sufficient consideration of information reliability during path planning. This weight allocation process is based on the core principle that higher lodging confidence corresponds to higher planning dependence. Specifically, firstly, the confidence value distribution of each grid in the lodging confidence distribution is analyzed to identify the spatial distribution patterns of high-confidence, medium-confidence, and low-confidence regions. Subsequently, a piecewise linear or smooth curve mapping method is used to convert the confidence values ​​into corresponding confidence weights. For example, grids with high confidence (e.g., confidence exceeding 0.8) are assigned larger confidence weights, indicating that the lodging information in this region is highly reliable and should be given sufficient trust and attention in path planning; grids with medium confidence (e.g., confidence between 0.4 and 0.8) are assigned medium confidence weights; and grids with low confidence (e.g., confidence below 0.4) are assigned smaller confidence weights, indicating that the lodging information in this region has significant uncertainty, and a conservative strategy should be adopted in path planning to avoid over-reliance on potentially inaccurate information. This weight allocation mechanism yields multiple reliable weights for each region's grid. These weights reflect the differences in the reliability of landslide information in different regions, providing a basis for information quality assessment in subsequent comprehensive weight calculations.

[0081] Subsequently, based on the multiple lodging ratio weights and multiple reliability weights obtained in the aforementioned steps, a multi-factor fusion method is used to calculate the final planning weights for each regional grid. This fusion process aims to balance the impact of two key factors: the severity of lodging and the reliability of information. Specifically, for each regional grid, its corresponding lodging ratio weight and reliability weight are mathematically fused. The fusion method can adopt a product mode or a weighted sum mode: the product mode obtains a comprehensive weight by multiplying the two types of weights. This method emphasizes the synergistic effect of the two factors, and the comprehensive weight will only reach a large value when both the lodging ratio and reliability are high; the weighted sum mode linearly combines the two types of weights by setting a fusion coefficient. This method allows the advantages of one factor to partially compensate for the shortcomings of the other. Preferably, special case handling strategies can also be considered: for grids with a high lodging ratio but low reliability, conservative weight assignments are used to avoid making incorrect path decisions based on unreliable information; for grids with a low lodging ratio but high reliability, appropriate weights are assigned to ensure that reliable information is used reasonably. Through comprehensive calculation, multiple planning weights corresponding to each regional grid are obtained. These planning weights are organized and arranged according to the spatial location of the grid to form a complete planning weight distribution, which comprehensively reflects the importance of each region in the path planning. It takes into account both the urgency of landslide treatment and the reliability of information, and provides a weight basis for subsequent path fitness calculation and optimization decision-making.

[0082] Furthermore, based on the harvester's operating direction through multiple grid areas within the first operating path, and combined with the identified lodging direction distribution and planning weight distribution, a first lodging harvesting fitness is calculated, including:

[0083] S431. Obtain the working direction of the harvester passing through multiple area grids within the first working path, and obtain multiple first working directions;

[0084] S432. Based on the identified lodging direction of multiple area grids within the identified lodging direction distribution, obtain the angle between the lodging direction and the multiple first working directions, obtain multiple first working angles, and calculate the similarity with the preset vertical angle to obtain multiple first lodging harvesting angle similarity.

[0085] S433. Based on the planning weight distribution, the similarity of the multiple first lodging harvesting angles is calculated by weighting to obtain the first harvesting angle fitness.

[0086] S434. Obtain the first operation time of the first operation path, calculate the ratio of the preset operation time to the first operation time, and obtain the first harvesting time adaptability;

[0087] S435. Calculate the first lodging harvesting adaptability based on the first harvesting angle adaptability and the first harvesting time adaptability.

[0088] In a preferred embodiment, firstly, the first working path is analyzed in detail to extract the specific working direction information of the harvester as it passes through each grid area. Since the harvester traverses multiple grid areas along the path, it is necessary to determine the local working direction of the harvester within each grid area. Specifically, the dominant direction of travel for each segment is calculated by analyzing the harvester's trajectory within each grid area. For grid areas completely traversed by the harvester, the working direction is determined by the average direction of the trajectory within that grid area; for grid areas only partially traversed by the harvester, the working direction is calculated based on the trajectory direction of the traversed portion. The working direction is represented by an angle value, using the same angle reference system as the lodging direction, i.e., with true north as the 0° reference. Through the above extraction process, multiple first working directions corresponding one-to-one with the grid areas traversed by the harvester are obtained.

[0089] Then, based on the distribution of lodging directions, the identified lodging directions corresponding to each grid area traversed by the harvester are obtained, and angle matching analysis is performed with the corresponding first working direction. This analysis aims to evaluate the rationality of the geometric relationship between the harvester's working direction and the lodging direction. For each grid traversed by the harvester, the angle between the first working direction and the identified lodging direction within that grid is calculated. Considering the periodicity of angles, the minimum angle between the two directions is selected as the actual angle value, ensuring that the angle range is between 0° and 180°. Through the above calculation, multiple corresponding first working angles are obtained. Subsequently, based on the principles of agricultural machinery, using a 90° vertical angle as the ideal working angle benchmark, the similarity between the actual angle of each grid and the preset vertical angle is calculated. The similarity calculation adopts a normalization method: when the actual angle equals 90°, the similarity is 100%; when the actual angle deviates from 90°, the similarity decreases accordingly. Specifically, the calculation method is: subtract the ratio of the difference between the actual angle and the ideal vertical angle to the maximum possible deviation angle from 1. For example, when the angle between the working direction and the lodging direction is 80°, the deviation from the 90° perpendicular angle is 10°. The similarity calculation is 1 minus 10° divided by 90°, resulting in approximately 89%. By performing the above calculation process on all the grids passed by the harvester, multiple corresponding first lodging harvesting angle similarities are obtained. These values ​​quantify the degree of matching between the working direction and the lodging direction within each grid.

[0090] Considering the varying importance of different regional grids in path planning, the similarity of lodging harvesting angles for each grid is differentiated based on the planning weight distribution, and a weighted comprehensive harvesting angle fitness is calculated. The planning weights corresponding to each grid are extracted from the planning weight distribution; these weights comprehensively reflect the severity of lodging and the reliability of information for each grid. Subsequently, the first lodging harvesting angle similarity of each grid is multiplied by its corresponding planning weight to obtain the weighted angle contribution value for each grid. The weighted angle contribution values ​​of all grids are normalized and then summed to obtain the first harvesting angle fitness for the entire first operation path. This first harvesting angle fitness comprehensively reflects the overall effectiveness of the first operation path in handling lodged crops; important grids with higher weights have a more significant impact on the fitness.

[0091] In addition to angle matching effectiveness, the time efficiency of the first work path is evaluated. First, the first work time corresponding to the first work path is determined, taking into account factors such as total path length, harvester speed, number of turns, and turn time. Simultaneously, a preset work time is established as a benchmark for time efficiency evaluation. This preset work time represents the ideal time standard under current working conditions, typically calculated based on the theoretical shortest work time under efficient path modes such as straight-line round trips, reflecting the expected time efficiency of the work area under ideal path planning. The ratio of the preset work time to the first work time is calculated to obtain the first harvesting time fitness. Specifically, the first harvesting time fitness is equal to the quotient of the preset work time divided by the first work time. When the first work time equals the preset work time, the ratio is 1, indicating optimal time fitness and the path's time efficiency reaches the ideal standard; when the first work time exceeds the preset work time, the ratio is less than 1, and the time fitness decreases accordingly, indicating that the path requires a longer work time and the time efficiency is below the ideal level; when the first work time is less than the preset work time, the ratio is greater than 1, indicating that the path's time efficiency exceeds the ideal expectation. The fitness calculation of the first harvesting time was used to quantitatively evaluate the performance of the first operation path in terms of operation efficiency, providing a time-dimensional evaluation basis for the comprehensive fitness assessment.

[0092] Subsequently, the first harvesting angle fitness and the first harvesting time fitness are fused to calculate the first lodging harvesting fitness, which serves as the comprehensive evaluation index for the first operational path. The fusion process employs a weighted combination method, balancing the two factors by setting weight coefficients for angle fitness and time fitness. Typically, since lodging treatment effectiveness is the primary objective of path planning, angle fitness is assigned a higher weight (e.g., 0.7), while time fitness is assigned a relatively lower weight (e.g., 0.3). However, the weight configuration can be adjusted according to actual operational needs: in scenarios with severe lodging and extremely high treatment quality requirements, the angle fitness weight can be increased to 0.8 or higher; in scenarios with strict operational time constraints, the time fitness weight can be appropriately increased to balance the two objectives. Through fusion calculation, the first lodging harvesting fitness is obtained, comprehensively reflecting the overall performance of the first operational path in terms of lodging treatment effectiveness and operational efficiency, providing a quantitative evaluation benchmark for subsequent dynamic path optimization.

[0093] Example 2, as Figure 2 As shown, based on the same inventive concept as the computer vision-based harvester operation path dynamic planning method provided in Embodiment 1, this embodiment of the invention also provides a computer vision-based harvester operation path dynamic planning system, including:

[0094] The lodging prediction module 11 is used to monitor and acquire abnormal meteorological data and growth stages within the growth area during crop growth, obtain abnormal meteorological data sequences and growth stage information sequences, perform crop lodging prediction, and obtain the predicted lodging information distribution.

[0095] The visual recognition module 12 is used to collect regional images within the growing area during crop harvesting, and to use computer vision for recognition to obtain information on the distribution of lodging.

[0096] The credibility analysis module 13 is used to perform lodging credibility analysis based on the predicted lodging information distribution and the identified lodging information distribution to obtain the lodging credibility distribution;

[0097] The path planning module 14 is used to plan the harvester's working path within the growth area. It calculates the lodging harvesting adaptability of the working path based on the lodging confidence distribution, performs dynamic path optimization, and obtains the optimal working path as the path planning result. The lodging harvesting adaptability is calculated based on the working direction within the working path and the lodging direction within the lodging information.

[0098] Furthermore, the lodging prediction module 11 includes the following execution steps:

[0099] During crop growth, meteorological data from multiple regional grids within the growth area are monitored and acquired.

[0100] When the monitored meteorological data meets the abnormal meteorological conditions, the abnormal meteorological data and the corresponding crop growth stage at the time are recorded and obtained, and the abnormal meteorological data sequence and growth stage information sequence of multiple regional grids are obtained.

[0101] The abnormal meteorological data sequences and growth stage information sequences of multiple regional grids are input into the lodging prediction network, and multiple predicted lodging information of multiple regional grids are predicted and output to obtain the distribution of predicted lodging information. Each predicted lodging information includes the lodging direction and the lodging ratio.

[0102] Furthermore, the lodging prediction network is obtained based on machine learning training, and the training steps include:

[0103] Based on crop lodging monitoring data, a set of abnormal meteorological data sequences and a set of sample growth stage information sequences were collected. The lodging direction and lodging ratio of crops under different abnormal meteorological data sequences and sample growth stage information sequences were collected and labeled to obtain a set of sample lodging information.

[0104] Construct a lodging prediction network based on machine learning;

[0105] The accuracy of the lodging prediction network is supervised and tested using the sample abnormal meteorological data sequence set, sample growth stage information sequence set, and sample lodging information set, and training is completed after meeting the requirements.

[0106] Furthermore, the visual recognition module 12 includes the following execution steps:

[0107] During crop harvesting, regional images of the growing area are acquired and divided into multiple raster images of multiple regions.

[0108] Multiple raster images are input into the landslide recognition network to identify multiple landslide information, which are used as the landslide information distribution. The landslide recognition network is constructed based on computer vision and is trained using sample raster images and labeled sample landslide information.

[0109] Furthermore, the credibility analysis module 13 includes the following execution steps:

[0110] Extract the predicted lodging information and identified lodging information of each grid cell within the growth area to obtain multiple sets of lodging information;

[0111] Calculate the similarity between predicted and identified lodging information within each group of lodging information to obtain the lodging confidence score. Specifically, calculate the similarity between the lodging direction and the lodging ratio within the predicted and identified lodging information, and obtain the lodging confidence score.

[0112] By combining the lodging confidence scores within multiple regional grids, the lodging confidence score distribution is obtained.

[0113] Furthermore, the path planning module 14 includes the following execution steps:

[0114] Within the growth area, a harvester operation path is planned to obtain a first operation path;

[0115] Extract the lodging ratio and lodging direction within the lodging information distribution to obtain the lodging ratio distribution and the lodging direction distribution. Combine the lodging confidence distribution to assign planning weights to multiple regional grids to obtain the planning weight distribution.

[0116] Based on the working direction of the harvester passing through multiple grid areas within the first working path, and combined with the identified lodging direction distribution and the planning weight distribution, the first lodging harvesting fitness is calculated.

[0117] Continue to randomly plan the operation path and perform dynamic path optimization. After the optimization converges, retain the operation path with the highest adaptability to lodging harvesting to obtain the optimal operation path.

[0118] Furthermore, the route planning module 14 also includes the following execution steps:

[0119] Extract the lodging ratio and lodging direction of multiple grid areas within the lodging information distribution to obtain the lodging ratio distribution and the lodging direction distribution.

[0120] Based on the identified lodging ratio distribution, multiple lodging ratio weights are calculated for multiple regional grids;

[0121] Based on the collapse confidence distribution, multiple confidence weights are calculated for multiple regional grids;

[0122] Based on the multiple collapse ratio weights and multiple reliable weights, multiple planning weights for multiple regional grids are calculated and allocated.

[0123] Furthermore, the route planning module 14 also includes the following execution steps:

[0124] Obtain the working direction of the harvester passing through multiple grid areas within the first working path to obtain multiple first working directions;

[0125] Based on the identified lodging direction of multiple grid areas within the lodging direction distribution, the angle between the lodging direction and the multiple first working directions is obtained, multiple first working angles are obtained, and the similarity with the preset vertical angle is calculated to obtain multiple first lodging harvesting angle similarity.

[0126] Based on the planning weight distribution, the similarity of the multiple first lodging harvesting angles is weighted and calculated to obtain the fitness of the first harvesting angle;

[0127] Obtain the first operation time of the first operation path, calculate the ratio of the preset operation time to the first operation time, and obtain the first harvesting time adaptability;

[0128] The first lodging harvesting fitness is calculated based on the first harvesting angle fitness and the first harvesting time fitness.

[0129] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0130] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0131] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0132] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0133] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0134] Although preferred embodiments of the invention have been described, those skilled in the art, once they have learned the basic inventive concept, can make other changes and modifications to these embodiments.

[0135] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of this invention and its equivalents, this invention also intends to include these modifications and variations.

Claims

1. A dynamic planning method for harvester operation paths based on computer vision, characterized in that, The method includes: During crop growth, abnormal meteorological data and growth stages within the growth area are monitored and acquired to obtain abnormal meteorological data sequences and growth stage information sequences, thereby enabling crop lodging prediction and obtaining the distribution of predicted lodging information. During crop harvesting, regional images of the growing area are collected, and computer vision is used to identify the distribution of lodging information. Based on the predicted lodging information distribution and the identified lodging information distribution, a lodging confidence analysis is performed to obtain the lodging confidence distribution; Harvester operation path planning is performed within the growth area. Based on the lodging confidence distribution, the lodging harvesting fitness of the operation path is calculated. The path is dynamically optimized to obtain the optimal operation path, which is taken as the path planning result. The lodging harvesting fitness is calculated based on the operation direction within the operation path and the lodging direction within the lodging information.

2. The harvester operation path dynamic planning method based on computer vision according to claim 1, characterized in that, During crop growth, abnormal meteorological data and growth stages within the growth area are monitored and acquired to obtain abnormal meteorological data sequences and growth stage information sequences. This data is then used to predict crop lodging and obtain the distribution of predicted lodging information, including: During crop growth, meteorological data from multiple regional grids within the growing area are monitored and acquired. When the monitored meteorological data meets the abnormal meteorological conditions, the abnormal meteorological data and the corresponding crop growth stage at the time are recorded and obtained, and the abnormal meteorological data sequence and growth stage information sequence of multiple regional grids are obtained. The abnormal meteorological data sequences and growth stage information sequences of multiple regional grids are input into the lodging prediction network, and multiple predicted lodging information of multiple regional grids are predicted and output to obtain the distribution of predicted lodging information. Each predicted lodging information includes the lodging direction and the lodging ratio.

3. The harvester operation path dynamic planning method based on computer vision according to claim 2, characterized in that, The lodging prediction network is obtained through machine learning training, and the training steps include: Based on crop lodging monitoring data, a set of abnormal meteorological data sequences and a set of sample growth stage information sequences were collected. The lodging direction and lodging ratio of crops under different abnormal meteorological data sequences and sample growth stage information sequences were collected and labeled to obtain a set of sample lodging information. Construct a lodging prediction network based on machine learning; The accuracy of the lodging prediction network is supervised and tested using the sample abnormal meteorological data sequence set, sample growth stage information sequence set, and sample lodging information set, and training is completed after meeting the requirements.

4. The harvester operation path dynamic planning method based on computer vision according to claim 1, characterized in that, During crop harvesting, images of the growing area are collected, and computer vision is used to identify lodging information, including: During crop harvesting, regional images of the growing area are acquired and divided into multiple raster images of multiple regions. Multiple raster images are input into the landslide recognition network to identify multiple landslide information, which are used as the landslide information distribution. The landslide recognition network is constructed based on computer vision and is trained using sample raster images and labeled sample landslide information.

5. The harvester operation path dynamic planning method based on computer vision according to claim 1, characterized in that, Based on the predicted lodging information distribution and the identified lodging information distribution, a lodging confidence analysis is performed to obtain the lodging confidence distribution, including: Extract the predicted lodging information and identified lodging information of each grid cell within the growth area to obtain multiple sets of lodging information; Calculate the similarity between predicted and identified lodging information within each group of lodging information to obtain the lodging confidence score. Specifically, calculate the similarity between the lodging direction and the lodging ratio within the predicted and identified lodging information, and obtain the lodging confidence score. By combining the lodging confidence scores within multiple regional grids, the lodging confidence score distribution is obtained.

6. The harvester operation path dynamic planning method based on computer vision according to claim 1, characterized in that, Harvester operation path planning is performed within the growth area. Based on the lodging confidence distribution, the lodging harvesting adaptability of the operation path is calculated. Dynamic path optimization is then performed to obtain the optimal operation path, including: Within the growth area, a harvester operation path is planned to obtain a first operation path; Extract the lodging ratio and lodging direction within the lodging information distribution to obtain the lodging ratio distribution and the lodging direction distribution. Combine the lodging confidence distribution to assign planning weights to multiple regional grids to obtain the planning weight distribution. Based on the working direction of the harvester passing through multiple grid areas within the first working path, and combined with the identified lodging direction distribution and the planning weight distribution, the first lodging harvesting fitness is calculated. Continue to randomly plan the operation path and perform dynamic path optimization. After the optimization converges, retain the operation path with the highest adaptability to lodging harvesting to obtain the optimal operation path.

7. The harvester operation path dynamic planning method based on computer vision according to claim 6, characterized in that, Extract the lodging ratio and lodging direction within the lodging information distribution to obtain the lodging ratio distribution and the lodging direction distribution. Combined with the lodging confidence distribution, assign planning weights to multiple regional grids to obtain the planning weight distribution, including: Extract the lodging ratio and lodging direction of multiple grid areas within the lodging information distribution to obtain the lodging ratio distribution and the lodging direction distribution. Based on the identified lodging ratio distribution, multiple lodging ratio weights are calculated for multiple regional grids; Based on the collapse confidence distribution, multiple confidence weights are calculated for multiple regional grids; Based on the multiple collapse ratio weights and multiple reliable weights, multiple planning weights for multiple regional grids are calculated and allocated.

8. The harvester operation path dynamic planning method based on computer vision according to claim 6, characterized in that, Based on the harvester's operating direction through multiple grid areas within the first operating path, and combined with the identified lodging direction distribution and planning weight distribution, the first lodging harvesting fitness is calculated, including: Obtain the working direction of the harvester passing through multiple grid areas within the first working path to obtain multiple first working directions; Based on the identified lodging direction of multiple grid areas within the lodging direction distribution, the angle between the lodging direction and the multiple first working directions is obtained, multiple first working angles are obtained, and the similarity with the preset vertical angle is calculated to obtain multiple first lodging harvesting angle similarity. Based on the planning weight distribution, the similarity of the multiple first lodging harvesting angles is weighted and calculated to obtain the fitness of the first harvesting angle; Obtain the first operation time of the first operation path, calculate the ratio of the preset operation time to the first operation time, and obtain the first harvesting time adaptability; The first lodging harvesting fitness is calculated based on the first harvesting angle fitness and the first harvesting time fitness.

9. A harvester operation path dynamic planning system based on computer vision, characterized in that, A method for implementing the computer vision-based dynamic planning method for harvester operation paths as described in any one of claims 1 to 8, the method comprising: The lodging prediction module is used to monitor and acquire abnormal meteorological data and growth stages within the crop growth area during the crop growth process, obtain abnormal meteorological data sequences and growth stage information sequences, perform crop lodging prediction, and obtain the predicted lodging information distribution. The visual recognition module is used to collect regional images of the growing area during crop harvesting, and to use computer vision to identify the distribution of lodging information. The credibility analysis module is used to perform lodging credibility analysis based on the predicted lodging information distribution and the identified lodging information distribution, and to obtain the lodging credibility distribution. The path planning module is used to plan the harvester's operating path within the growth area. Combining the lodging confidence distribution, it calculates the lodging harvesting fitness of the operating path, performs dynamic path optimization, and obtains the optimal operating path as the path planning result. The lodging harvesting fitness is calculated based on the operating direction within the operating path and the lodging direction within the lodging information.

Images