Crop yield estimation method based on unmanned aerial vehicle

Abstract

The invention discloses a crop yield estimation method based on an unmanned aerial vehicle, and belongs to the technical field of agricultural detection and intelligent agriculture, and the method comprises the steps: employing the unmanned aerial vehicle technology, selecting YOLOv10n as a target detection model, selecting a BOT-SORT algorithm as a basic model of multi-target tracking, fusing the YOLOv10n model and the BOT-SORT algorithm, and carrying out the path optimization of a black wing plinary algorithm. According to the invention, the flight path of the unmanned aerial vehicle is intelligently planned, a comprehensive data set is constructed, ripe fruits of crops are accurately identified and tracked, and the accuracy and response speed of detection operation are improved. Accurate counting of crop fruits in static images and real-time tracking of crop fruit changes in dynamic video data are ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of agricultural detection and smart agriculture, and specifically is a crop yield estimation method based on unmanned aerial vehicle (UAV). Background Art

[0002] In the rapidly developing modern agriculture, accurate assessment of crop yields has become a key step in improving agricultural production efficiency and making decisions.

[0003] Existing technical means mainly rely on time-consuming and labor-intensive manual sampling and subjective judgment, but these methods are difficult to meet the needs of modern agriculture in terms of efficiency and accuracy. Therefore, this field has begun to explore the integrated application of drone technology and advanced algorithms to achieve more efficient and accurate monitoring. Although drones have shown potential in agricultural monitoring due to their flexibility and wide monitoring range, as the technology is still under development and updating, it is still insufficient in terms of crop identification accuracy and comprehensiveness of data collection.

[0004] Especially in terms of target detection algorithms, although the YOLOv10 algorithm has been widely used in the field of image processing for its fast and accurate recognition capabilities, it still faces the need for further optimization in the field of agricultural yield estimation, especially in the real-time and accuracy of dynamic monitoring and crop identification. In addition, although the black kite optimization algorithm has shown its advantage in simulating natural predation behavior in path planning, its application in drone flight path optimization is still in the exploratory stage. Similarly, although multi-target tracking technology is crucial for dynamic monitoring of crop growth, its practical application in the field of agricultural monitoring is still limited. Summary of the invention

[0005] In order to solve the technical problems of the above-mentioned UAVs in dynamic monitoring and crop identification, the purpose of the present invention is to provide a crop yield estimation method based on UAV, which achieves intelligent planning of the UAV's flight path, builds a comprehensive data set, accurately identifies and tracks ripe fruits of crops, improves the accuracy and response speed of detection operations, ensures accurate counting of crop fruits in static images and real-time tracking of crop fruit changes in dynamic video data, by using UAV technology, target detection algorithm YOLOv10, path optimization Black Kite algorithm, and multi-target tracking technology.

[0006] The technical solution adopted by the present invention to achieve the above technical purpose is: Step 1: Capture the geographical features of the target agricultural area through aerial scanning by drones and build a regional map; Step 2: The flight path of the drone is intelligently planned by combining the regional map data in step 1 with the black kite optimization algorithm to achieve seamless coverage of the agricultural area; Step 3: Use the high-definition images and video data collected by the drone in step 1 to build a dataset of crop static morphology and growth dynamics; Step 4: Select the YOLOv10n algorithm and the BOT-SORT algorithm, compare the performance of the YOLOv10n model and the BOT-SORT algorithm, select YOLOv10n as the target detection model, select the BOT-SORT algorithm as the basic model for multi-target tracking, and perform the training process by fusing the YOLOv10n model with the BOT-SORT algorithm. Initialize the model parameters according to the characteristics of the algorithm, adjust the model configuration according to the characteristics of the data set, and use the gradient descent algorithm and its variants to optimize the model in order to minimize the loss function. Step 5: By deploying the model trained and verified by the algorithm in step 4 to high-performance servers and portable mobile devices, the data set constructed in step 3 can be processed and intelligently analyzed in real time; Step 6: Evaluate the economic feasibility of the adopted technical solution and the degree of optimization of resource allocation by conducting a cost-benefit analysis on step 5.

[0007] Further optimization, the step 1 includes the following steps: Step 1 (a): Optimize flight parameters according to the height of crops in the target agricultural area, the undulation of terrain and the accuracy requirements of the expected map, and calculate and adjust the flight altitude and speed of the drone; Step 1 (b): Plan the flight path for the drone using automated intelligent software or manually set it up; Step 1 (c): Calibrate the high-precision sensors on the drone before performing the flight mission; Step 1 (d): operate the drone to perform flight missions along the predetermined route and conduct real-time monitoring and collection of regional map data; Step 1 (e): Real-time or near real-time processing of the raw data collected by the drone, including data synchronization, preliminary correction and effective storage management; Step 1 (f): Use photogrammetry and remote sensing software technology to stitch the collected image data to generate orthophotos, and create digital elevation models (DEMs) and digital surface models (DSMs) according to specific needs; Step 1(g): Verify the accuracy and reliability of the generated regional map by careful comparison with ground control points or known map data; Step 1 (h): Optimize the regional map to enhance the display of specific terrain features or crop areas according to the specific needs of crop yield estimation, and output the map into a format suitable for further analysis and application; Step 1 (i): Data integration of the optimized topographic map with other Geographic Information System (GIS) data layers.

[0008] Further optimization, the step 2 includes the following steps: Step 2 (a): According to the geographical features of the agricultural area and the monitoring targets presented in the topographic map, the parameters of the UAV’s flight altitude, speed and sensor configuration are set; Step 2 (b): Based on the UAV flight parameters and terrain map in step 1, the flight path is generated through the algorithm and the initial population is formed. The initialization process is implemented by the following formula: BK ij =BK lb +rand×(BK ub -BK lb ), Among them BK ij is the position of the i-th black-winged kite in the j-th dimension, BK lb and BK ub are the lower and upper bounds of the j-th dimension, respectively, and rand is a random number in the range [0, 1]; Step 2 (c): Perform fitness evaluation on each initial path to quantify its compliance with the comprehensive coverage of the monitoring target; Step 2 (d): Use the black kite optimization algorithm to continuously optimize the flight path through an iterative process. Each iteration includes the following two main parts: attack behavior and migration behavior; Step 2 (e): Among the paths that have undergone multiple rounds of optimization, select the path with the highest fitness as the execution plan; Step 2 (f): Incorporate real-time obstacle avoidance strategy into path planning; Step 2(g): Verify the selected optimal flight path through simulated flight or field testing.

[0009] Further optimization, in step 2 (d), the attack behavior is expressed as follows: Where n is a coefficient that changes according to the number of iterations t and the total number of iterations T, r is a random number in the range [0, 1], and p is a constant 0.9; The migration behavior is expressed as follows: Where XLeaderPos is the current leader's position, cauchy is the Cauchy mutation, and m is a coefficient calculated from a random number r.

[0010] Further optimization, the step three includes the following steps: Step 3 (a): Clarify the goal of dataset construction and collect images and videos of fruits of diverse crops, covering key growth stages from fruiting to maturity; Step 3 (b): Select data sources, giving priority to raw images and video data directly collected by drones, while integrating existing agricultural databases and ground sensor data; Step 3 (c): Optimize the configuration of data collection tools and equipment; Step 3 (d): The drone performs the flight mission according to the path planned in step 2, systematically collects crop data, and organizes and stores them in detail according to varieties, growth stages and collection conditions; Step 3 (e): Detailed annotation of the collected images and videos, recording key information such as crop varieties, growth stages, and health conditions, and adding metadata such as timestamps and locations; Step 3 (f): Divide the dataset into training set, test set and validation set; Step 3 (g): Use a variety of image processing techniques to enhance the data, simulate variable growth conditions and collection environments, and improve the model's generalization ability to different environments; Step 3 (h): Conduct in-depth multi-dimensional data analysis to study the morphological characteristics and growth dynamics of crop fruits, including fruit size, shape, density and their changing trends over time; Step 3(i): Regularly update and maintain the dataset, and continue to add images and videos of crops at different growth stages.

[0011] Further optimization, the step 4 includes the following steps: Step 4 (a): Study the YOLOv10n target detection algorithm and the BOT-SORT multi-target tracking algorithm, and evaluate their performance on the dataset; Step 4 (b): Compare the performance of YOLOv10n and BOT-SORT algorithms through the key indicators of mean average precision mAP, mean recall rate mAR, and frames per second FPS. The calculation method of mean average precision mAP, mean recall rate mAR, and frames per second FPS is shown in the following formula: Among them, AP c is the average precision calculated for each category, N is the total number of categories in the detection task, and Recall c is the recall rate of the cth category, the total number of frames is the total number of video frames processed by the algorithm, and the total processing time is the total time required for the algorithm to process these frames; Step 4 (c): By comparing the performance of the YOLOv10n model and the BOT-SORT algorithm, YOLOv10n is selected as the target detection model; Step 4 (d): By comparing the performance of the YOLOv10n model and the BOT-SORT algorithm, the BOT-SORT algorithm is selected as the basic model for multi-target tracking; Step 4 (e): The YOLOv10n model is integrated with the BOT-SORT algorithm. The YOLOv10n model is responsible for real-time processing of images and videos to identify crop fruits, while the BOT-SORT algorithm uses this information for continuous tracking to ensure stable tracking even in the case of occlusion or intersection. Step 4(f): Evaluate the performance of the fusion model in actual crop monitoring through field flight tests; Step 4 (g): Integrate and import the constructed dataset containing static and dynamic characteristics of crop fruits into the model training environment; Step 4 (h): Divide the dataset into training subset, validation subset and test subset; Step 4 (i): Initialize the model parameters according to the characteristics of YOLOv10n and BOT-SORT algorithms, and adjust the model configuration according to the characteristics of the dataset; Step 4 (j): Train the model to learn the key features of crop fruits and use the gradient descent algorithm and its variants to optimize the model and minimize the loss function; Step 4 (k): Based on the performance feedback of the validation subset, adjust the hyperparameters such as learning rate, batch size, and number of iterations; Step 4(l): Regularly evaluate the model performance on the validation subset using metrics such as accuracy, precision, and recall. Step 4 (m): Implement an early stopping strategy to avoid overfitting and stop training when the performance on the validation set no longer improves; Step 4(n): Fine-tune the model based on the evaluation results, including optimizing the network structure or retraining some network layers; Step 4 (o): Perform final evaluation on the test subset to verify generalization ability. Further optimization, in step 4 (c), the category and position of the target frame are calculated as follows: C(x,y)={(x c ,y c ,w,h,p c )1,(x c ,y c ,w,h,p c )2,...,(x c ,y c ,w,h,p c ) B}, Among them, C(x,y) is the category and location of B bounding boxes predicted by each feature map cell, (xc ,y c ,w,h) represents the position of the target box, (x c ,y c ) is the offset of the center of the bounding box relative to the cell, w and h are the width and height of the bounding box, and p c The class probabilities predicted for each bounding box.

[0012] Further optimization, in step 4 (d), during the target tracking process of the BOT-SORT algorithm, the target matching formula is as follows: The Kalman filter state for predicting the target state can be given by the following formula: in, is the predicted state at time t-1, F is the state transfer matrix, B is the control input matrix, ut is the control input, P t|t-1 is the covariance matrix of the predicted state, Q is the process noise covariance matrix, The Kalman filter state update of the target state is as follows: in, is the predicted state at time t for time t, z t is the observation vector at time t, H is the observation matrix, K t is the Kalman gain at time t, R is the observation noise covariance matrix, and I is the identity matrix.

[0013] Further optimization, the step five includes the following steps: Step 5 (a): Deploy the processed model to hardware facilities, i.e., different platforms and devices, and ensure that the deployed model runs stably on different platforms and devices; Step 5 (b): The drone flies along a predetermined path, collects image and video data of crop fruits in real time, and transmits the data seamlessly to a server or mobile device; Step 5 (c): Preprocess the raw image and video data transmitted by the drone, including format conversion, size adjustment, and color correction; Step 5 (d): Use the deployed model to accurately analyze static images, identify and count the number of fruits in each image, and provide basic data for yield estimation; Step 5 (e): Perform dynamic tracking on the video data, use the model to capture the changes of fruits at different time points, analyze the dynamic trend of yield, and provide a basis for predicting the yield changes during crop growth; Step 5 (f): Combine real-time processed data and model output to make real-time production forecasts, and continuously adjust and optimize the forecast model through historical data; Step 5 (g): Use charts, maps or other visualization tools to visually display the forecast and analysis results; Step 5 (h): Aggregate static and dynamic yield data and conduct a comprehensive analysis to generate a detailed report containing yield estimates, regional yield distribution, and yield fluctuations.

[0014] Further optimized, the step six (cost-benefit analysis) includes the following steps: Step 6 (a): Comprehensively evaluate the cost of the UAV system, sensors, data processing algorithms, and their hardware and software; Step 6(b): Based on the results of the cost assessment, select low-cost and efficient UAV and sensor components; Step 6 (c): Use model conversion technologies such as TensorRT and ONNX to convert the trained model into a format suitable for low-cost hardware and optimize the deployment strategy; Step 6 (d): Regularly adopt transfer learning and incremental learning methods to continuously update and maintain the model based on the latest collected data; Step 6 (e): Regularly perform performance testing and monitoring, evaluate the performance of the model on actual hardware through test cases and data, and conduct comparative analysis with competing products on the market; Step 6(f): Develop and implement a maintenance plan for drones and sensors to regularly inspect and repair or replace hardware; Step 6(g): Comprehensively evaluate the economic benefits of the yield estimation method, including improved yield forecast accuracy, reduced manual inspection costs, and optimized agricultural resource allocation; Step 6 (h): Determine the return on investment by calculating and comparing the system's costs and benefits, providing quantitative data support for the feasibility of the project; Step 6 (i): Prepare a cost-benefit analysis report to provide comprehensive data support for decision makers and evaluate and continuously improve the economic benefits of crop yield estimation methods.

[0015] Compared with the prior art, the present invention has the following beneficial effects:

[0016] 1) The present invention selects YOLOv10n as the target detection model and the BOT-SORT algorithm as the basic model for multi-target tracking. The YOLOv10n model is integrated with the BOT-SORT algorithm to perform a customized training process, initialize the model parameters according to the algorithm characteristics, adjust the model configuration according to the characteristics of the data set, and use the gradient descent algorithm and its variants to optimize the model in order to minimize the loss function, so that the model can accurately identify and track crop fruits, significantly improving the accuracy and response speed of the detection operation;

[0017] 2) The high-precision sensors carried by the drones in the present invention conduct detailed aerial scanning, accurately capture the geographical features of the target agricultural area, and generate detailed and high-resolution regional maps, laying a solid spatial data foundation for subsequent accurate monitoring and estimation;

[0018] 3) The topographic map data in the present invention is combined with the black kite optimization algorithm to intelligently plan the flight path of the drone, thereby achieving seamless coverage of the crop area. The algorithm optimizes flight efficiency and safety, ensuring the efficiency of the data collection process and the reliability of the drone operation;

[0019] 4) The high-definition images and video data collected by the drone in the present invention construct a data set of static morphology and growth dynamics of crops, which records in detail the static morphology and growth dynamics of crop fruits, and provides multi-dimensional, high-quality input data for the training of deep learning models. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is the overall flow chart of the present invention;

[0021] Figure 2 This is a flow chart of full coverage path planning based on the black kite optimization algorithm of the present invention;

[0022] Figure 3 It is a flowchart of the complete data flow in the BOT-SORT multi-target tracking algorithm of the present invention;

[0023] Figure 4 This is a cross-line counting chart used for wheat ear counting according to the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments that are easily changed or replaced by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0025] A method for estimating crop yield based on drones, such as Figure 1 As shown, Step 1: Capture the geographical features of the target agricultural area through aerial scanning by drones, build regional maps, and generate detailed and high-resolution topographic maps, laying a solid spatial data foundation for subsequent accurate monitoring and estimation; Step 2: The flight path of the drone is intelligently planned by combining the regional map data in step 1 with the black kite optimization algorithm to achieve seamless coverage of the field area. The algorithm optimization takes flight efficiency and safety into consideration to ensure the efficiency of the data collection process and the reliability of the drone operation. Step 3: Use the high-definition images and video data collected by the drone in step 1 to build a dataset of static morphology and growth dynamics of crops. This dataset records the static morphology and growth dynamics of crop fruits in detail, providing multi-dimensional, high-quality input data for the training of deep learning models; Step 4: Select the YOLOv10n algorithm and the BOT-SORT algorithm, compare the performance of the YOLOv10n model and the BOT-SORT algorithm, select YOLOv10n as the target detection model, select the BOT-SORT algorithm as the basic model for multi-target tracking, and conduct a training process by integrating the YOLOv10n model with the BOT-SORT algorithm. Initialize the model parameters according to the characteristics of the algorithm, adjust the model configuration according to the characteristics of the data set, and use the gradient descent algorithm and its variants to optimize the model in order to minimize the loss function. Through deep learning technology, the model can accurately identify and track wheat ears, significantly improving the accuracy and response speed of the detection operation; Step 5: By deploying the model trained and verified by the algorithm in step 4 to high-performance servers and portable mobile devices, the dataset constructed in step 3 can be processed and intelligently analyzed in real time, ensuring accurate counting of wheat ears in static images and real-time tracking of wheat ear changes in dynamic video data. Step 6: By conducting a cost-benefit analysis on step 5, the economic feasibility of the adopted technical solution and the degree of optimization of resource allocation are evaluated, ensuring the maximization of the economic benefits of technology implementation, while providing data support and decision-making basis for future technology upgrades and resource reallocation.

[0026] Step 1 (a): Optimize flight parameters according to the height of crops in the target agricultural area, the undulation of terrain and the accuracy requirements of the expected map, and accurately calculate and adjust the flight altitude and speed of the drone; Step 1 (b): Plan the flight path for the drone using automated intelligent software or manually set it up; Step 1 (c): Calibrate the high-precision sensors on the drone before performing the flight mission; Step 1 (d): operate the drone to perform flight missions along the predetermined route and conduct real-time monitoring and collection of regional map data; Step 1 (e): Real-time or near real-time processing of the raw data collected by the drone, including data synchronization, preliminary correction and effective storage management; Step 1 (f): Use photogrammetry and remote sensing software technology to stitch the collected image data to generate orthophotos, and create digital elevation models (DEMs) and digital surface models (DSMs) according to specific needs; Step 1(g): Verify the accuracy and reliability of the generated regional map by careful comparison with ground control points or known map data; Step 1 (h): Optimize the regional map to enhance the display of specific terrain features or crop areas according to the specific needs of crop yield estimation, and output the map into a format suitable for further analysis and application; Step 1 (i): Data integration of the optimized topographic map with other Geographic Information System (GIS) data layers.

[0027] The construction of agricultural area maps is the core foundation of the entire yield estimation method. During the process, drones carry high-precision sensors such as multispectral cameras, LiDAR or high-resolution cameras to perform detailed grid scanning over the target agricultural area. Among them, the setting of flight parameters is crucial and will directly affect the efficiency and quality of data collection; therefore, the flight altitude and speed will be precisely adjusted according to the specific height of the crop and the undulation of the terrain to ensure that the sensor can capture clear images. During the data collection process, the real-time monitoring system of the drone ensures the synchronization and accuracy of the data to avoid any data loss or error; the collected raw data is then processed by professional photogrammetry and remote sensing software to achieve accurate splicing of image data; the generated orthophotos, digital elevation models (DEMs) and digital surface models (DSMs) and other terrain products not only provide an accurate basis for subsequent path planning, but also provide the necessary spatial information for yield estimation. These spatial information are optimized and customized to meet specific analysis needs, and finally integrated with the geographic information system (GIS) data layer to provide comprehensive and accurate basic map support for the crop yield estimation model, significantly enhancing the geospatial analysis capabilities of the model.

[0028] The step 2 includes the following steps: Step 2 (a): According to the field area characteristics and monitoring targets presented in the topographic map, the parameters of the UAV's flight altitude, speed and sensor configuration are set; Step 2 (b): Based on the UAV flight parameters and terrain map in step 1, the flight path is generated through the algorithm and the initial population is formed. The initialization process is implemented by the following formula: BK ij =BK lb +rand×(BK ub -BK lb ), Among them BK ij is the position of the i-th black-winged kite in the j-th dimension, BK lb and BK ub are the lower and upper bounds of the j-th dimension, respectively, and rand is a random number in the range [0, 1]; Step 2 (c): Perform fitness evaluation on each initial path to quantify its compliance with the comprehensive coverage of the monitoring target; Step 2 (d): Use the black kite optimization algorithm to continuously optimize the flight path through an iterative process. Each iteration includes the following two main parts: attack behavior and migration behavior; Step 2 (e): Among the paths that have undergone multiple rounds of optimization, select the path with the highest fitness as the execution plan; Step 2 (f): Incorporate real-time obstacle avoidance strategy into path planning; Step 2(g): Verify the selected optimal flight path through simulated flight or field testing.

[0029] In step 2 (d), the attack behavior can be expressed as follows: Where n is a coefficient that changes according to the number of iterations t and the total number of iterations T, r is a random number in the range [0, 1], and p is a constant 0.9; the migration behavior can be expressed as follows: Where XLeaderPos is the current leader's position, cauchy is the Cauchy mutation, and m is a coefficient calculated from a random number r.

[0030] Based on the accurate terrain map obtained in step 1, this step 2 uses the black kite optimization algorithm to carefully plan the flight path of the drone to ensure the efficiency and comprehensiveness of the monitoring work. The flight path of the initial population is randomly generated according to the terrain map and the preset flight parameters. Each path represents a potential monitoring plan; the coverage and path length are used as fitness functions to evaluate the coverage and flight efficiency of each path for the monitoring target. This function quantifies the path performance and provides the optimization direction for the algorithm; by simulating the iterative process of the black kite's predation behavior, the algorithm gradually optimizes the path to improve the fitness score; in the optimization process, we particularly emphasize the safety of flight, and integrate the real-time obstacle avoidance strategy into the path planning to ensure that the flight of the drone in the complex farmland environment is both efficient and safe; after multiple rounds of iterations, the path with the highest fitness is selected as the final execution plan; before implementation, we verify the selected path through simulated flight or field testing to ensure that it can achieve the expected monitoring effect in actual application and meet the standards of safe flight. This path is not only optimal in theory, but also proves its comprehensiveness and feasibility in actual operation.

[0031] The step three includes the following steps: Step 3 (a): Clarify the goal of dataset construction and collect images and videos of fruits of diverse crops, covering the key growth stages from fruiting to maturity; Step 3 (b): Select data sources, giving priority to raw images and video data directly collected by drones, while integrating existing agricultural databases and ground sensor data; Step 3 (c): Optimize the configuration of data collection tools and equipment; Step 3 (d): The drone performs the flight mission according to the path planned in step 2, systematically collects crop data, and organizes and stores them in detail according to varieties, growth stages and collection conditions; Step 3 (e): Detailed annotation of the collected images and videos, recording key information such as crop varieties, growth stages, and health conditions, and adding metadata such as timestamps and locations; Step 3 (f): Divide the dataset into training set, test set and validation set; Step 3 (g): Use data augmentation techniques such as image rotation, scaling, cropping, and color adjustment to simulate variable growth conditions and collection environments to improve the model's generalization ability to different environments; Step 3 (h): Conduct in-depth multi-dimensional data analysis to study the morphological characteristics and growth dynamics of crop fruits, including fruit size, shape, density and their changing trends over time; Step 3(i): Regularly update and maintain the dataset, and continue to add images and videos of crops at different growth stages.

[0032] In the data set construction stage of the present invention, step three focuses on collecting and organizing images and video data of crop fruits to ensure that the data set can fully reflect the key features of the crop growth cycle. First, the goal of data collection is clarified, aiming to capture diverse images and videos of different varieties of crops from fruiting to maturity, ensuring that the data set can record all stages of crop growth in detail; the selected data sources include original images and video data directly collected by drones, while integrating existing agricultural databases and ground sensor data to enrich the diversity and coverage of the data set; high-specification image and video acquisition equipment is used to adapt to the changing field lighting and growth conditions to ensure high-quality data; the collected data is thoroughly labeled and classified, recording key information such as crop varieties, growth stages, health conditions, and metadata such as timestamps and locations, providing comprehensive contextual information for model training; the data is scientifically divided into training sets, test sets, and validation sets to ensure the rigor of model training and evaluation; data enhancement techniques such as image rotation, scaling, cropping, and color adjustment are used to simulate changing growth conditions and acquisition environments to improve the model's generalization ability to different environments. Step three ensures the timeliness and accuracy of the dataset through regular multi-dimensional data analysis and dataset updates, providing a solid foundation for creating a comprehensive, high-quality crop fruit dataset.

[0033] like Figure 1 As shown, the step 4 includes the following steps: Step 4 (a): Study the YOLOv10n target detection algorithm and the BOT-SORT multi-target tracking algorithm, and evaluate their performance on the dataset; Step 4 (b): Compare the performance of YOLOv10n and BOT-SORT algorithms through the key indicators of mean average precision mAP, mean recall rate mAR, and frames per second FPS. The calculation method of mean average precision mAP, mean recall rate mAR, and frames per second FPS is shown in the following formula: Among them, AP c is the average precision calculated for each category, N is the total number of categories in the detection task, and Recall c is the recall rate of the cth category, the total number of frames is the total number of video frames processed by the algorithm, and the total processing time is the total time required for the algorithm to process these frames; Step 4 (c): By comparing the performance of the YOLOv10n model with the BOT-SORT algorithm, YOLOv10n is selected as the target detection model. YOLOv10n achieves high-speed and high-precision target detection with its innovative anchor-free design and optimized CNN architecture. Step 4 (d): By comparing the performance of the YOLOv10n model and the BOT-SORT algorithm, the BOT-SORT algorithm is selected as the basic model for multi-target tracking. It combines the advantages of deep learning features and the SORT algorithm and is particularly suitable for crop fruit tracking in complex agricultural scenarios. Step 4 (e): The YOLOv10n model is integrated with the BOT-SORT algorithm. The YOLOv10n model is responsible for real-time processing of images and videos to identify crop fruits, while the BOT-SORT algorithm uses this information for continuous tracking to ensure stable tracking even in the case of occlusion or intersection. The YOLOv10n model is responsible for real-time processing of images and video frames to identify crop fruits, while the BOT-SORT algorithm uses this information for continuous tracking. Step 4(f): Evaluate the performance of the fusion model in actual crop monitoring through field flight tests; Step 4 (g): Integrate and import the constructed dataset containing static and dynamic characteristics of crop fruits into the model training environment to provide the model with necessary learning materials; Step 4 (h): Divide the dataset into training subset, validation subset and test subset to ensure the effectiveness of model training and the accuracy of evaluation; Step 4 (i): Initialize the model parameters according to the characteristics of YOLOv10n and BOT-SORT algorithms, and adjust the model configuration according to the characteristics of the dataset; Step 4 (j): Train the model to learn the key features of crop fruits and use the gradient descent algorithm and its variants to optimize the model and minimize the loss function; Step 4 (k): Based on the performance feedback of the validation subset, adjust hyperparameters such as learning rate, batch size, and number of iterations to improve model performance; Step 4 (l): Regularly evaluate the model performance on the validation subset, using indicators such as accuracy, precision, and recall to ensure the efficiency of model training; Step 4 (m): Implement an early stopping strategy to avoid overfitting and stop training when the performance on the validation set no longer improves; Step 4(n): Fine-tune the model based on the evaluation results, including optimizing the network structure or retraining some network layers to further improve model performance; Step 4 (o): Perform a final evaluation on the test subset to verify the generalization ability and ensure that the model has good generalization ability and can run stably in various practical scenarios.

[0034] The BOT-SORT algorithm combines deep learning and SORT tracking technology. In the initial stage, the YOLOv10n target detection model is responsible for identifying crop fruits, and then the deep appearance extractor extracts the key features of the target; in the first association stage, we use the fusion matching technology of Kalman filter prediction and IoU and ReID features to identify and associate track blocks; in the second association stage, further optimized matching is performed through IoU, and the track block management immediately updates the Kalman filter state, refreshes the appearance features, and performs track creation and deletion operations to maintain active tracking tracks; the post-processing step further smoothes the track output to ensure accurate and robust multi-target tracking even in complex situations where crop fruits occlude each other or move quickly.

[0035] A method for estimating crop yield based on drones, in step 4 (c), the category and position of the target frame are calculated using the following formula: C(x,y)={(x c ,y c ,w,h,p c )1,(x c ,y c ,w,h,p c )2,...,(x c ,y c ,w,h,p c ) B}, Among them, C(x,y) is the category and location of B bounding boxes predicted by each feature map cell, (x c ,y c ,w,h) represents the position of the target box, (x c ,y c ) is the offset of the center of the bounding box relative to the cell, w and h are the width and height of the bounding box, and p c The class probabilities predicted for each bounding box.

[0036] A method for estimating crop yield based on drones, in step 4 (d), during the target tracking process of the BOT-SORT algorithm, the target matches the following formula: The Kalman filter state for predicting the target state is as follows: in, is the predicted state at time t-1, F is the state transfer matrix, B is the control input matrix, ut is the control input, P t|t-1is the covariance matrix of the predicted state, Q is the process noise covariance matrix, and the Kalman filter state update for updating the target state can be given by the following formula: in, is the predicted state at time t for time t, z t is the observation vector at time t, H is the observation matrix, K t is the Kalman gain at time t, R is the observation noise covariance matrix, and I is the identity matrix.

[0037] like Figure 1 and 4 As shown in the figure, when the crop is wheat, after the wheat ear is identified and tracked by the target detection model Heng Gong, the system will continue to track the center point of its positioning frame. If the center point crosses the preset boundary line, the system will automatically record this event and increase the counter accordingly to achieve real-time statistics of the number of wheat ears. This mechanism not only improves the accuracy of counting, but also provides real-time data support for yield estimation.

[0038] The step five includes the following steps: Step 5 (a): Deploy the rigorously evaluated and optimized model to the server or mobile terminal, and ensure that the deployed model runs stably on different platforms and devices; Step 5 (b): The drone flies along a predetermined path, collects image and video data of crop fruits in real time, and transmits the data seamlessly to a server or mobile device; Step 5 (c): Preprocess the raw image and video data transmitted by the drone, including format conversion, size adjustment, and color correction; Step 5 (d): Use the deployed model to accurately analyze static images, identify and count the number of fruits in each image, and provide basic data for yield estimation; Step 5 (e): Perform dynamic tracking on the video data, use the model to capture the changes of fruits at different time points, analyze the dynamic trend of yield, and provide a basis for predicting the yield changes during crop growth; Step 5 (f): Combine real-time processed data and model output to make real-time production forecasts, and continuously adjust and optimize the forecast model through historical data; Step 5 (g): Use charts, maps or other visualization tools to visually display the forecast and analysis results; Step 5 (h): Aggregate static and dynamic yield data and conduct a comprehensive analysis to generate a detailed report containing yield estimates, regional yield distribution, and yield fluctuations.

[0039] The rigorously trained and verified model in step 5 is then deployed to high-performance servers and portable mobile devices to achieve real-time processing and intelligent analysis of drone-collected data. Step 5 ensures accurate counting of crop fruits in static images and real-time tracking of crop fruit changes in dynamic video data; the model is deployed across platforms to ensure its stable operation on different devices and achieve real-time data collection and transmission; in the real-time data preprocessing phase, the original image and video data transmitted by the drone undergo necessary preprocessing steps to meet the model input requirements; using the deployed model, the crop fruits in static images are accurately analyzed and counted, and dynamic tracking of video data is performed to deeply analyze the dynamic trend of yield. Real-time yield forecasts were made by combining real-time processed data and model outputs, and the forecast model was continuously adjusted and optimized through historical data to improve forecast accuracy. In addition, the results were visualized using tools such as charts and maps to intuitively display the forecast and analysis results, allowing users to clearly understand the distribution and changes in yield. Finally, static and dynamic yield data were summarized for a comprehensive analysis, and a detailed report containing yield estimates, regional yield distribution, and yield fluctuations was generated, providing a scientific basis for agricultural decision-making.

[0040] The step six (cost-benefit analysis) includes the following steps: Step 6 (a): Comprehensively evaluate the cost of the UAV system, sensors, data processing algorithms, and related hardware and software; Step 6(b): Based on the results of the cost assessment, select low-cost and efficient UAV and sensor components; Step 6 (c): Use advanced technologies such as TensorRT and ONNX to convert the trained model into a format suitable for low-cost hardware and optimize the deployment strategy; Step 6 (d): Regularly use transfer learning and incremental learning methods to continuously update and maintain the model based on the latest collected data; Step 6 (e): Regularly perform performance testing and monitoring, evaluate the performance of the model on actual hardware through test cases and data, and conduct comparative analysis with competing products on the market; Step 6(f): Develop and implement a maintenance plan for drones and sensors to regularly inspect and repair or replace hardware; Step 6(g): Comprehensively evaluate the economic benefits of the yield estimation method, including improved yield forecast accuracy, reduced manual inspection costs, and optimized agricultural resource allocation; Step 6 (h): Determine the return on investment by calculating and comparing the system's costs and benefits, providing quantitative data support for the feasibility of the project; Step 6 (i): Prepare a cost-benefit analysis report to provide comprehensive data support for decision makers to help evaluate and continuously improve the economic benefits of crop yield estimation methods.

[0041] Step 6: Evaluate the cost-effectiveness of the entire yield estimation system. First, evaluate the cost of components such as drone systems, sensors, and data processing algorithms; select cost-effective components to reduce initial investment and operating costs; use tools such as TensorRT and ONNX to optimize model deployment and adapt it to low-cost hardware; regularly update the model, using transfer learning and incremental learning methods to adapt to changes in the crop growth cycle. Implement hardware maintenance plans to ensure the long-term stable operation of equipment; evaluate the economic benefits of yield estimation methods, including improving prediction accuracy, reducing labor costs, and optimizing resource allocation; calculate cost-effectiveness ratios, prepare detailed reports, and provide data support for decision makers.

[0042] It should be noted that the image processing method described in the present invention is not a limitation of the present invention. Based on this purpose, technicians in this field can use other similar image processing methods to enable the model to accurately identify and track crop fruits, thereby significantly improving the accuracy and response speed of the detection operation.

[0043] The above is only used to illustrate the technical solution of the present invention rather than to limit it. Ordinary technicians in the field should understand that the specific implementation methods of the present invention can be modified or replaced by equivalents with reference to the above embodiments. Any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention are within the scope of protection of the claims of the present invention.

Claims

1. A method for estimating crop yield based on drones, characterized in that: The following steps are involved: Step 1: Capture the geographical features of the target agricultural area through aerial scanning by drones and build a regional map; Step 2: The flight path of the drone is intelligently planned by combining the regional map data in step 1 with the black kite optimization algorithm to achieve seamless coverage of the agricultural area; Step 3: Use the high-definition images and video data collected by the drone in step 1 to build a dataset of crop static morphology and growth dynamics; Step 4: Select the YOLOv10n algorithm and the BOT-SORT algorithm, compare the performance of the YOLOv10n model and the BOT-SORT algorithm, select YOLOv10n as the target detection model, select the BOT-SORT algorithm as the basic model for multi-target tracking, and perform the training process by fusing the YOLOv10n model with the BOT-SORT algorithm. Initialize the model parameters according to the characteristics of the algorithm, adjust the model configuration according to the characteristics of the data set, and use the gradient descent algorithm and its variants to optimize the model in order to minimize the loss function. Step 5: By deploying the model trained and verified by the algorithm in step 4 to high-performance servers and portable mobile devices, the data set constructed in step 3 can be processed and intelligently analyzed in real time; Step 6: Evaluate the economic feasibility of the adopted technical solution and the degree of optimization of resource allocation by conducting a cost-benefit analysis on step 5.

2. The method for estimating crop yield based on drone according to claim 1, characterized in that: The step 1 includes the following steps: Step 1 (a): Optimize flight parameters according to the height of crops in the target agricultural area, the undulation of terrain and the accuracy requirements of the expected map, and calculate and adjust the flight altitude and speed of the drone; Step 1 (b): Plan the flight path for the drone using automated intelligent software or manually set it up; Step 1 (c): Calibrate the high-precision sensors on the drone before performing the flight mission; Step 1 (d): operate the drone to perform flight missions along the predetermined route and conduct real-time monitoring and collection of regional map data; Step 1 (e): Real-time or near real-time processing of the raw data collected by the drone, including data synchronization, preliminary correction and effective storage management; Step 1 (f): Use photogrammetry and remote sensing software technology to stitch the collected image data to generate orthophotos, and create digital elevation models (DEMs) and digital surface models (DSMs) according to specific needs; Step 1(g): Verify the accuracy and reliability of the generated regional map by careful comparison with ground control points or known map data; Step 1 (h): Optimize the regional map to enhance the display of specific terrain features or crop areas according to the specific needs of crop yield estimation, and output the map into a format suitable for further analysis and application; Step 1 (i): Data integration of the optimized topographic map with other Geographic Information System (GIS) data layers.

3. The method for estimating crop yield based on drone according to claim 2, characterized in that: The step 2 includes the following steps: Step 2 (a): According to the geographical features of the agricultural area and the monitoring targets presented in the topographic map, the parameters of the UAV’s flight altitude, speed and sensor configuration are set; Step 2 (b): Based on the UAV flight parameters and terrain map in step 1, the flight path is generated through the algorithm and the initial population is formed. The initialization process is implemented by the following formula: UK ij =BK lb +rand×(BK ub -BK lb ), Among them BK ij is the position of the i-th black-winged kite in the j-th dimension, BK lb and BK ub are the lower and upper bounds of the j-th dimension, respectively, and rand is a random number in the range [0, 1]; Step 2 (c): Perform fitness evaluation on each initial path to quantify its compliance with the comprehensive coverage of the monitoring target; Step 2 (d): Use the black kite optimization algorithm to continuously optimize the flight path through an iterative process. Each iteration includes the following two main parts: attack behavior and migration behavior; Step 2 (e): Among the paths that have undergone multiple rounds of optimization, select the path with the highest fitness as the execution plan; Step 2 (f): Incorporate real-time obstacle avoidance strategy into path planning; Step 2(g): Verify the selected optimal flight path through simulated flight or field testing.

4. The method for estimating crop yield based on drone according to claim 3, characterized in that: In step 2 (d), the attack behavior is expressed as follows: Where n is a coefficient that changes according to the number of iterations t and the total number of iterations T, r is a random number in the range [0, 1], and p is a constant 0.9; The migration behavior is expressed as follows: Among them XLeader Pos is the current leader position, cauchy is the Cauchy mutation, and m is a coefficient calculated from the random number r.

5. The method for estimating crop yield based on drone according to claim 1, characterized in that: The step three includes the following steps: Step 3 (a): Clarify the goal of dataset construction and collect images and videos of fruits of diverse crops, covering key growth stages from fruiting to maturity; Step 3 (b): Select data sources, giving priority to raw images and video data directly collected by drones, while integrating existing agricultural databases and ground sensor data; Step 3 (c): Optimize the configuration of data collection tools and equipment; Step 3 (d): The drone performs the flight mission according to the route planned in step 2, systematically collects crop data, and organizes and stores them in detail according to varieties, growth stages and collection conditions; Step 3(e): Detailed annotation of the collected images and videos, recording key information such as crop varieties, growth stages, and health conditions, and adding metadata such as timestamps and locations; Step 3(f): Divide the dataset into training, test, and validation sets; Step 3 (g): Use a variety of image processing techniques to enhance the data, simulate variable growth conditions and collection environments, and improve the model's generalization ability to different environments; Step 3(h): Conduct in-depth multi-dimensional data analysis to study the morphological characteristics and growth dynamics of crop fruits, including fruit size, shape, density and their changing trends over time; Step 3(i): Regularly update and maintain the dataset, and continuously add images and videos of crops at different growth stages.

6. The method for estimating crop yield based on drone according to claim 1, characterized in that: The step 4 includes the following steps: Step 4 (a): Study the YOLOv10n target detection algorithm and the BOT-SORT multi-target tracking algorithm, and evaluate their performance on the dataset; Step 4 (b): Compare the performance of YOLOv10n and BOT-SORT algorithms through the key indicators of mean average precision mAP, mean recall rate mAR, and frames per second FPS. The calculation method of mean average precision mAP, mean recall rate mAR, and frames per second FPS is shown in the following formula: Among them, AP c is the average precision calculated for each category, N is the total number of categories in the detection task, and Recall c is the recall rate of the cth category, the total number of frames is the total number of video frames processed by the algorithm, and the total processing time is the total time required for the algorithm to process these frames; Step 4 (c): By comparing the performance of the YOLOv10n model and the BOT-SORT algorithm, YOLOv10n is selected as the target detection model; Step 4 (d): By comparing the performance of the YOLOv10n model and the BOT-SORT algorithm, the BOT-SORT algorithm is selected as the basic model for multi-target tracking; Step 4 (e): The YOLOv10n model is integrated with the BOT-SORT algorithm. The YOLOv10n model is responsible for real-time processing of images and videos to identify crop fruits, while the BOT-SORT algorithm uses this information for continuous tracking to ensure stable tracking even in the case of occlusion or intersection. Step 4(f): Evaluate the performance of the fusion model in actual crop monitoring through field flight tests; Step 4 (g): Integrate and import the constructed dataset containing static and dynamic characteristics of crop fruits into the model training environment; Step 4 (h): Divide the dataset into training subset, validation subset and test subset; Step 4 (i): Initialize the model parameters according to the characteristics of YOLOv10n and BOT-SORT algorithms, and adjust the model configuration according to the characteristics of the dataset; Step 4 (j): Train the model to learn the key features of crop fruits and use the gradient descent algorithm and its variants to optimize the model and minimize the loss function; Step 4 (k): Based on the performance feedback of the validation subset, adjust the hyperparameters such as learning rate, batch size, and number of iterations; Step 4(l): Regularly evaluate the model performance on the validation subset using metrics such as accuracy, precision, and recall. Step 4 (m): Implement an early stopping strategy to avoid overfitting and stop training when the performance on the validation set no longer improves; Step 4(n): Fine-tune the model based on the evaluation results, including optimizing the network structure or retraining some network layers; Step 4 (o): Perform final evaluation on the test subset to verify generalization ability.

7. The method for estimating crop yield based on drone according to claim 6, characterized in that: In step 4 (c), the category and position of the target frame are calculated using the following formula: C(x,y)={(x c ,y c ,w,h,p c )1,(x c ,y c ,w,h,p c )2,...,(x c ,y c ,w,h,p c ) B }, Among them, C(x,y) is the category and location of B bounding boxes predicted by each feature map cell, (x c ,y c ,w,h) represents the position of the target box, (x c ,y c ) is the offset of the center of the bounding box relative to the cell, w and h are the width and height of the bounding box, and p c The class probabilities predicted for each bounding box.

8. The method for estimating crop yield based on drone according to claim 6, characterized in that: In step 4 (d), during the target tracking process of the BOT-SORT algorithm, the target matching formula is as follows: The Kalman filter state for predicting the target state can be given by the following formula: P t|t-1 =FP t-1|t-1 F T +Q, in, is the predicted state at time t-1, F is the state transfer matrix, B is the control input matrix, ut is the control input, P t|t-1 is the covariance matrix of the predicted state, Q is the process noise covariance matrix, and the Kalman filter state update for the target state is as follows: K t =P t|t-1 H T (HP t|t-1 H T +R) -1 , P t|t =(I-K t H)P t|t-1 , in, is the predicted state at time t for time t, z t is the observation vector at time t, H is the observation matrix, K t is the Kalman gain at time t, R is the observation noise covariance matrix, and I is the identity matrix.

9. The method for estimating crop yield based on drone according to claim 1, characterized in that: The step five includes the following steps: Step 5 (a): Deploy the processed model to hardware facilities, i.e., different platforms and devices, and ensure that the deployed model runs stably on different platforms and devices; Step 5 (b): The drone flies along a predetermined path, collects image and video data of crop fruits in real time, and transmits the data seamlessly to a server or mobile device; Step 5 (c): Preprocess the raw image and video data transmitted by the drone, including format conversion, size adjustment, and color correction; Step 5 (d): Use the deployed model to accurately analyze static images, identify and count the number of fruits in each image, and provide basic data for yield estimation; Step 5 (e): Perform dynamic tracking on the video data, use the model to capture the changes of fruits at different time points, analyze the dynamic trend of yield, and provide a basis for predicting the yield changes during crop growth; Step 5 (f): Combine real-time processed data and model output to make real-time production forecasts, and continuously adjust and optimize the forecast model through historical data; Step 5 (g): Use charts, maps or other visualization tools to visually display the forecast and analysis results; Step 5 (h): Aggregate static and dynamic yield data and conduct a comprehensive analysis to generate a detailed report containing yield estimates, regional yield distribution, and yield fluctuations.

10. The crop yield estimation method based on drone according to claim 1, characterized in that: The cost-benefit analysis of step six includes the following steps: Step 6 (a): Comprehensively evaluate the cost of the UAV system, sensors, data processing algorithms, and their hardware and software; Step 6(b): Based on the results of the cost assessment, select low-cost and efficient UAV and sensor components; Step 6 (c): Use model conversion technologies such as TensorRT and ONNX to convert the trained model into a format suitable for low-cost hardware and optimize the deployment strategy; Step 6 (d): Regularly adopt transfer learning and incremental learning methods to continuously update and maintain the model based on the latest collected data; Step 6 (e): Regularly perform performance testing and monitoring, evaluate the performance of the model on actual hardware through test cases and data, and conduct comparative analysis with competing products on the market; Step 6(f): Develop and implement a maintenance plan for drones and sensors to regularly inspect and repair or replace hardware; Step 6(g): Comprehensively evaluate the economic benefits of the yield estimation method, including improved yield forecast accuracy, reduced manual inspection costs, and optimized agricultural resource allocation; Step 6 (h): Determine the return on investment by calculating and comparing the system's costs and benefits, providing quantitative data support for the feasibility of the project; Step 6 (i): Prepare a cost-benefit analysis report to provide comprehensive data support for decision makers and evaluate and continuously improve the economic benefits of crop yield estimation methods.

Images