Crop yield prediction method and system based on image analysis

Abstract

The invention provides a crop yield prediction method and system based on image analysis, relates to the field of yield prediction, and obtains a farmland multi-view remote sensing image through an unmanned aerial vehicle multispectral sensor, and executes aerial triangulation and ortho-rectification to obtain accurate exterior orientation elements and ortho-images. Then, vegetation indexes are extracted and combined with field yield measurement data, a yield regression model is constructed, and rapid prediction and grading of the crop yield are achieved. In order to reduce multi-view brightness inconsistency and give consideration to crop growth characteristics, prior information such as canopy row direction, leaf angle distribution and sealing degree is introduced, iterative correction is performed on the image by adopting a BRDF model, and radiation difference is corrected while geometric accuracy is guaranteed. Particularly, through staged trend analysis and multiple rounds of detection, priori conflicts can be found in time, incremental images can be flexibly collected, and resolving robustness and yield prediction precision are improved. The method can be widely applied to unmanned aerial vehicle remote sensing agricultural monitoring scenes.

Description

Technical Field

[0001] The present application relates to the field of yield prediction, and in particular, to a crop yield prediction method and system based on image analysis. Background Art

[0002] With the development of digital agriculture, the demand for crop growth monitoring and yield prediction is increasing. Traditional yield measurement generally relies on manual sampling or large-scale mechanical measurement, which has a long cycle and limited applicability, making it difficult to meet the needs of large-scale, multi-stage monitoring. In recent years, unmanned aerial vehicle remote sensing technology has attracted widespread attention due to its flexibility, high resolution, and wide coverage. However, after acquiring multi-temporal imagery, it often faces difficulties in resolving exterior orientation elements, inconsistent luminosity due to canopy reflectance differences, and the difficulty of efficiently integrating prior information into optimization, thus failing to fully realize its low-cost, high-precision potential. Therefore, there is an urgent need for a method that can effectively integrate multi-view remote sensing data with crop prior information, improve the resolution accuracy of exterior orientation elements and point clouds, and construct a yield regression model using vegetation indices to achieve accurate prediction and tiered management of crop yields. Summary of the Invention

[0003] In response to the deficiencies of the existing technology, the present application provides a crop yield prediction method and system based on image analysis.

[0004] In a first aspect, the present application provides a crop yield prediction method based on image analysis, comprising:

[0005] Using the spectral sensor onboard the drone, route planning and field data collection are performed on the target farmland to obtain remote sensing images covering the target farmland from multiple angles. The remote sensing images include reflectance information in the infrared, red, and green light bands.

[0006] performing stitching and orthorectification on the remote sensing images to obtain an orthophoto of the target farmland;

[0007] Identifying crop planting areas on the orthophoto and extracting vegetation indices, combining the obtained vegetation indices with field sample yield measurement data to construct a yield regression model;

[0008] Based on the yield regression model, a predicted value of the crop yield in the target farmland is obtained, and a yield classification result is output.

[0009] As an optional implementation, the vegetation index includes: green normalized difference vegetation index; the extracted vegetation index includes:

[0010] In the crop planting area, the reflectance information of the near-infrared band and the green light band is firstly subjected to difference calculation and sum calculation, and then the difference and sum are normalized to obtain the green normalized difference vegetation index.

[0011] As an optional implementation, the vegetation index includes: a dual-band enhanced vegetation index; the extracted vegetation index includes:

[0012] In the crop planting area, a dual-band enhanced vegetation index is generated through comprehensive calculation of the difference, correction coefficient and additional correction term for the reflectance information of the near-infrared band and the red light band.

[0013] As an optional implementation, before stitching and orthorectifying the remote sensing images, the method further includes:

[0014] Using aerial triangulation to obtain exterior orientation elements and a preliminary three-dimensional point cloud of the remote sensing image;

[0015] Combined with the target prior information, the bidirectional reflectance distribution function model is selected to fit the radiation differences of the same object under different observation angles and solar incidence angles;

[0016] Iteratively reflecting the bidirectional reflectance distribution function model fitting correction results into the adjustment process of the aerial triangulation to update the exterior orientation elements and the three-dimensional point cloud;

[0017] Based on the updated exterior orientation elements and 3D point cloud, the orthophoto of the target farmland is obtained.

[0018] As an optional implementation, the target prior information includes: crop canopy direction, leaf angle distribution, and closure; obtaining the target prior information includes:

[0019] Determine the crop canopy direction through ground measurements of crop growth stages and planting row spacing;

[0020] Collect samples of the crop leaf inclination angles and plant type characteristics or refer to a database of known varieties to obtain an initial range of leaf angle distribution;

[0021] The value of the closure is determined based on coverage measurement during the crop growth period or remote sensing image threshold judgment.

[0022] As an optional implementation, updating the exterior orientation elements and the three-dimensional point cloud includes:

[0023] Perform preliminary geometric reconstruction of external orientation elements and 3D point clouds based on feature matching of UAV images;

[0024] Photometric correction is performed on multi-view pixel values of the same ground object by applying scattering parameters corresponding to crop canopy row direction, leaf angle distribution and closure to the bidirectional reflectance distribution function model;

[0025] In a bundle adjustment process of aerial triangulation, photometric consistency constraints are established for the corrected pixel values, the exterior orientation elements are updated and photometric correction is repeated;

[0026] In response to the photometric consistency constraint and the geometric residual satisfying a preset threshold, a final exterior orientation element and a three-dimensional point cloud are output.

[0027] As an optional implementation, the iterative updating of the exterior orientation elements and the three-dimensional point cloud includes:

[0028] In response to acquiring the first batch of flight images and completing the preliminary coupling of aerial triangulation and bidirectional reflectance distribution function models, the geometric residuals and photometric correction errors generated by the iterative process of the first batch of flight images are recorded and analyzed to obtain the initial trend analysis results;

[0029] After collecting the second batch of flight images, the second batch of flight images are combined with the initial trend analysis results, and the iterative update of the exterior orientation elements and the bidirectional reflectance distribution function model is performed again to obtain the preliminary solution results corresponding to the second batch of flight images.

[0030] As an optional implementation, after obtaining the preliminary solution results corresponding to the second batch of flight images, the method further includes:

[0031] Under the same canopy prior information, the error convergence curves or photometric consistency residuals of the first batch and the second batch of flight images are compared to generate a secondary trend analysis result;

[0032] In response to the primary trend analysis result and the secondary trend analysis result both satisfying consistency determination conditions within a preset convergence rate threshold and a photometric error threshold, taking the current exterior orientation elements and the three-dimensional point cloud as preliminary solution results;

[0033] In response to the primary trend analysis result and the secondary trend analysis result not simultaneously satisfying the consistency determination condition, continuing to perform iterative updating or collecting subsequent images for supplementary analysis based on the second batch of flight images.

[0034] As an optional implementation, it also includes:

[0035] In response to the preliminary solution result continuously satisfying the consistency judgment condition in the preset multiple rounds of testing after re-analysis, and the crop canopy row direction, leaf angle distribution and closure do not conflict with the current solution result, outputting the final exterior orientation elements and three-dimensional point cloud;

[0036] In response to the analysis results of the above multiple rounds of detection not meeting the consistency judgment conditions, or there is a conflict between the canopy prior and the solution results, continue to collect new flight images or perform coupled iterations again until the termination conditions are met and the final exterior orientation elements and three-dimensional point cloud are output.

[0037] In a second aspect, the present application provides a crop yield prediction system based on image analysis, comprising:

[0038] The acquisition unit is used to plan routes and conduct field acquisition of target farmland based on the spectral sensor carried by the UAV, thereby obtaining remote sensing images covering the target farmland from multiple angles; wherein the remote sensing images include reflectance information of infrared, red light, and green light bands;

[0039] A correction unit, configured to perform stitching and orthorectification on the remote sensing images to obtain an orthophoto of the target farmland;

[0040] a processing unit, configured to identify crop planting areas on the orthophoto and extract vegetation indices, combine the obtained vegetation indices with field sample yield measurement data, and construct a yield regression model;

[0041] The prediction unit is used to obtain the predicted value of the crop yield in the target farmland based on the yield regression model and output the yield classification result.

[0042] Compared with existing technologies, the image analysis-based crop yield prediction method proposed in this application further combines the iterative coupling of aerial triangulation and bidirectional reflectance distribution function (BRDF) with the conventional process of drone aerial photography, image orthorectification, and vegetation index extraction. It specifically introduces prior information such as crop canopy direction, leaf angle distribution, and closure for multiple rounds of detection and conflict determination, significantly improving the radiometric consistency and geometric accuracy of multi-view images. This not only produces more accurate and stable exterior orientation elements and three-dimensional point clouds, but also combines vegetation index calculations with field yield measurement data, effectively constructing and optimizing yield regression models, thereby achieving rapid estimation and spatial classification of crop yields, meeting the needs of modern agriculture for high timeliness and high precision, reducing manual measurement costs, and improving farmland management efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A flowchart of a crop yield prediction method based on image analysis provided in an embodiment of the present application;

[0044] Figure 2 A GNDVI index image calculated from a drone image provided in an embodiment of the present application;

[0045] Figure 3 A grading diagram of rice yield estimated by drone imagery provided in an embodiment of the present application;

[0046] Figure 4 An example diagram of a wheat field requiring yield estimation identified by drone imagery provided in an embodiment of the present application;

[0047] Figure 5 A grading chart of estimated wheat yields for an unmanned farm provided in an embodiment of the present application;

[0048] Figure 6 A schematic diagram of a crop yield prediction system based on image analysis provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0050] See also Figure 1 FIG. 1 is a flowchart of a crop yield prediction method based on image analysis provided by an embodiment of the present application. The method includes steps S101 to S104, wherein:

[0051] S101: Using the spectral sensor carried by the drone, route planning and field data collection are performed on the target farmland to obtain remote sensing images covering the target farmland from multiple angles; wherein the remote sensing images include reflectance information in infrared, red, and green light bands;

[0052] S102: performing stitching and orthorectification on the remote sensing images to obtain an orthophoto of the target farmland;

[0053] S103: Identifying crop planting areas on the orthophoto and extracting vegetation indices, combining the obtained vegetation indices with field sample yield measurement data, and constructing a yield regression model;

[0054] S104: Based on the yield regression model, a predicted value of the crop yield in the target farmland is obtained, and a yield classification result is output.

[0055] Regarding S101 above:

[0056] In the specific implementation, a multi-rotor or fixed-wing drone is used as an aerial survey platform, equipped with a spectral imaging sensor with three bands of infrared, red and green light. For example, it can be a multispectral camera or an imaging device with corresponding filters to meet the calculation requirements of commonly used vegetation indices such as red light, green light, and near infrared.

[0057] The drone is also integrated with satellite navigation systems such as GPS / Beidou, as well as an inertial measurement unit (IMU) to obtain flight position information and attitude information for subsequent image geometric correction or aerial triangulation.

[0058] For route planning, in specific implementation, a reasonable flight altitude and route coverage range can be set according to the area and shape of the target farmland to ensure that each area of the farmland is covered by at least two or more route images.

[0059] It is best to conduct flight photography around noon, when the weather is clear and the wind is light, to reduce the impact of lighting differences and cloud shadows; at the same time, set the forward overlap of adjacent images to 70% or above, and the lateral overlap to above 60% to ensure the accuracy of subsequent stitching and orthorectification.

[0060] In addition, to obtain multi-angle images, different shooting angles can be set on the same farmland or multiple flight paths can be executed to obtain reflectance data in the infrared, red, and green bands from multiple directions.

[0061] For field data collection, drones can be automatically or manually controlled to fly along a predetermined route to collect images. The sensor captures images at set intervals or spatial intervals during each flight segment, recording each image's corresponding timestamp, GPS coordinates, attitude angle, and other exterior orientation information.

[0062] If necessary, several ground control points (GCPs) can be deployed on the farmland surface and their real coordinates can be measured using high-precision GNSS equipment to improve the absolute positioning accuracy of the image in subsequent processes.

[0063] After the shooting is completed, the collected remote sensing image data is preliminarily checked to confirm whether it covers the entire target farmland and has the required spectral information, such as infrared, red light, green light, etc.

[0064] For example, you can use a DJI drone to collect image data. After starting the DJITerra software and logging in, click New Task and select the multispectral task type.

[0065] For example, the required data can also be imported manually, and images can be added in the form of files or folders. The imported images should include visible light photos in .jpg format and multispectral single-band photos in .tif format.

[0066] Each group consists of 6 photos, and the last digits 0 to 6 correspond to visible light, red light, green light, blue light, red edge, and near-infrared photos respectively. Therefore, the number of imported photos should be an integer multiple of 6. If not, you need to check whether there is any data loss.

[0067] Regarding S102 above:

[0068] In specific implementation, the multispectral image data acquired in the field can be imported into the processing software and radiometric correction can be performed first, such as using a whiteboard to calibrate the image and convert the original digital quantity into a physical reflectance value.

[0069] Preliminary lens distortion correction is performed based on the camera's intrinsic orientation elements. At the same time, the GPS / IMU data recorded by the drone is used as the initial value of the exterior orientation elements to provide a reference for subsequent stitching and orthorectification.

[0070] Furthermore, feature matching algorithms such as SIFT and SURF are used to identify identical feature points between adjacent images and register the multi-view images. Aerial triangulation, such as bundle adjustment, is used to further optimize the exterior orientation elements of each image. Where GCPs are available, their coordinates are used as known constraints to correct for accumulated errors. Based on the optimized image position and attitude parameters, multiple images are gradually mosaicked together to produce a mosaicked image covering the entire target farmland.

[0071] If the farmland area has significant terrain undulations, photogrammetry software can be used to generate a local digital elevation model (DEM); otherwise, an average elevation approximation can be used. Each pixel is transformed to geographic coordinates using projection equations. Difference correction is performed based on the DEM and exterior orientation elements, and the oblique image is corrected to a vertical orthophoto. The final output is a single or multi-band orthophoto image, whose coordinate system can be projected or geodetic, and aligned with ground control points.

[0072] After radiometric correction of multispectral images, a reconstructed map with reflectivity as output is obtained. It is important to note that since multispectral reconstruction uses aerial triangulation of visible light images, and then uses the aerial triangulation results to correct other multispectral images through the overall camera structure offset, the imported imagery must include visible light images.

[0073] Regarding S103 above:

[0074] In specific implementation, orthophoto visualization or automatic classification algorithms (such as threshold segmentation and supervised classification) are used to identify crop areas within the farmland and non-crop areas such as ridges, roads, and water bodies.

[0075] For continuous areas where crops are grown purely, the entire field can be directly regarded as the crop area; if the farmland is more complex, the specific planting distribution boundaries can be determined in combination with field survey information.

[0076] For three-band orthophotos (near infrared, red, and green), vegetation indices commonly used for target crops are calculated, such as the Normalized Difference Vegetation Index (NDVI), the Green Normalized Difference Vegetation Index (GNDVI), and other dual-band enhanced vegetation indices.

[0077] Read near-infrared and red light, or green light reflectance information pixel by pixel, calculate the vegetation index according to the formula, and output the corresponding index image.

[0078] When crop areas are mixed with non-crop areas, a threshold method (such as NDVI > 0.2) can be used to generate a mask first to ensure that subsequent index calculations and average statistics are only for valid crop areas.

[0079] For example, you can use DJI Maps, which supports the output of five vegetation indices: NDVI, GNDVI, NDRE, LCI, and OSAVI. Different indices require photo data of different wavelengths, and the software will display the indices that can be calculated based on the wavelengths contained in the user-imported photos.

[0080] In addition, it also supports the output of DSM digital surface models, multispectral band image data and RGB two-dimensional orthophotos.

[0081] In practice, several plots (e.g., 1m x 1m or with fixed row spacing) can be laid out in the target farmland. Mature crops can be harvested and yield measured, with the actual yield of each plot recorded. Alternatively, harvesters equipped with yield monitoring modules can be used to record yield distribution across field zones and positionally match them within the same coordinate system.

[0082] In the orthophoto coordinate system, find the pixel range corresponding to the field sample plot or area, and calculate the mean or maximum value of the vegetation index and other statistics within the range.

[0083] The yield measurements were paired with the vegetation index in the image to obtain several sets of vegetation index and actual yield data points for subsequent regression analysis.

[0084] Among them, a univariate or multivariate statistical regression model can be used to fit the functional relationship between vegetation index and yield, such as linear regression, or a nonlinear model can be applied based on crop characteristics, and the regression coefficient can be solved using the least squares method or other regression algorithms.

[0085] If multiple vegetation indices are included, multiple regression or machine learning algorithms can be used to improve the accuracy of yield predictions.

[0086] Regarding S104 above:

[0087] In specific implementation, the vegetation index value of each crop pixel in the target farmland can be read and substituted into the constructed regression model to calculate the pixel-level yield prediction value, such as the yield per unit area, thereby generating a yield distribution image.

[0088] Among them, if the regression model is in linear form, the regression coefficient is used directly. If it is a more complex model, such as multivariate or nonlinear, the prediction calculation is performed according to the corresponding function / algorithm process.

[0089] To facilitate agricultural management, farmland can be divided into several sub-areas, such as each field or each management unit, and the pixel yield of each area can be averaged or summed to obtain the estimated yield value of the sub-area.

[0090] Based on field experience or agricultural standards, the yield values are divided into several levels, such as "high-yield area", "middle-yield area", "low-yield area", etc., and the farmland is visualized or a report is generated.

[0091] The yield grading results can be displayed as layers in the GIS system to assist decision makers in formulating precise fertilization, irrigation and other management measures.

[0092] Ultimately, users can export the forecast results as vector / raster files or paper charts, allowing farmers to intuitively understand the yield distribution across the field. This can also be combined with other supporting information, such as soil testing data and previous years' yield data, for further analysis. If necessary, actual yields after harvest can be compared to assess the accuracy of the model forecast and optimize the regression model parameters for the next season or flight.

[0093] By combining multi-angle drone remote sensing imagery with a small amount of field sample yield data, crop yield estimates can be quickly obtained across a wide area of farmland, significantly reducing the labor costs of manual measurement. Drones' rapid aerial photography and sophisticated image processing processes enable timely identification of growth discrepancies during critical crop growth periods, assisting in refined agricultural decision-making.

[0094] For example, the Green Normalized Difference Vegetation Index (GNDVI), constructed from drone imagery, can be used to estimate rice yield because it effectively assesses plant health, chlorophyll content, biomass, and cover. GNDVI is extremely sensitive to chlorophyll content, reflecting photosynthesis intensity and plant growth status, thereby indirectly estimating yield. Overall, GNDVI makes yield prediction more accurate and efficient by comprehensively reflecting multiple factors of rice growth. The calculation formula for GNDVI is:

[0095]

[0096] in, 、 Represent the spectral reflectance of near-infrared band and green band respectively.

[0097] See Figure 2 , Figure 2 This is a GNDVI index image calculated from a drone image provided in an embodiment of the present application.

[0098] In addition, because we only need to analyze the yield of crops within the planting area, we need to mask out the GNDVI of the area before performing analysis and processing. The masking method is to open the Arc Toolbox in ArcMap, select the "Extract By Mask" tool, input the crop planting area and GNDVI index image as required, and finally obtain the GNDVI image of the crops within the planting area.

[0099] According to the study by Tian Ting, Zhang Qing, Zhang Haidong, et al. Rice yield estimation based on UAV remote sensing [J]. China Rice, 2022, 28(1): 67-71,77. DOI: 10.3969 / j.issn.1006-8082.2022.01.014, the rice yield estimation model at the heading stage is:

[0100]

[0101] Where k is a constant coefficient. For example, the rice yield estimation model is used to estimate the rice yield in the rice planting area. Then, according to the rice yield grading standard, the rice yield per mu is divided into five levels: poor, general, medium, high, and very high. The rice yield grading chart is also output. Figure 3 , Figure 3 This is a grading diagram of rice yield estimated by drone imagery provided in an embodiment of the present application.

[0102] The graded yield data were statistically analyzed on a plot basis to determine the average yield per mu of the plot, as well as the area and proportion of each yield grade, and the corresponding yield evaluation was given.

[0103] For example, see Figure 4 , Figure 4 An example image of a wheat field requiring yield estimation identified by drone imagery is provided in an embodiment of the present application.

[0104] As an optional implementation method, you can load the multispectral image in ArcMap, open the toolbox (ArcToolbox), select the "Raster Calculator" tool, and then enter the calculation formula of the vegetation index to obtain the calculated vegetation index image data.

[0105] In practice, since we only need to analyze the yield of crops within the planting area, we need to mask out the EVI2 index within the area before performing analysis and processing. To do this, open the Arc Toolbox in ArcMap, select the "Extract by Mask" tool, input the crop planting area and EVI2 index image as required, and finally obtain the EVI2 image of the crops within the planting area.

[0106] According to the study by Zhu Wanxue et al. Field-scale winter wheat yield estimation based on optimized UAV remote sensing vegetation index [J]. Transactions of the Chinese Society of Agricultural Engineering, 2018, 34(11): 78-86. DOI: 10.11975 / j.issn.1002-6819.2018.11.010, the yield estimation model for winter wheat at the heading and grain filling stage is:

[0107]

[0108] The unit of yield is kilograms per mu. k is a constant coefficient, which is obtained by fitting sampling survey data for wheat in different planting areas or planting seasons.

[0109] For example, Figure 5 As shown, Figure 5 This is a grading chart of the estimated wheat yield of an unmanned farm provided in an embodiment of the present application.

[0110] In the specific implementation, the wheat yield estimation model is used to estimate the distribution map of wheat yield per mu in the wheat planting area, and then according to the wheat yield grading standard, the wheat yield per mu is divided into five levels: poor, general, medium, higher, and high, and the wheat yield grading map is output.

[0111] For example, a drone remote sensing monitoring platform (model: DJI Mavic 3 Multispectral) can be used to obtain clear and complete images of rice fields at a height of 100 meters during the heading and booting period. These images can then be preprocessed using DJI Maps to obtain visible light RGB orthophoto remote sensing images and multispectral remote sensing images.

[0112] Then, using algebraic combinations of the bands in multispectral imagery, various vegetation indices reflecting aboveground rice growth were constructed, such as the Normalized Difference Vegetation Index (NDVI), the Leaf Area Index (LAI), and the Green Normalized Difference Vegetation Index (GNDVI). Based on the extracted spectral characteristics of rice, combined with UAV RGB imagery, ground-based measured data, and historical yield data, a rice yield prediction model was constructed using statistical analysis and machine learning algorithms, enabling accurate plot-level predictions of rice yield. Rice yields within the target area were then classified according to the rice yield grading standard into five categories: poor, fair, medium, high, and very high.

[0113] As an optional implementation, before stitching and orthorectifying the remote sensing images, the method further includes:

[0114] Using aerial triangulation to obtain exterior orientation elements and a preliminary three-dimensional point cloud of the remote sensing image;

[0115] Combined with the target prior information, the bidirectional reflectance distribution function model is selected to fit the radiation differences of the same object under different observation angles and solar incidence angles;

[0116] Iteratively reflecting the bidirectional reflectance distribution function model fitting correction results into the adjustment process of the aerial triangulation to update the exterior orientation elements and the three-dimensional point cloud;

[0117] Based on the updated exterior orientation elements and 3D point cloud, the orthophoto of the target farmland is obtained.

[0118] This application is based on multi-view remote sensing images obtained by drones, and uses aerial triangulation technology to calculate the camera's exterior orientation elements and reconstruct a preliminary three-dimensional point cloud of the farmland surface.

[0119] In practice, feature detection and matching algorithms such as SIFT, SURF, or ORB can be used to extract and match feature points with the same name between overlapping images. For multispectral images, each band can be aligned first or one band can be selected for feature matching.

[0120] According to the correspondence between feature points with the same name in different image coordinate systems, a bundle adjustment model is established to solve a series of unknown quantities including the camera's exterior orientation elements and the three-dimensional coordinates of ground feature points.

[0121] It should be noted that if ground control points (GCPs) are placed in the farmland, their measured coordinates can be used as known quantities in the adjustment, thereby improving the absolute accuracy of the three-dimensional reconstruction.

[0122] In this way, using multi-view stereo matching algorithms such as Patch-based Multi-view Stereo, overlapping images are densely matched based on known exterior orientation elements to obtain the coordinates of pixels with the same name in 3D space, thereby generating a preliminary 3D point cloud covering the target farmland area. At this point, the point cloud may still contain local inconsistencies caused by differences in the reflectance characteristics of the crop canopy, requiring further optimization in subsequent steps.

[0123] For example, commercial photogrammetry software or a proprietary aerial triangulation algorithm can be used to process multiple drone images. After the initial adjustment, the resulting exterior orientation elements can maintain an attitude error within 0.1-0.3 degrees and a sub-meter position error. The initial 3D point cloud resolution can reach GSD (Ground Sampling Distance) levels. Because crop canopies are complex, their surface point clouds may contain noise or irregular textures, so subsequent iterative corrections will be performed using a BRDF model.

[0124] Furthermore, in multi-angle remote sensing, the same ground object exhibits varying reflective brightness depending on the observation angle and solar incidence angle. This phenomenon can be described using the bidirectional reflectance distribution function (BRDF) model. To more accurately correct and ensure consistency for the complex reflectance characteristics of crop canopies, this implementation incorporates prior information about the crop canopy, including its orientation, leaf angle distribution, and enclosure, into the parameterization of the BRDF model.

[0125] As an optional implementation, the target prior information includes: crop canopy direction, leaf angle distribution, and closure; obtaining the target prior information includes:

[0126] Determine the crop canopy direction through ground measurements of crop growth stages and planting row spacing;

[0127] Collect samples of the crop leaf inclination angles and plant type characteristics or refer to a database of known varieties to obtain an initial range of leaf angle distribution;

[0128] The value of the closure is determined based on coverage measurement during the crop growth period or remote sensing image threshold judgment.

[0129] In practice, the leaf inclination range, canopy orientation, and canopy closure of crops are known. This information can help select a suitable BRDF model and reasonably set its initial parameters.

[0130] Common BRDF models include the Lambertian, Ross-Li, and Rahman-Pinty-Verstraete (RPV) models. For plant canopies, the Ross-Li model or other semi-empirical models based on the principle of radiation transfer are often used to better describe the directional reflection characteristics of plant leaves.

[0131] The scattering phase function in the BRDF model and the weighted correction term for the reflectivity in the main observation direction are inferred based on prior information. For example, if the crop rows are neat and the leaf inclination angle is small, a stronger "forward scattering" component can be set in the model; if the degree of enclosure is high and the occlusion between leaves is significant, the mutual occlusion coefficient can be increased in the BRDF equation.

[0132] Furthermore, based on aerial triangulation, in order to eliminate the brightness differences of multi-view images caused by the BRDF effect and reduce the inconsistency errors of images during bundle adjustment, the correction results of the BRDF model can be fed back into the solution of exterior orientation elements and three-dimensional point clouds through iterative coupling.

[0133] In practice, after aerial triangulation, each pixel with the same name has corresponding multi-view observed brightness values and known observation geometry, such as the angle of incidence and the angle of observation. These brightness observations are substituted into the BRDF model to calculate the difference between the theoretical reflectance and the actual image brightness, namely the photometric residual.

[0134] Based on the multi-view luminosity residuals, the BRDF model parameters are adjusted using least squares or nonlinear optimization, and the search range of the model parameters is limited by reference to prior information. If the crop is highly closed, a larger mutual occlusion coefficient can be set in the model.

[0135] Once the BRDF model is updated, the pixel brightness in the camera projection model can be re-estimated, and the photometric consistency constraints or image likelihood terms in bundle adjustment can be recalculated. Simultaneously, the exterior orientation elements such as camera position and pose, as well as the 3D point cloud positions, are fine-tuned to optimize the photometric residual consistency of pixels with the same name across multiple views.

[0136] In this way, the "luminance residual calculation - BRDF parameter update - exterior orientation element correction" cycle is repeated. When both the photometric residual and the geometric residual are less than the preset threshold, or when the maximum number of iterations is reached and the rate of change tends to be stable, the algorithm is considered to have converged, and the final exterior orientation elements and the corrected 3D point cloud are output.

[0137] This process is essentially a joint optimization of geometric and radiation coupling: the exterior orientation elements and three-dimensional coordinates determine the observation angle of each pixel, and the radiation model (BRDF) determines the expected reflection value at different angles. The coordinated correction of the two can reduce the multi-perspective brightness mismatch caused by the ups and downs of the crop canopy and the directional reflection of the leaves, thereby obtaining more accurate image correction and three-dimensional reconstruction results.

[0138] Furthermore, after completing the BRDF iterative correction, the exterior orientation elements and the 3D point cloud have achieved a more ideal consistency and accuracy. At this time, the final orthophoto is generated:

[0139] In practice, each image is ortho-projected using the corrected exterior orientation elements, mapping each pixel in the oblique image to a unified geographic coordinate system. The generation process is similar to conventional orthorectification, but due to the improved geometric accuracy and reduced photometric differences, the resulting orthomosaic has a more natural overall color transition.

[0140] For multispectral images, they are ortho-projected separately and then aligned and merged in the same coordinate system to facilitate subsequent vegetation index calculation and other analyses.

[0141] Perform manual or automated inspections on the generated orthophotos, such as observing overlapped areas for visible seams and brightness differences. If BRDF iterative correction is effective, boundary transitions should be smoother. Use ground control point data to verify that absolute positioning accuracy meets requirements.

[0142] When fitting the BRDF model to UAV images, it is necessary to know the planting direction of the crop canopy in order to estimate the differences in illumination and reflection in the same ridge or leaf arrangement direction.

[0143] In specific implementation, a rangefinder or laser rangefinder is used to draw a line along the crop row to record the boundary of each crop row in the field. During measurement, several observation points can be arranged at one end and the middle of the farmland, and a compass or GPS direction finder can be used to record the row angle (relative to the north direction or relative to the east-west direction or other agreed benchmarks). If the working area is large, RTK (Real-Time Kinematic) mobile measurement equipment can also be used to record the coordinates of the starting and ending points of each crop row, and calculate the row angle by connecting the vectors. For example, by averaging or counting the mode of the measurement data of multiple ridges, a main row direction can be obtained. (Overall direction of the canopy).

[0144] The canopy orientation θc measured above can be considered a global or partitioned parameter, used to indicate the planar arrangement characteristics of crop vegetation during subsequent BRDF model initialization. This information may affect the modeling of the forward / backscatter components of the ground object during iterative optimization.

[0145] In addition, the distribution of crop leaf inclination angles often directly affects the absorption and reflection of light and near-infrared radiation. In order to accurately depict the directional reflection of vegetation in the BRDF model, this embodiment collects the following information:

[0146] In the middle of crop growth or the critical leaf area formation period, select several representative plant clusters, for example, randomly select 5 to 10 locations, and use a goniometer, leaf inclinometer or other mechanical / optical angle measurement device to measure the angle between the leaves and the horizontal plane one by one.

[0147] The measured leaf inclination angles are divided into several intervals, such as 0°-15°, 15°-30°, and 30°-45°. The number of leaf angles occurring or their percentages are counted to generate a preliminary leaf angle distribution curve. For crops with high canopies, such as corn, the leaf inclination angles of the upper, middle, and lower layers can be measured in layers to obtain a more detailed distribution.

[0148] If measurement conditions are limited, you can query the variety database provided by crop research institutions or seed companies. For example, the typical leaf angle distribution of certain wheat, corn or rice varieties has a reference range given in the literature.

[0149] It should be noted that when determining the varieties and sowing times to be the same or similar, the leaf angle distribution range in the database can be used as an initial estimate, and then combined with a small amount of measurement correction to obtain a leaf inclination angle distribution that is closer to the actual situation.

[0150] Furthermore, Coverage or Canopy Closure is often used to describe the coverage of the crop canopy on the ground, which has a significant impact on the overall reflectance of the vegetation. The closure can be obtained in the following ways:

[0151] Several small plots, such as 1m×1m, are randomly selected in the field, and the proportion of plant coverage relative to the soil or gaps within the vertical viewing angle above them is observed.

[0152] The closure ratio is recorded for each plot, such as 70%, 80%, etc., and the average closure or graded closure is calculated for the entire field. This method is intuitive but susceptible to subjective errors in manual operation.

[0153] On drone multispectral images, the vegetation coverage area is extracted through threshold segmentation (NDVI>0.2 or 0.3), and then the area is compared with the overall area of the sample plot to quickly estimate the canopy closure.

[0154] It should be noted that if there are higher requirements for accuracy, it is also possible to combine a more advanced canopy height model (CHM) or three-dimensional point cloud information to count the height distribution of vegetation on the field surface, thereby deducing the proportion of space occupied by plants.

[0155] Crop enclosure typically varies with growth stage, gradually increasing from a low degree in the seedling stage to a high degree in the heading or grain filling stage. This example is based on the measurements of the crop growth stage during this remote sensing capture. If long-term tracking is required or the BRDF model is applied to different growth stages, the enclosure should be measured separately at different time phases.

[0156] After obtaining the closure value through the above method, it can be mapped to the mutual occlusion factor or other scattering parameters in the BRDF model: if the closure is high (>90%), the vegetation blocks a large amount of the surface, making the canopy present a stronger overall reflection characteristic; if the closure is low, the actual ground soil contribution to the reflection cannot be ignored, and the radiation component of the soil background must be added when fitting the BRDF.

[0157] In this way, before the BRDF iterative fitting, the canopy direction, leaf angle distribution function, closure, etc. measured or queried above are used as the initial input of the model.

[0158] Taking the Ross-Li model as an example, parameters related to leaf angle and row direction can be injected into its volume scattering term and geometric scattering term respectively, and the closure degree determines the initial values of correction factors such as the geometric occlusion degree and hot spot effect.

[0159] In the least squares or nonlinear optimization process, search boundaries are set for the BRDF parameters. For example, if the actual measurement shows that the leaf angle is mainly concentrated in 25°±10°, the scattering direction parameters related to the tilt angle in the model should not arbitrarily exceed this range to avoid unphysical solutions.

[0160] Similarly, if the closure is greater than 90%, the occlusion coefficient and interreflection coefficient cannot be too low, otherwise it will not match the actual crop conditions.

[0161] In this way, this application ensures that the acquisition and application of prior information such as crop leaf angles, row directions, and closure are both operational and physically reasonable, helping BRDF fitting to more accurately reflect the true scattering characteristics of the canopy, thereby reducing the photometric inconsistency error of multi-view images during the bundle adjustment process, and significantly optimizing the exterior orientation elements and point cloud solutions.

[0162] As an optional implementation, updating the exterior orientation elements and the three-dimensional point cloud includes:

[0163] Perform preliminary geometric reconstruction of external orientation elements and 3D point clouds based on feature matching of UAV images;

[0164] Photometric correction is performed on multi-view pixel values of the same ground object by applying scattering parameters corresponding to crop canopy row direction, leaf angle distribution and closure to the bidirectional reflectance distribution function model;

[0165] In a bundle adjustment process of aerial triangulation, photometric consistency constraints are established for the corrected pixel values, the exterior orientation elements are updated and photometric correction is repeated;

[0166] In response to the photometric consistency constraint and the geometric residual satisfying a preset threshold, a final exterior orientation element and a three-dimensional point cloud are output.

[0167] In practice, drones equipped with multispectral / visible light cameras capture crop-growing areas from multiple angles along pre-planned routes. Adjacent aerial images maintain sufficient forward and lateral overlap (e.g., 70% / 60%), and the sensors' GPS / IMU data are recorded. Several high-precision control points (GCPs) are deployed on the ground for calibration during aerial triangulation to improve absolute positioning accuracy. Alternatively, an approximate initial solution for exterior orientation elements can be obtained solely using the RTK drone's built-in navigation information.

[0168] Perform feature extraction (SIFT, SURF, ORB, etc.) on multi-temporal or multi-angle aerial images and match identical feature points in overlapping areas. Feature matching can be performed on a single band or multiple bands can be processed together after registration.

[0169] The matching results are used for bundle adjustment to solve the exterior orientation elements of each image, such as position, attitude angle, and coordinates of feature points in three-dimensional space, and a sparse three-dimensional point cloud is initially obtained.

[0170] Dense matching algorithms (Patch-based Multi-View Stereo, Depth from Motion, etc.) are further used to estimate the disparity of pixels in the image area based on known exterior orientation elements and sparse feature points to obtain a preliminary three-dimensional point cloud covering the target farmland.

[0171] This point cloud may contain local errors or noise caused by crop canopies. The exterior orientation elements obtained at this point are only geometrically optimal solutions and have not yet been corrected for photometric consistency, requiring further improvement.

[0172] Using the methods described above, pre-measure or query prior data such as crop canopy orientation, leaf angle distribution, and closure. Based on the crop type and its actual growth stage, define a set of scattering parameters that correspond to the core parameters or initialization conditions in the BRDF model.

[0173] In this embodiment, a semi-empirical BRDF function suitable for plant canopies, such as the Ross-Li model or the Rahman-Pinty-Verstraete (RPV) model, may be selected.

[0174] The canopy row direction is mapped to the deviation correction of the observation azimuth relative to the crop row; the scattering term, shading factor or hotspot term coefficient in the model is initially set according to the leaf inclination distribution and closure.

[0175] If the crops have a typical "upright plant shape", the scattering parameters can be inclined to stronger forward scattering; if the enclosure is high, the mutual blocking coefficient can be increased to reflect the multiple reflections and shielding of light by the canopy.

[0176] In the specific implementation, the BRDF model is used to predict the theoretical reflection value (or brightness) of the same object under different observation angles and solar incidence angles, and the difference between the theoretical reflection value and the measured value of the corresponding pixel in the actual image is used to define the photometric residual.

[0177] In the bundle adjustment process of aerial triangulation, in addition to the traditional geometric reprojection error (pixel plane position difference), an additional photometric consistency constraint is introduced: that is, the brightness of the multi-view pixels is required to match the BRDF model as closely as possible. This constraint can be set to minimize the sum of the squares of the photometric residuals.

[0178] In each round of iteration, the reprojection error and photometric residual obtained by geometric feature matching are combined, and a nonlinear optimization method is used to simultaneously update: camera exterior orientation elements, such as camera center coordinates, attitude angle, etc.; the spatial coordinates of each point on the 3D point cloud, especially the adjustment of slight fluctuations on the canopy surface; BRDF scattering parameters.

[0179] In addition, if the crop closure is extremely high (e.g., above 90%), the impact of occlusion on brightness will be greater, and it is necessary to focus on optimizing the items related to "multiple reflections" or "hotspot enhancement" in the scattering parameters; if the canopy direction is clear and the ridges are basically straight, the azimuth correction can be kept within a small range.

[0180] As an optional implementation, the iterative updating of the exterior orientation elements and the three-dimensional point cloud includes:

[0181] In response to acquiring the first batch of flight images and completing the preliminary coupling of aerial triangulation and bidirectional reflectance distribution function models, the geometric residuals and photometric correction errors generated by the iterative process of the first batch of flight images are recorded and analyzed to obtain the initial trend analysis results;

[0182] After collecting the second batch of flight images, the second batch of flight images are combined with the initial trend analysis results, and the iterative update of the exterior orientation elements and the bidirectional reflectance distribution function model is performed again to obtain the preliminary solution results corresponding to the second batch of flight images.

[0183] In practice, the first batch of drone flights will be conducted over target farmland to acquire multi-angle, multi-band images. These images can cover the main crop growing areas, or just key areas of interest or specific growth stages.

[0184] Perform preliminary aerial triangulation on the first batch of flight images: use feature matching and bundle adjustment to obtain the exterior orientation elements of the first batch of flight images and generate a preliminary 3D point cloud.

[0185] If prior information such as crop canopy direction, leaf angle distribution and closure is known, it can be applied to the initialization stage of the BRDF model, and preliminary coupling adjustments can be made to the external orientation elements and point cloud through photometric consistency constraints.

[0186] Furthermore, after multiple rounds of iterations (bundle adjustment + photometric correction) on the first batch of flight images, the geometric residual and photometric correction error of each round can be counted and output separately, that is, the difference between the theoretical value of the BRDF model and the actual image brightness.

[0187] The error value of each iteration is recorded and a curve is plotted to observe the convergence trend. When it stabilizes or meets the threshold after multiple iterations, the preliminary solution results of the first batch of flight images are obtained.

[0188] The convergence curves or statistical indicators such as mean and variance of the above residuals with the number of iterations are summarized and analyzed to determine whether the current round of solution has converged smoothly, whether there may be local optimal or abnormal noise areas, etc.

[0189] If the geometric and photometric errors continue to decrease within the expected range, the solution process can be considered stable. If the errors increase or fluctuate significantly after a certain iteration, it is necessary to pay attention to the overfitting of the exterior orientation elements or BRDF parameters or the prior discrepancies, and make corresponding corrections. This analysis result is recorded as the "initial trend analysis result" and serves as an important reference for subsequent acquisition of new images or iterative optimization.

[0190] After the first batch of flight images is processed, a second batch can be added as needed. This batch of images can be acquired at different times, such as later in the same growing season or during the next key growth period, and can also be collected from different heights or angles to fill in coverage gaps or increase the amount of multi-view information.

[0191] If there is a time difference, the crop growth conditions may have changed and the prior information needs to be updated accordingly; if the crop stages are roughly the same, the previous prior can be used.

[0192] The second batch of flight images was added to the previous geometric reconstruction and photometric correction framework. However, instead of simply merging them, the previously recorded "initial trend analysis results" such as geometric residuals and photometric error curves were used as a priori references:

[0193] Initial solution: The final exterior orientation elements of the first batch can be used as the starting values for the second batch of flight image adjustment, reducing the need for large-scale searches. Model constraints: If the optimal range of leaf angle distribution in the first batch of results already matches the actual situation well, it can be maintained consistent or slightly adjusted in the second batch to avoid repeated large-scale adjustments. If the convergence process of the first batch reveals some abnormal areas, the second batch can focus on multi-view matching and photometric differences in these areas. Error threshold setting: Based on the convergence rate of the first batch, the maximum number of iterations or convergence criteria for the second batch can be reasonably set.

[0194] Finally, the aerial triangulation is re-coupled with the photometric correction: in the bundle adjustment, the possible overlapping synonyms between the first and second batches of flight images are simultaneously considered to obtain a more complete and accurate geometric and photometric consistency solution.

[0195] If the second batch of flight images covers a new area or the crop growth stage is different, it is necessary to appropriately adjust the initialization of BRDF parameters, such as closure and row direction. It is also possible to perform differentiated processing on the first and second batches of flight images during iteration, such as different incidence angles and different lighting conditions.

[0196] After multiple cycles, the preliminary solution results corresponding to the second batch of flight images are output, including new exterior orientation elements, updated 3D point clouds, and photometric correction parameters.

[0197] In this way, by first completing the iterative solution of the first batch of aerial images and analyzing their error convergence trends, and then merging and adjusting them after the second batch of data is added, we can better leverage existing solution experience and convergence status to achieve incremental optimization. If certain noise characteristics or areas of inconsistent luminosity are observed in the first batch of results, we can increase coverage or optimize the shooting angle in the second batch of aerial photography to improve the robustness and accuracy of the final solution. Acquiring images in batches can also adapt to changes in crops during different growth stages, gradually correcting external orientation elements to make the coupling of the 3D point cloud and the BRDF model more consistent with the actual canopy evolution process.

[0198] With the improvement of the accuracy of exterior orientation elements and point clouds, the generated orthophotos are more accurate in both geometric and radiometric consistency, making yield predictions based on vegetation indices more reliable.

[0199] If different batches of images span the growth cycle, the phased exterior orientation solutions and canopy three-dimensional structures can be output at each stage, laying the foundation for multi-temporal yield dynamic monitoring.

[0200] As an optional implementation, after obtaining the preliminary solution results corresponding to the second batch of flight images, the method further includes:

[0201] Under the same canopy prior information, the error convergence curves or photometric consistency residuals of the first batch and the second batch of flight images are compared to generate a secondary trend analysis result;

[0202] In response to the primary trend analysis result and the secondary trend analysis result both satisfying consistency determination conditions within a preset convergence rate threshold and a photometric error threshold, taking the current exterior orientation elements and the three-dimensional point cloud as preliminary solution results;

[0203] In response to the primary trend analysis result and the secondary trend analysis result not simultaneously satisfying the consistency determination condition, continuing to perform iterative updating or collecting subsequent images for supplementary analysis based on the second batch of flight images.

[0204] The first and second batches of flight imagery were collected for the same crop growth stage or the same farmland area. Therefore, the same canopy prior information was used, or only minor updates were made when no significant crop growth changes occurred. This ensured consistent input conditions for the BRDF model parameterization between the first and second batches of flight imagery, making the error convergence process comparable.

[0205] In specific implementation, after obtaining the "preliminary solution results corresponding to the second batch of flight images", the system also produces the changing curves of geometric residuals and photometric residuals after multiple rounds of iterations.

[0206] Compare it side by side with the curve recorded in the initial trend analysis of the first batch of flight images, for example, compare the error reduction rate, convergence shape, etc. of the two iterations at the same step number or the same threshold.

[0207] Based on the above comparison, key indicators are extracted: such as the final error value, average iteration step size, convergence rate (that is, the ratio of unit iteration error reduction, photometric residual variance, etc.), and the secondary trend analysis results are comprehensively generated.

[0208] In addition, if the geometric / photometric errors of the second batch of data decrease rapidly and stabilize under the same number of iterations, it can be regarded as "good convergence"; if oscillation occurs or the error curve is significantly higher than that of the first batch, it indicates that further improvement is needed, which may be due to insufficient prior information, differences in image quality, etc.

[0209] For example, a set of thresholds can be predefined to evaluate the convergence status based on crop growth characteristics or agricultural application accuracy requirements:

[0210] For the convergence rate threshold, if the error decreases by more than a certain proportion with the number of iterations (for example, the reduction in each iteration is lower than a certain threshold), it is considered to have converged.

[0211] For the photometric error threshold, if the average difference between the BRDF model and the observed brightness is less than a certain absolute / relative value, such as 0.01 or 1% normalized error, it is considered to have achieved acceptable consistency.

[0212] Compare the "primary trend analysis results" of the first batch with the "secondary trend analysis results" of the second batch to see whether they both meet the above threshold judgment:

[0213] If both satisfy: It means that under the same canopy prior information, the photometric and geometric residuals of the two batches of images converge to the expected range at a similar rate. It can be determined that the algorithm is overall stable and the prior parameters are well applicable.

[0214] If not satisfied: that is, the error of the first or second batch (or both) does not meet the standard, or the difference between the error results of the two is too large, further iteration or collection of additional data is required.

[0215] Furthermore, if the primary and secondary trend analyses simultaneously meet the consistency judgment conditions, that is, the geometric residual and photometric error are both within the preset threshold, and the convergence trends of the iterative curves of the two batches of images are consistent, then the current exterior orientation elements and three-dimensional point clouds can be regarded as "preliminary solution results."

[0216] "Preliminary solution results" mean that there is a high probability that sufficient accuracy and stability have been achieved, and subsequent applications such as orthophoto generation, vegetation index calculation, and yield prediction can be carried out.

[0217] If the consistency criteria are not met simultaneously in both trend analyses, this indicates that there is room for improvement in the exterior orientation elements and 3D point cloud of this embodiment. In this case, a more detailed iterative optimization can be performed on the second batch of flight images. For example, increasing the number of iterations and monitoring the error changes; adjusting the prior information; adjusting the BRDF model parameter range or photometric constraint weights; etc.

[0218] If the second batch of flight images is not sufficient to achieve overall convergence, or if the error is large due to external factors such as severe cloud shadows or unstable lighting, a third or more batches of flight images can be collected as needed to further cover the missing areas or reduce observation noise.

[0219] After acquiring new images, they are combined with the aforementioned "primary trend analysis results" and "secondary trend analysis results" to gradually accumulate multi-temporal and multi-perspective information, continuously improving the accuracy of exterior orientation elements and three-dimensional point clouds.

[0220] In this way, this embodiment is particularly suitable for scenarios with multiple flights over large fields: by comparing trends to determine whether the system has the stability to solve with high precision, problems can be discovered in a timely manner, and unnecessary high-intensity iterations can be reduced.

[0221] Furthermore, for remote sensing monitoring of fields spanning a wide seasonal span, the suitability of prior information (such as growing season information) can be quickly assessed after the initial and secondary solutions. If there are significant temporal differences, such as significant changes in leaf angle distribution or closure, local corrections can be attempted before re-solving to ensure photometric consistency.

[0222] For example, take the first batch of flight images as an example: after N iterations, the geometric residual is reduced to about 0.6 pixels, and the mean photometric residual is reduced to 0.012; after the remaining M iterations, it tends to be stable, and the initial trend analysis results are obtained, such as a smooth decline in the error curve and a low final value. After the second batch of flight images are merged, the same or similar iterative process is adopted, and the final photometric residual is about 0.010; if it is similar to the first batch and the decrease is similar, then the initial and secondary analysis results both meet the threshold (such as <0.015 is an acceptable range), which can be regarded as good solution consistency, and the preliminary solution results are output. If the second batch error is much higher than the first batch, or it cannot be effectively converged, it enters the stage of continued iteration or supplementary acquisition until the threshold is met or other disposal is performed.

[0223] As an optional implementation, it also includes:

[0224] In response to the preliminary solution result continuously satisfying the consistency judgment condition in the preset multiple rounds of testing after re-analysis, and the crop canopy row direction, leaf angle distribution and closure do not conflict with the current solution result, outputting the final exterior orientation elements and three-dimensional point cloud;

[0225] In response to the analysis results of the above multiple rounds of detection not meeting the consistency judgment conditions, or there is a conflict between the canopy prior and the solution results, continue to collect new flight images or perform coupled iterations again until the termination conditions are met and the final exterior orientation elements and three-dimensional point cloud are output.

[0226] The following is an illustrative example of a specific implementation method for the "multi-round detection" technical feature, intended to highlight the innovative approach and technical benefits achieved by this application compared to existing technologies. This implementation method can be considered an optional refinement of the aforementioned iterative process, illustrating how, after obtaining a "preliminary solution," multiple rounds of detection and prior consistency judgments can be used to flexibly determine whether to directly output the final exterior orientation elements and 3D point cloud, continue to acquire new flight images, or perform additional coupled iterations to ensure high accuracy and robustness.

[0227] Furthermore, this application specifically introduces a multi-round detection mechanism that combines phased trend analysis with canopy prior information to address the iterative solution problem of multi-phase and multi-batch flight images.

[0228] If a "large-scale bundle adjustment" is performed on all acquired images at once, without considering the phased convergence of the photometric and geometric residuals of each batch of images, then if the crop undergoes significant growth phase transitions between flights, such as a significant change in leaf angle at heading, this "one-shot" solution can easily lead to localized deviations or oscillations later in the process. Even blindly increasing the number of iterations will fail to identify prior conflicts or locate the root causes of the problem in a timely manner, resulting in wasted resources and uncertain accuracy.

[0229] This application first performs an iterative solution on each batch of images, outputting preliminary / preliminary results and corresponding trend analysis. It then performs multiple rounds of testing for "photometric consistency" and "geometric residuals" based on canopy prior information. If the test consistently meets the preset threshold and the priors are consistent, the final exterior orientation elements and 3D point cloud are quickly output. Otherwise, new images are collected promptly or further iterative corrections are made to avoid long-term computational overhead in inappropriate bundle adjustments.

[0230] This phased trend analysis method significantly reduces the total amount of calculation and repeated adjustment operations while maintaining high-precision exterior orientation element solutions. It also makes the judgment of the matching degree between the prior and the solution more timely, avoiding the defect of "the accuracy / prior contradiction is difficult to expose in time" in traditional incremental photogrammetry methods.

[0231] In the specific implementation, in the previous steps recorded above, the incremental coupling solution of the first batch and the second batch or multiple batches of images has been completed, and after the initial and secondary trend analysis and judgment, the preliminary solution results are obtained - at this time, the exterior orientation elements and the three-dimensional point cloud are in a relatively optimal state, but have not yet been finally confirmed.

[0232] Based on the preliminary solution results, the system can perform one or more "fine-tuning iterations", or recalibrate the image under different lighting / postures to confirm whether the error hovers within a small range and remains stable.

[0233] For example, a photometric error threshold, such as 0.010.02, a geometric reprojection error threshold, such as 0.51.0 pixels, and a lower limit of the convergence rate are set in advance; if all indicators meet or exceed the threshold in multiple rounds of detection, it is considered to meet the consistency judgment conditions.

[0234] Furthermore, this embodiment incorporates prior matching calculations for crop canopy row orientation, leaf angle distribution, and closure. For example, this involves comparing the canopy texture orientation in the point cloud with the row angle measured on the ground, or estimating multi-view photometric residual features based on leaf angle distribution. If no significant deviations are observed, the priors are considered to be consistent.

[0235] In the analysis of multiple rounds of detection, if the geometric error and photometric error both steadily decrease to the target range and no longer fluctuate significantly, and the crop prior information still matches the updated exterior orientation elements and 3D point cloud well, for example, the row angle difference is less than 1.5° and the leaf angle distribution consistency is greater than 95%, then the iterative solution can be considered to have "reached the optimal or near-optimal" state.

[0236] The system quickly outputs the final exterior orientation elements and 3D point cloud as the final result, which is then used for subsequent orthophoto generation, vegetation index calculation, and yield forecasting. Compared to traditional one-time large-scale adjustments, this application achieves higher-quality solutions through controlled and simple multiple rounds of testing, reducing blind recalculation and ensuring a priori consistency.

[0237] Furthermore, if the photometric or geometric residuals rebound, fluctuate, or remain consistently above the threshold during multiple rounds of testing, the solution is not yet stable and requires further coupled iteration or adjustment of the BRDF model weights or prior parameter ranges. If significant deviations are observed after comparing leaf angle distribution or canopy orientation, for example, if the error between the field orientation calculated from the point cloud solution and the field measurement exceeds the specified range, this could be due to inaccurate prior data or a local optimum in the iterative algorithm. New flight imagery should be collected or the discrepancy should be compensated for in the next batch of imagery.

[0238] To avoid unlimited iterations, the system can set an upper limit or lower threshold for trial batches. Once exceeded, the user will be prompted to further improve the working environment or measurement process for higher accuracy, thereby achieving a closed-loop process.

[0239] This eliminates the need to perform a global solution on all images at once, allowing for timely detection of problems and optimized resource usage. Multiple rounds of testing allow for early comparison of crop priors with solution results, allowing the system to correct priors or supplement data at this stage without significantly degrading the final solution quality.

[0240] By introducing multiple rounds of detection and prior conflict determination, a phased trend analysis is performed after outputting the "preliminary solution results." If the indicators consistently meet the standards and do not conflict with the crop canopy priors, the final exterior orientation elements and 3D point cloud are output. If there are conflicts or high errors, additional images or further coupled iterations can be used to ensure overall solution quality. This method demonstrates significant advantages over traditional incremental bundle adjustment in terms of efficiency and precision control, addressing the pain points of solving multi-batch flight imagery and providing technical support for high-precision agricultural remote sensing and yield prediction.

[0241] It should be noted that the above-mentioned specific steps and thresholds may be adjusted accordingly depending on the target crop variety, field environment, and yield measurement requirements. Any modifications or additions made by those skilled in the art based on the disclosure of this application and in accordance with actual circumstances shall fall within the scope of protection of this application.

[0242] Based on the same inventive concept, the embodiments of the present application also provide a crop yield prediction system based on image analysis corresponding to the crop yield prediction method based on image analysis. Since the principle of problem solving by the system in the embodiments of the present application is similar to the above-mentioned crop yield prediction method based on image analysis in the embodiments of the present application, the implementation of the system can refer to the implementation of the method, and the repeated parts will not be repeated.

[0243] Reference Figure 6 FIG. 1 is a schematic diagram of a crop yield prediction system based on image analysis provided in an embodiment of the present application, wherein the system includes:

[0244] The acquisition unit 10 is used to plan routes and collect data in the field for the target farmland based on the spectral sensor carried by the drone, thereby obtaining remote sensing images covering the target farmland from multiple angles; wherein the remote sensing images include reflectance information of infrared, red light, and green light bands;

[0245] The correction unit 20 is used to stitch and orthorectify the remote sensing images to obtain an orthophoto of the target farmland;

[0246] The processing unit 30 is configured to identify the crop planting area on the orthophoto image and extract the vegetation index, and combine the obtained vegetation index with the field sample yield measurement data to construct a yield regression model;

[0247] The prediction unit 40 is used to obtain a predicted value of the crop yield in the target farmland based on the yield regression model and output a yield classification result.

[0248] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

Claims

1. A crop yield prediction method based on image analysis, characterized in that: include: Using the spectral sensor onboard the drone, route planning and field data collection are performed on the target farmland to obtain remote sensing images covering the target farmland from multiple angles. The remote sensing images include reflectance information in the infrared, red, and green light bands. performing stitching and orthorectification on the remote sensing images to obtain an orthophoto of the target farmland; Identifying crop planting areas on the orthophoto and extracting vegetation indices, combining the obtained vegetation indices with field sample yield measurement data to construct a yield regression model; Based on the yield regression model, a predicted value of the crop yield in the target farmland is obtained, and a yield classification result is output.

2. The method according to claim 1, characterized in that The vegetation index includes: green normalized difference vegetation index; the extracted vegetation index includes: In the crop planting area, the reflectance information of the near-infrared band and the green light band is firstly subjected to difference calculation and sum calculation, and then the difference and sum are normalized to obtain the green normalized difference vegetation index.

3. The method according to claim 2, characterized in that The vegetation index includes: dual-band enhanced vegetation index; the extracted vegetation index includes: In the crop planting area, a dual-band enhanced vegetation index is generated through comprehensive calculation of the difference, correction coefficient and additional correction term for the reflectance information of the near-infrared band and the red light band.

4. The method according to any one of claims 1 to 3, characterized in that Before stitching and orthorectifying the remote sensing images, the following steps are also included: Using aerial triangulation to obtain exterior orientation elements and a preliminary three-dimensional point cloud of the remote sensing image; Combined with the target prior information, the bidirectional reflectance distribution function model is selected to fit the radiation differences of the same object under different observation angles and solar incidence angles; Iteratively reflecting the bidirectional reflectance distribution function model fitting correction results into the adjustment process of the aerial triangulation to update the exterior orientation elements and the three-dimensional point cloud; Based on the updated exterior orientation elements and 3D point cloud, the orthophoto of the target farmland is obtained.

5. The method according to claim 4, characterized in that The target prior information includes: crop canopy direction, leaf angle distribution and closure; the acquisition of the target prior information includes: Determine the crop canopy direction through ground measurements of crop growth stages and planting row spacing; Collect samples of the crop leaf inclination angles and plant type characteristics or refer to a database of known varieties to obtain an initial range of leaf angle distribution; The value of the closure is determined based on coverage measurement during the crop growth period or remote sensing image threshold judgment.

6. The method according to claim 5, characterized in that The updating of the exterior orientation elements and the three-dimensional point cloud comprises: Perform preliminary geometric reconstruction of external orientation elements and 3D point clouds based on feature matching of UAV images; Photometric correction is performed on multi-view pixel values of the same ground object by applying scattering parameters corresponding to crop canopy row direction, leaf angle distribution and closure to the bidirectional reflectance distribution function model; In a bundle adjustment process of aerial triangulation, photometric consistency constraints are established for the corrected pixel values, the exterior orientation elements are updated and photometric correction is repeated; In response to the photometric consistency constraint and the geometric residual satisfying a preset threshold, a final exterior orientation element and a three-dimensional point cloud are output.

7. The method according to claim 6, characterized in that The iterative update of the exterior orientation elements and the three-dimensional point cloud includes: In response to acquiring the first batch of flight images and completing the preliminary coupling of aerial triangulation and bidirectional reflectance distribution function models, the geometric residuals and photometric correction errors generated by the iterative process of the first batch of flight images are recorded and analyzed to obtain the initial trend analysis results; After collecting the second batch of flight images, the second batch of flight images are combined with the initial trend analysis results, and the iterative update of the exterior orientation elements and the bidirectional reflectance distribution function model is performed again to obtain the preliminary solution results corresponding to the second batch of flight images.

8. The method according to claim 7, characterized in that After obtaining the preliminary solution results corresponding to the second batch of flight images, the following steps are further performed: Under the same canopy prior information, the error convergence curves or photometric consistency residuals of the first batch and the second batch of flight images are compared to generate a secondary trend analysis result; In response to the primary trend analysis result and the secondary trend analysis result both satisfying consistency determination conditions within a preset convergence rate threshold and a photometric error threshold, taking the current exterior orientation elements and the three-dimensional point cloud as preliminary solution results; In response to the primary trend analysis result and the secondary trend analysis result not simultaneously satisfying the consistency determination condition, continuing to perform iterative updating or collecting subsequent images for supplementary analysis based on the second batch of flight images.

9. The method according to claim 8, characterized in that Also includes: In response to the preliminary solution result continuously satisfying the consistency judgment condition in the preset multiple rounds of testing after re-analysis, and the crop canopy row direction, leaf angle distribution and closure do not conflict with the current solution result, outputting the final exterior orientation elements and three-dimensional point cloud; In response to the analysis results of the above multiple rounds of detection not meeting the consistency judgment conditions, or there is a conflict between the canopy prior and the solution results, continue to collect new flight images or perform coupled iterations again until the termination conditions are met and the final exterior orientation elements and three-dimensional point cloud are output.

10. A crop yield prediction system based on image analysis, characterized in that: include: The acquisition unit is used to plan routes and conduct field acquisition of target farmland based on the spectral sensor carried by the UAV, thereby obtaining remote sensing images covering the target farmland from multiple angles; wherein the remote sensing images include reflectance information of infrared, red light, and green light bands; A correction unit, configured to perform stitching and orthorectification on the remote sensing images to obtain an orthophoto of the target farmland; a processing unit, configured to identify crop planting areas on the orthophoto and extract vegetation indices, combine the obtained vegetation indices with field sample yield measurement data, and construct a yield regression model; The prediction unit is used to obtain the predicted value of the crop yield in the target farmland based on the yield regression model and output the yield classification result.

Images