Unmanned aerial vehicle multi-view image and weather information coordinated winter wheat thousand seed weight estimation method
By combining attention mechanism feature fusion algorithm with meteorological information, the problem of insufficient accuracy in estimating the thousand-grain weight of winter wheat in UAV remote sensing was solved, and efficient and accurate estimation results were achieved.
Patent Information
- Application Number
- CN202610113153.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-28
- Publication Date
- 2026-05-19
AI Technical Summary
Existing technologies make it difficult to efficiently and accurately estimate the thousand-grain weight of winter wheat using UAV remote sensing, and traditional methods fail to fully utilize the synergistic effect of multi-view images and meteorological information, resulting in insufficient estimation accuracy.
The attention mechanism feature fusion algorithm (MV-AMFF) is adopted to combine multi-view imagery and meteorological information. The spectral information of the UAV multi-view imagery is extracted adaptively and combined with meteorological features. Then, the backpropagation neural network is used to estimate the thousand-grain weight.
This method significantly improves the estimation accuracy and robustness of the thousand-grain weight of winter wheat, and provides a new, efficient, and accurate UAV remote sensing estimation method.
Smart Images
Figure CN122063053A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of agricultural remote sensing monitoring and is a method for estimating the thousand-grain weight of winter wheat by combining multi-view images from unmanned aerial vehicles with meteorological information. Background Technology
[0002] Rapid, accurate, and non-destructive estimation of the thousand-grain weight of winter wheat is crucial for accelerating intelligent breeding and ensuring food security. As one of the three major yield components, remote sensing research on the estimation of thousand-grain weight is significantly insufficient compared to the number of ears per unit area and the number of grains per ear. Limited research also faces challenges such as low accuracy in ground-based canopy hyperspectral estimation or low efficiency of near-end grain imaging systems. Given these limitations, researchers have turned their attention to the more flexible and efficient UAV remote sensing platform. Although existing studies have attempted to indirectly estimate the dynamic thousand-grain weight of wheat using UAV spectroscopy, these methods are impractical due to the long mechanistic chain, the gradual accumulation of errors, and the cumbersome reproduction process. Addressing the urgent needs of intelligent breeding, a new and practical technological path is urgently needed to achieve efficient and accurate estimation of the thousand-grain weight of wheat based on UAV remote sensing.
[0003] Over the past decade, multi-angle remote sensing, utilizing information acquired from multiple fixed observation angles, has demonstrated significant advantages over single zenith observation angles in improving crop growth parameter estimation. However, the exponentially increased workload associated with multi-angle image acquisition on UAV platforms has limited its widespread application, leading to a long-standing reliance on orthophotos stitched together from zenith observation images. Given the central projection imaging principle of cameras, a single raw image acquired at the zenith observation angle actually contains continuously changing perspectives from the center point (zenith viewpoint) to the edge (tilt viewpoint). In the large number of highly overlapping raw images acquired by UAVs at the zenith observation angle, the same ground feature is often repeatedly captured from multiple perspectives, forming an "implicit" multi-angle observation dataset. Effectively mining and utilizing this multi-view information can both circumvent the data acquisition burden and leverage the unique advantages of tilted perspectives over zenith perspectives in crop parameter perception. Currently, UAV multi-view imagery has shown potential superior to conventional stitched orthophotos in monitoring crop growth parameters, but existing information extraction methods rely on basic statistics and fail to fully exploit the information gain brought by perspective differences. Attention mechanisms, as a technique capable of adaptively weighting different features, have demonstrated superior performance compared to traditional methods in tasks such as corn rust monitoring. Therefore, developing a feature fusion algorithm based on attention mechanisms to extract multi-view information from UAVs closely related to thousand-grain weight holds promise for breaking through the upper limit of spectral estimation accuracy based on orthophotos.
[0004] Since crop yield is the result of accumulation during the growth process, temporal spectral features have been widely used to improve crop yield prediction. Compared to directly using the original temporal spectral features, feature engineering reconstruction based on yield formation mechanisms, such as calculating cumulative values and rates of change, has proven to be more effective in improving the accuracy of crop yield prediction. Wheat thousand-grain weight is a direct product of material accumulation during the grain-filling process, while heading and flowering stages lay the foundation for the "sink" capacity of grain filling. Given that thousand-grain weight and yield share similar "source-sink" physiological bases, corresponding feature engineering reconstruction of temporal spectral features from heading to late grain filling is expected to be an effective way to improve the accuracy of thousand-grain weight estimation. Furthermore, since meteorological information is a key driver of crop yield, combining remote sensing and meteorological features can more accurately estimate crop yield compared to relying solely on remote sensing data. However, it should be noted that while temperature, light, and water are necessary factors for crop growth, the impact of different meteorological factors on thousand-grain weight is not homogeneous and constant. Dominant meteorological factors change dynamically with the growth stage, and a certain factor may even have a negative impact on thousand-grain weight at a specific stage, such as precipitation during the grain-filling period. This means that the traditional method of combining all meteorological factors such as temperature, light, and water with remote sensing data at once may not yield optimal results for thousand-grain weight estimation. Therefore, it is essential to clarify the optimal combination of remote sensing and meteorological factors to synergistically improve thousand-grain weight estimation by gradually introducing meteorological factors according to their importance to thousand-grain weight. Summary of the Invention
[0005] The technical problem solved by this invention is to provide a method for estimating the thousand-grain weight of winter wheat by combining multi-view UAV imagery with meteorological information. This method involves developing an attention mechanism feature fusion algorithm (MV-AMFF) to adaptively extract spectral information from multi-view UAV imagery; verifying the effectiveness of the developed algorithm by comparing it with three image information extraction methods; subsequently, using correlation analysis to select three spectral indices sensitive to thousand-grain weight, and reconstructing them using three temporal feature reconstruction methods; finally, combining the reconstructed sensitive spectral indices with meteorological features to select the optimal feature combination, and using a backpropagation neural network to achieve accurate estimation of the thousand-grain weight of winter wheat. This invention effectively mines multi-view UAV imagery information and achieves effective collaboration between it and meteorological information, significantly improving the accuracy and robustness of winter wheat thousand-grain weight estimation, and providing a new technical solution for UAV remote sensing estimation of crop thousand-grain weight.
[0006] The technical solution to achieve the purpose of this invention is as follows:
[0007] A method for estimating the thousand-grain weight of winter wheat by combining multi-view imagery from unmanned aerial vehicles (UAVs) with meteorological information includes the following steps:
[0008] Step 1: Acquire AIRPHEN multispectral UAV images of winter wheat from heading to late grain filling in the experimental plot, and obtain orthophoto and multi-view reflectance images after preprocessing; specifically including:
[0009] Step 1-1: The drone imagery acquisition period includes four stages for winter wheat: heading stage, flowering stage, pre-grain filling stage, and post-grain filling stage.
[0010] Steps 1-2: The AIRPHEN multispectral UAV images are preprocessed in Agisoft PhotoScan Pro software through image alignment, geometric correction, camera optimization, dense point cloud construction, mesh generation, radiometric correction and orthophoto generation, and then exported as orthophotos.
[0011] Steps 1-3: The AIRPHEN multispectral UAV images are interactively processed in Agisoft PhotoScan Pro and PyCharm software to extract raw image metadata, perform halo correction, exposure time correction, metadata information writing, band synthesis, geometric correction, and radiometric correction preprocessing, and then exported to obtain all single images, i.e., multi-view reflectance images.
[0012] Step 2: Based on multi-view reflectance imagery, select effective viewing angles for each cell and extract the corresponding raw reflectance, observed zenith angle, and quadrant position; specifically including:
[0013] Step 2-1: Use PyCharm software and corresponding algorithms to calculate the coverage of effective wheat canopy pixels in the region of interest (ROI) of each cell in all single multi-view reflectance images, and at the same time extract the average reflectance of each band in the ROI of each cell.
[0014] Step 2-2: Given that the number of plots in the test field covered by a single image in this study is relatively small, a coverage rate of ≥90% within the ROI is selected as the standard for the effective viewing angle of each plot. The effective viewing angle and the corresponding original reflectance of each band are selected according to this standard.
[0015] Steps 2-3: Using PyCharm software and corresponding algorithms, based on the geometric optics model, calculate the observation zenith angle (VZA) of each pixel relative to the center of each image according to the pixel coordinates. Extract the observation zenith angle and quadrant position Q of the corresponding ROI center coordinates in the effective viewpoint image of each cell to characterize the observation zenith angle and quadrant of each ROI. The calculation steps for the image observation zenith angle (VZA) include:
[0016] Step 2-3-1: Read the image information, including image size, spatial resolution, image center pixel coordinates, coordinates of any pixel, latitude of the southernmost and northernmost points of the image, and sensor flight altitude;
[0017] Step 2-3-2: Define the center pixel C as the zenith direction, i.e., VZA=0°, and let the projection of the viewing distance offset of any pixel P relative to the center C onto the horizontal plane be r;
[0018] Step 2-3-3: Calculate the latitude correction factor d, using the following formula:
[0019]
[0020]
[0021] Where latitude is the center latitude of the geographic range of the image, Ymin is the southernmost latitude of the image, Ymax is the northernmost latitude of the image, and R is the Earth's radius of 6,371,000 meters.
[0022] Steps 2-3-4: Calculate the geographic offset distance dx of any pixel P relative to the image center C in the east-west direction and the geographic offset distance dy in the north-south direction, using the following formulas:
[0023]
[0024]
[0025] Where cols and rows are the number of columns and rows of pixels in the image size, respectively; cell_w and cell_h represent the size of the spatial resolution, that is, the geographic width and height of the pixel, in meters; and i and j represent the coordinate position of any pixel P in the image.
[0026] Steps 2-3-5: Calculate the horizontal projection distance r, using the following formula:
[0027]
[0028] Steps 2-3-6: Calculate the observed zenith angle VZA, where h represents the sensor's flight altitude. The formula is as follows:
[0029]
[0030] Step 3: Using a dedicated neural network architecture based on an attention mechanism as the core fusion model, and with measured thousand-grain weight as the supervised target, an "end-to-end" learning approach is adopted. Three parallel feature transformation branches are used to process the original reflectivity, observed zenith angle, and quadrant position information, respectively, thereby developing the attention mechanism feature fusion algorithm MV-AMFF for UAV multi-view imagery; specifically including:
[0031] Step 3-1: The MV-AMFF method uses a dedicated neural network architecture based on an attention mechanism as its core fusion model;
[0032] Step 3-2: The model processes the original reflectance, observed zenith angle and quadrant position information through three parallel feature transformation branches. The reflectance branch maps the spectral data of the AIRPHEN sensor in six bands to the hidden space through a two-layer fully connected network. The observed zenith angle branch transforms the data into high-dimensional features through a separate neural network. The quadrant branch converts discrete position categories into dense vectors through an embedding layer.
[0033] Step 3-3: The reflectance, observed zenith angle, and quadrant position information, which have been processed separately by the branch attention mechanism, are concatenated with features and then the attention score of reflectance at each observed zenith angle and quadrant is calculated by a neural network containing the "tanh" activation function.
[0034] Steps 3-4: Use the "softmax" function to normalize the attention score into attention weights, and use these weights to perform a weighted fusion of the original reflectance while preserving its physical meaning;
[0035] Steps 3-5: Using the mean square error (MSE) between the estimated thousand-grain weight and the measured thousand-grain weight as the loss function, backpropagation drives the fusion model to perform "end-to-end" learning, optimizing the weight allocation strategy so that spectral features from different quadrant positions and observed zenith angles can obtain differentiated weights based on their effectiveness in indicating the thousand-grain weight, thereby achieving dynamic and adaptive feature fusion and generating a new reflectance that is more beneficial for thousand-grain weight estimation.
[0036] Step 4: Three methods are used to extract the reflectance of UAV orthophotos and multi-view images, and basic statistical analysis is performed. The method for extracting reflectance from orthophotos is denoted as Mosaic, while two different methods are used to extract reflectance from multi-view images: the MV-mean method and the developed MV-AMFF algorithm; specifically including:
[0037] Step 4-1: Extract the reflectance of the orthophoto using the Mosaic method. Specifically, this involves using PyCharm software to extract the average reflectance of each band of the mosaic orthophoto within each region of interest (ROI) as the reflectance of each cell.
[0038] Step 4-2: The reflectance of multi-view images is extracted using the MV-mean method. Specifically, the average value of the reflectance of each band under all effective views of each cell obtained in Step 2 is calculated as the reflectance data corresponding to the thousand-grain weight of each cell.
[0039] Step 4-3: Use the MV-AMFF algorithm to extract the reflectance of multi-view images. Specifically, the original reflectance, observation zenith angle and quadrant position information of all effective views of each cell obtained in Step 2 are used to run the MV-AMFF algorithm developed in Step 3 in PyCharm software, and its output value is used as the reflectance data corresponding one-to-one with the measured thousand-grain weight of each cell.
[0040] Step 4-4: Perform basic statistical analysis on the reflectance extracted by the three methods to clarify the differences in reflectance.
[0041] Step 5: Based on the three types of reflectance data obtained in Step 4, calculate 15 conventional spectral indices. Through correlation analysis, select three spectral indices (SIs) sensitive to thousand-grain weight. Then, use three time-series feature reconstruction methods—original value, cumulative value, and change—to process the time-series sensitive spectral indices (SIs) and obtain the reconstructed time-series spectral indices (SIs). Specifically, this includes:
[0042] Step 5-1: Based on the three types of reflectance data obtained in Step 4, calculate 15 conventional spectral indices and perform correlation analysis (evaluation index: squared correlation coefficient r). 2 Three spectral indices (SIs) that are sensitive to thousand-grain weight in all four stages from heading to late grain filling were selected.
[0043] Step 5-2: Using three time-series feature reconstruction methods—original value, cumulative value, and change amount—the time-series sensitive spectral indices (SIs) are processed to obtain the reconstructed sensitive time-series spectral indices (SIs). The original value refers to the SIs value obtained directly from each period from heading to the late grain-filling stage. The cumulative value refers to the sum of SIs obtained from all periods from heading to the late grain-filling stage. The change amount refers to the difference in SIs obtained from adjacent periods (the later period minus the earlier period).
[0044] Step 6: The three reconstructed sensitive temporal spectral indices (SIs) under different image processing methods are compared with the cumulative mean temperature (AMT) of the cumulative meteorological characteristics from heading to late grain filling. H-LF ASH cumulative sunshine hours H-LF Accumulated precipitation AP H-LF A random combination of features was used to select the optimal feature combination, and the backpropagation neural network (BPNN) algorithm was employed to construct and independently validate a model for estimating the thousand-grain weight of winter wheat. Finally, the impact of the number of effective viewpoints and variety on the thousand-grain weight estimation model was clarified. Specifically, this included:
[0045] Step 6-1: Calculate the cumulative mean temperature (AMT) from heading to late grain filling by summing the daily average temperature, sunshine duration, and precipitation from the date of acquisition of the H UAV image during the heading stage to the date of acquisition of the LF UAV image during the late grain filling stage. H-LF ASH cumulative sunshine hours H-LF Accumulated precipitation APH-LF It should be noted that for the experiment with different sowing dates, for the first sowing date, the calculation of the three meteorological characteristics starts from the date of acquisition of the drone image at the heading stage minus the difference between the second and first sowing dates. For the third sowing date, the starting date is the date of acquisition of the drone image at the heading stage plus the difference between the third and second sowing dates.
[0046] Step 6-2: Based on the correlation ranking of meteorological factors with thousand-grain weight in previous studies, a strategy of gradually introducing meteorological features is adopted to progressively correlate the three reconstructed sensitive temporal spectral indices (SIs) under different image processing methods with the cumulative mean temperature (AMT). H-LF ASH cumulative sunshine hours H-LF Accumulated precipitation AP H-LF The optimal feature combination was selected and the backpropagation neural network (BPNN) algorithm was used to construct a model for estimating the thousand-grain weight of winter wheat.
[0047] Step 6-3: Conduct independent validation of the winter wheat thousand-grain weight estimation model using independent experiments. The validation results are expressed by the coefficient of determination R. val 2 The root mean square error (RMSE) and relative root mean square error (RRMSE) were evaluated. Simultaneously, the optimal feature combination was used to validate the results and clarify the impact of the number of effective viewpoints and variety on the thousand-grain weight estimation model.
[0048] Compared with the prior art, the present invention, employing the above technical solution, has the following technical effects:
[0049] 1. The present invention provides a method for estimating the thousand-grain weight of winter wheat by combining multi-view images from UAVs with meteorological information. The attention mechanism feature fusion algorithm MV-AMFF developed by the invention is suitable for extracting information from multi-view images from UAVs. It is simple and efficient to operate and can effectively tap the gains brought by the perspective information, thereby improving the upper limit of the spectral estimation accuracy of the thousand-grain weight of winter wheat.
[0050] 2. The present invention provides a method for estimating the thousand-grain weight of winter wheat by combining multi-view images from unmanned aerial vehicles with meteorological information. It reconstructs time-series spectral features, can track the dynamic changes in the grain filling process of winter wheat, improves the spectral estimation accuracy of the thousand-grain weight of winter wheat, and has strong mechanism, simple operation, and strong universality.
[0051] 3. The present invention provides a method for estimating the thousand-grain weight of winter wheat by combining multi-view images from UAVs with meteorological information. This method clarifies the meteorological factors suitable for estimating the thousand-grain weight of winter wheat, realizes its effective coordination with multi-view information, significantly improves the accuracy of the thousand-grain weight estimation of winter wheat, and provides a new technical solution for remote sensing estimation of crop thousand-grain weight by UAVs. Attached Figure Description
[0052] Figure 1This is a technical roadmap for the method of estimating the thousand-grain weight of winter wheat by combining multi-view images from unmanned aerial vehicles with meteorological information according to the present invention.
[0053] Figure 2 This invention relates to a method for estimating the thousand-grain weight of winter wheat using multi-view images from unmanned aerial vehicles (UAVs) and meteorological information, and the distribution of zenith angles observed within the ROIs of P9 and P16.
[0054] Figure 3 This is a schematic diagram of the MV-AMFF algorithm architecture of the method for estimating the thousand-grain weight of winter wheat by combining UAV multi-view imagery and meteorological information according to the present invention.
[0055] Figure 4 Box plots of multi-view reflectance and orthophoto reflectance values under different processing methods in Exps. 1-2 of the present invention's method for estimating the thousand-grain weight of winter wheat using multi-view imagery and meteorological information from unmanned aerial vehicles.
[0056] Figure 5 This invention relates to the method for estimating the thousand-grain weight of winter wheat using multi-view UAV imagery and meteorological information. The spectral indices and thousand-grain weight at different times, derived from orthophotos and multi-view images, are shown in Figures 1-2. 2 value.
[0057] Figure 6 This invention relates to a method for estimating the thousand-grain weight of winter wheat based on the synergy of UAV multi-view imagery and meteorological information, using "SIs + AMT" under a method that reconstructs different temporal features from stitched or multi-view images. H-LF "A scatter plot of the estimated and measured thousand-grain weights in Exp.3 using the combined BPNN model."
[0058] Figure 7 The method for estimating the thousand-grain weight of winter wheat by combining UAV multi-view imagery and meteorological information is derived from MV-AMFF and Comb. #10 of the variation. It uses the BPNN model to create a scatter plot of the thousand-grain weight estimated in Exp. 3 and the measured thousand-grain weight ((A): different number of effective views, (B): different varieties). Detailed Implementation
[0059] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.
[0060] This invention is based on multi-stage UAV multispectral images from heading to late grain filling and winter wheat thousand-grain weight data acquired at maturity under three plot experimental conditions in Suining County, Jiangsu Province (35°56'N, 117°54'E). It is used to develop an adaptive attention mechanism feature fusion algorithm for extracting multi-view UAV image information. Through the effective coordination of multi-view information and meteorological information, the accurate estimation of winter wheat thousand-grain weight is achieved. The basic information of the specific winter wheat plot experiment is shown in Table 1.
[0061] Table 1 Summary of the designs of the three winter wheat plot experiments
[0062] The present invention is based on UAV multispectral imagery and meteorological information. The spectral and meteorological features used are shown in Table 2. The band selection follows the band parameters of the AIRPHEN multispectral sensor.
[0063] Table 2. Spectral and meteorological characteristics used in the study
[0064]
[0065] This implementation focuses on developing an attention mechanism feature fusion algorithm and constructing a thousand-grain weight model for winter wheat based on experimental data from heading to late grain filling in Exps. 1-2. The thousand-grain weight model was independently validated using experimental data from heading to late grain filling in Exps. 3, and the robustness of the model has been evaluated.
[0066] A method for estimating the thousand-grain weight of winter wheat by combining multi-view imagery from unmanned aerial vehicles (UAVs) with meteorological information, such as... Figure 1 As shown, the main components include the acquisition of UAV imagery, meteorological data, and thousand-grain weight; UAV imagery preprocessing and information extraction; reconstruction of time-sensitive spectral index features; selection of the optimal combination of reconstructed sensitive time-series spectral indices and meteorological features; and construction and independent validation of the thousand-grain weight estimation model. Specifically, the steps include:
[0067] Step 1: Acquire AIRPHEN multispectral UAV images of winter wheat from heading to late grain filling in the experimental plot, and obtain orthophoto and multi-view reflectance images after preprocessing; specifically including:
[0068] Step 1-1: Acquire UAV images of the three experimental fields shown in Table 1, covering four stages: winter wheat heading, flowering, pre-filling, and post-filling. The UAV flight parameters are set as follows: flight altitude 30 meters, flight speed 3 meters per second, forward overlap rate and lateral overlap rate of 95% and 90% respectively, and sensor observation angle is 0 degrees, i.e., vertically downward.
[0069] Steps 1-2: The AIRPHEN multispectral UAV imagery undergoes preprocessing in Agisoft PhotoScan Pro software, including image alignment, geometric correction, camera optimization, dense point cloud construction, mesh generation, radiometric correction, and orthorectification, before being exported as orthorectified images. During radiometric correction, a 3 m × 1 m rectangular gray reference plate is used to convert the image's DN values to reflectance. The reference coefficients for the six bands of the AIRPHEN imagery are 7.008% (450 nm), 7.291% (530 nm), 7.239% (570 nm), 7.442% (675 nm), 7.667% (730 nm), and 8.433% (850 nm).
[0070] Steps 1-3: AIRPHEN multispectral UAV imagery undergoes interactive processing in Agisoft PhotoScan Pro and PyCharm software. This processing includes raw image metadata extraction, halo correction, exposure time correction, metadata writing, band synthesis, geometric correction, and radiometric correction preprocessing. The result is a multi-view reflectance image. Metadata extraction is performed by calling code (using Python 3.5) from the Agisoft PhotoScan Pro software's "Console" to extract the metadata stored in the AIRPHEN raw imagery.
[0071] Step 2: Based on multi-view reflectance imagery, select effective viewing angles for each cell and extract the corresponding raw reflectance, observed zenith angle, and quadrant position; specifically including:
[0072] Step 2-1: Use PyCharm software and corresponding algorithms to calculate the coverage of effective wheat canopy pixels in the region of interest (ROI) of each cell in all single multi-view reflectance images, and at the same time extract the average reflectance of each band in the ROI of each cell.
[0073] Step 2-2: Given that the number of plots in the test field covered by a single image in this study is relatively small, a coverage rate of ≥90% within the ROI is selected as the standard for the effective viewing angle of each plot. The effective viewing angle and the corresponding original reflectance of each band are selected according to this standard.
[0074] Steps 2-3: Using PyCharm software and corresponding algorithms, calculate the observation zenith angle of each pixel relative to the center of each image based on the image pixel position, and extract the observation zenith angle and quadrant position Q of the corresponding ROI center coordinates in all effective viewpoint images of each cell to characterize the observation zenith angle and quadrant of each ROI. The calculation steps for the image observation zenith angle VZA include:
[0075] Step 2-3-1: Read the image information, including image size, spatial resolution, image center pixel coordinates, coordinates of any pixel, latitude of the southernmost and northernmost points of the image, and sensor flight altitude;
[0076] Step 2-3-2: Define the center pixel C as the zenith direction, i.e., VZA=0°, and let the projection of the viewing distance offset of any pixel P relative to the center C onto the horizontal plane be r;
[0077] Step 2-3-3: Calculate the latitude correction factor d, using the following formula:
[0078]
[0079]
[0080] Where latitude is the center latitude of the geographic range of the image, Ymin is the southernmost latitude of the image, Ymax is the northernmost latitude of the image, and R is the Earth's radius of 6,371,000 meters.
[0081] Steps 2-3-4: Calculate the geographic offset distance dx of any pixel P relative to the image center C in the east-west direction and the geographic offset distance dy in the north-south direction, using the following formulas:
[0082]
[0083]
[0084] Where cols and rows are the number of columns and rows of pixels in the image size, respectively; cell_w and cell_h represent the size of the spatial resolution, that is, the geographic width and height of the pixel, in meters; and i and j represent the coordinate position of any pixel P in the image.
[0085] Steps 2-3-5: Calculate the horizontal projection distance r, using the following formula:
[0086]
[0087] Step 2-3-6: Calculate the observed zenith angle VZA, using the following formula:
[0088]
[0089] Where h refers to the sensor's flight altitude, and in the test data of this invention, h is set to 30 meters.
[0090] like Figure 2 As shown, the observation zenith angle and observation quadrant position are different for different cells in a single multi-view image.
[0091] Step 3: As Figure 3As shown, an attention-based neural network architecture is used as the core fusion model, and the measured thousand-grain weight is used as the supervised target for end-to-end learning. Three parallel feature transformation branches are used to process the original reflectivity, observed zenith angle, and quadrant position information, respectively, thus developing an attention-based feature fusion algorithm MV-AMFF for UAV multi-view imagery; specifically including:
[0092] Step 3-1: The MV-AMFF method uses a dedicated neural network architecture based on an attention mechanism as its core fusion model;
[0093] Step 3-2: The model processes the original reflectance, observed zenith angle and quadrant position information through three parallel feature transformation branches. The reflectance branch maps the spectral data of the AIRPHEN sensor in six bands to the hidden space through a two-layer fully connected network. The observed zenith angle branch transforms the data into high-dimensional features through a separate neural network. The quadrant branch converts discrete position categories into dense vectors through an embedding layer.
[0094] Step 3-3: The reflectance, observed zenith angle, and quadrant position information, which have been processed separately by the branch attention mechanism, are concatenated with features and then the attention score of reflectance at each observed zenith angle and quadrant is calculated by a neural network containing the "tanh" activation function.
[0095] Steps 3-4: Use the "softmax" function to normalize the attention score into attention weights, and use these weights to perform a weighted fusion of the original reflectance while preserving its physical meaning;
[0096] Steps 3-5: Using the mean square error (MSE) between the estimated thousand-grain weight and the measured thousand-grain weight as the loss function, backpropagation drives the fusion model to perform "end-to-end" learning, optimizing the weight allocation strategy so that spectral features from different quadrant positions and observed zenith angles can obtain differentiated weights based on their effectiveness in indicating the thousand-grain weight, thereby achieving dynamic and adaptive feature fusion and generating a new reflectance that is more beneficial for thousand-grain weight estimation.
[0097] Step 4: Three methods are used to extract the reflectance of UAV orthophotos and multi-view images, and basic statistical analysis is performed. The method for extracting reflectance from orthophotos is denoted as Mosaic, while two different methods are used to extract reflectance from multi-view images: the MV-mean method and the developed MV-AMFF algorithm; specifically including:
[0098] Step 4-1: Extract the reflectance of the orthophoto using the Mosaic method. Specifically, this involves using PyCharm software to extract the average reflectance of each band of the mosaic orthophoto within each region of interest (ROI) as the reflectance of each cell.
[0099] Step 4-2: The reflectance of multi-view images is extracted using the MV-mean method. Specifically, the average value of the reflectance of each band under all effective views of each cell obtained in Step 2 is calculated as the reflectance data corresponding to the thousand-grain weight of each cell.
[0100] Step 4-3: Use the MV-AMFF algorithm to extract the reflectance of multi-view images. Specifically, the original reflectance, observation zenith angle and quadrant position information of all effective views of each cell obtained in Step 2 are used to run the MV-AMFF algorithm developed in Step 3 in PyCharm software, and its output value is used as the reflectance data corresponding one-to-one with the measured thousand-grain weight of each cell.
[0101] Step 4-4: As Figure 4 As shown, box plots of reflectance extracted by the three methods clearly show the differences in reflectance, mainly exhibiting the phenomenon of MV-AMFF > MV-mean > Mosaic, indicating that reflectance is significantly affected by different zenith observation angles.
[0102] Step 5: Based on the three types of reflectance data obtained in Step 4, calculate 15 conventional spectral indices. Through correlation analysis, select three spectral indices (SIs) sensitive to thousand-grain weight. Then, use three time-series feature reconstruction methods—original value, cumulative value, and change—to process the time-series sensitive spectral indices (SIs) and obtain the reconstructed time-series spectral indices (SIs). Specifically, this includes:
[0103] Step 5-1: Based on the three types of reflectance data obtained in Step 4, calculate 15 conventional spectral indices according to the formulas shown in Table 2, such as... Figure 5 As shown, through correlation analysis (evaluation index: the square of the correlation coefficient r) 2 Three spectral indices (SIs) sensitive to thousand-grain weight were selected from these: BNDVI, NEI, and NDRE.
[0104] Step 5-2: Using three time-series feature reconstruction methods—original value, cumulative value, and change amount—the time-series sensitive spectral indices (SIs) are processed to obtain the reconstructed sensitive time-series spectral indices (SIs). The original value refers to the SIs value obtained directly from each period from heading to the late grain-filling stage. The cumulative value refers to the sum of SIs obtained from all periods from heading to the late grain-filling stage. The change amount refers to the difference in SIs obtained from adjacent periods (the later period minus the earlier period).
[0105] Step 6: The three reconstructed sensitive temporal spectral indices (SIs) under different image processing methods are compared with the cumulative mean temperature (AMT) of the cumulative meteorological characteristics from heading to late grain filling. H-LF ASH cumulative sunshine hours H-LF Accumulated precipitation AP H-LFA random combination of features was used to select the optimal feature combination, and the backpropagation neural network (BPNN) algorithm was employed to construct and independently validate a model for estimating the thousand-grain weight of winter wheat. Finally, the impact of the number of effective viewpoints and variety on the thousand-grain weight estimation model was clarified. Specifically, this included:
[0106] Step 6-1: Calculate the cumulative mean temperature (AMT) from heading to late grain filling by summing the daily average temperature, sunshine duration, and precipitation from the date of acquisition of the H UAV image during the heading stage to the date of acquisition of the LF UAV image during the late grain filling stage. H-LF ASH cumulative sunshine hours H-LF Accumulated precipitation AP H-LF It should be noted that for the experiment with different sowing dates, for the first sowing date, the calculation of the three meteorological characteristics starts from the date of acquisition of the drone image at the heading stage minus the difference between the second and first sowing dates. For the third sowing date, the starting date is the date of acquisition of the drone image at the heading stage plus the difference between the third and second sowing dates.
[0107] Step 6-2: Based on the correlation ranking of meteorological factors with thousand-grain weight in previous studies, a strategy of gradually introducing meteorological features was adopted. The three reconstructed sensitive temporal spectral indices (SIs) under different image processing methods were gradually combined with cumulative mean temperature (AMTH-LF), cumulative sunshine hours (ASHH-LF), and cumulative precipitation (APH-LF). Based on the Exps. 1+2 dataset, the thousand-grain weight of wheat was estimated using the backpropagation neural network (BPNN) algorithm. This resulted in the conclusion that under different image processing and temporal feature reconstruction methods, the result is always "SIs + AMT". H-LF "The combined performance is better than combinations containing other meteorological features, and the best combination is 'SIs + AMT' under image processing and temporal spectral feature reconstruction using the MV-AMFF method." H-LF "combination;
[0108] Step 6-3: Conduct independent validation of the winter wheat thousand-grain weight estimation model using independent experiments. The validation results are expressed by the coefficient of determination R. val 2 Evaluation methods include Root Mean Square Error (RMSE) and Relative Root Mean Square Error (RRMSE). Figure 6 As shown, the thousand-grain weight model was validated based on the Exp. 3 dataset, revealing that the MV-AMFF method still yields the best image processing and time-series feature reconstruction results under different image processing and time-series spectral feature reconstruction methods, specifically the "SIs + AMT" model. H-LF The combination exhibited the highest verification accuracy (R0). val 2 = 0.55, RMSE = 2.15 g, RRMSE = 4.64%. Meanwhile, as... Figure 7As shown, in the analysis of the impact of the number of effective perspectives and variety on the thousand-grain weight estimation model based on the validation results of the optimal feature combination, it was observed that the thousand-grain weight estimation method provided by this invention has excellent estimation accuracy for most wheat varieties when the number of effective perspectives is sufficient (eVZA_N ≥ 20). At the same time, this also reveals the potential impact of variety differences on the estimation results; for example, identifiable deviation patterns were observed in specific varieties such as V4 (Ningmai 13) and V5 (Yangmai 16). This finding further clarifies the applicability conditions of the method of this invention and provides a clear direction for targeted optimization of the model.
[0109] The above description is only a partial embodiment of the present invention. It should be noted that those skilled in the art can make several improvements without departing from the principle of the present invention, and these improvements should be within the scope of protection of the present invention.
Claims
1. A method for estimating the thousand-grain weight of winter wheat by combining multi-view imagery from unmanned aerial vehicles (UAVs) with meteorological information, comprising the following steps: Step 1: Acquire AIRPHEN multispectral UAV images of winter wheat from heading to late grain filling in the plot to be tested, and preprocess to obtain multi-view reflectance images; Step 2: Based on multi-view reflectance images, select effective viewing angles for each cell and extract the corresponding raw reflectance, observed zenith angle, and quadrant position; Step 3: Process the original reflectivity, observed zenith angle and quadrant position information of each cell's effective viewing angle obtained in Step 2 using the attention mechanism feature fusion algorithm MV-AMFF to obtain a new reflectivity that corresponds one-to-one with the measured thousand-grain weight; Step 4: Calculate 15 conventional spectral indices using the new reflectance data output by the MV-AMFF algorithm in Step 3. Select 3 spectral indices SIs that are sensitive to thousand-grain weight through correlation analysis. Then, process the time-sensitive spectral indices SIs using the change amount time-series feature reconstruction method to obtain the reconstructed time-series spectral indices SIs. Step 5: Reconstruct the time-series spectral indices (SIs) from the changes and combine them with the cumulative mean temperature (AMT) of the cumulative meteorological characteristics from heading to late grain filling. H-LF The optimal feature combination, along with the corresponding measured thousand-grain weight, is input into the backpropagation neural network (BPNN) algorithm for training, in order to construct a winter wheat thousand-grain weight estimation model and achieve accurate estimation of winter wheat thousand-grain weight.
2. The method for estimating the thousand-grain weight of winter wheat by combining multi-view UAV imagery with meteorological information according to claim 1, characterized in that, Step 1 specifically includes: Step 1-1: The drone imagery acquisition period includes four stages for winter wheat: heading stage, flowering stage, pre-grain filling stage, and post-grain filling stage. Steps 1-2: AIRPHEN multispectral UAV images undergo raw image metadata extraction, halo correction, exposure time correction, metadata writing, band synthesis, geometric correction, and radiometric correction preprocessing to obtain all individual images, which are multi-view reflectance images.
3. The method for estimating the thousand-grain weight of winter wheat by combining multi-view UAV imagery with meteorological information according to claim 1, characterized in that, Step 2 specifically includes: Step 2-1: Calculate the coverage of effective wheat canopy pixels within the region of interest (ROI) of each cell in all single multi-view reflectance images, and simultaneously extract the average reflectance of each band within the ROI of each cell; Step 2-2: Select coverage rate ≥90% within ROI as the standard for effective viewing angle of each cell, and select the effective viewing angle and corresponding original reflectance of each band according to this standard; Steps 2-3: Calculate the observation zenith angle of each pixel relative to the center of each image, and extract the observation zenith angle and quadrant position Q of the corresponding ROI center coordinates in the effective view image of each cell to characterize the observation zenith angle and quadrant of each ROI.
4. The method for estimating the thousand-grain weight of winter wheat by combining multi-view UAV imagery with meteorological information according to claim 3, characterized in that, In steps 2-3, the calculation of the observed zenith angle is based on a geometric optics model. The observed zenith angle VZA of each pixel relative to the image center is calculated in real-time using the pixel coordinates. Specifically, this includes: Step 2-3-1: Read the image information, including image size, spatial resolution, image center pixel coordinates, coordinates of any pixel, latitude of the southernmost and northernmost points of the image, and sensor flight altitude; Step 2-3-2: Define the center pixel C as the zenith direction, i.e., VZA=0°, and let the projection of the viewing distance offset of any pixel P relative to the center C onto the horizontal plane be r; Step 2-3-3: Calculate the latitude correction factor d, using the following formula: Where latitude is the center latitude of the geographic range of the image, Ymin is the southernmost latitude of the image, Ymax is the northernmost latitude of the image, and R is the Earth's radius of 6,371,000 meters. Steps 2-3-4: Calculate the geographic offset distance dx of any pixel P relative to the image center C in the east-west direction and the geographic offset distance dy in the north-south direction, using the following formulas: Where cols and rows are the number of columns and rows of pixels in the image size, respectively; cell_w and cell_h represent the size of the spatial resolution, that is, the geographic width and height of the pixel, in meters; and i and j represent the coordinate position of any pixel P in the image. Steps 2-3-5: Calculate the horizontal projection distance r, using the following formula: Steps 2-3-6: Calculate the observed zenith angle VZA, where h represents the sensor's flight altitude. The formula is as follows:
5. The method for estimating the thousand-grain weight of winter wheat by combining multi-view UAV imagery with meteorological information according to claim 1, characterized in that, Step 3 specifically includes: Step 3-1: The MV-AMFF algorithm uses a dedicated neural network architecture based on an attention mechanism as its core fusion model; Step 3-2: The fusion model processes the original reflectance, observed zenith angle, and quadrant position information through three parallel feature transformation branches, which are denoted as the reflectance branch, observed zenith angle branch, and quadrant branch, respectively. The reflectance branch maps the spectral data of the 6 bands of the AIRPHEN sensor to the hidden space through a two-layer fully connected network. The observed zenith angle branch transforms the data into high-dimensional features through a separate neural network. The quadrant branch converts the discrete position categories into dense vectors through an embedding layer. Step 3-3: The reflectance, observed zenith angle, and quadrant position information, after being processed by the feature transformation branch, are concatenated and then the attention score of reflectance at each observed zenith angle and quadrant is calculated by a neural network containing the "tanh" activation function. Steps 3-4: Use the "softmax" function to normalize the attention score into attention weights, and use these weights to perform a weighted fusion of the original reflectance while preserving its physical meaning; Steps 3-5: Using the mean square error (MSE) between the estimated thousand-grain weight and the measured thousand-grain weight as the loss function, backpropagation drives the fusion model to perform "end-to-end" learning, optimizing the weight allocation strategy. This allows spectral features from different quadrant positions and observed zenith angles to obtain differentiated weights based on their effectiveness in indicating the thousand-grain weight, thereby achieving dynamic and adaptive feature fusion and generating a new reflectance that is more conducive to the estimation of the thousand-grain weight.
6. The method for estimating the thousand-grain weight of winter wheat by combining multi-view UAV imagery with meteorological information according to claim 5, characterized in that, The AIRPHEN sensor has six wavelength bands: 450 nm, 530 nm, 570 nm, 675 nm, 730 nm and 850 nm.
7. The method for estimating the thousand-grain weight of winter wheat by combining multi-view UAV imagery with meteorological information according to claim 1, characterized in that, Step 4 specifically includes: Step 4-1: Calculate 15 conventional spectral indices using the new reflectance data output by the MV-AMFF algorithm, and screen out 3 spectral indices SIs that are sensitive to thousand-grain weight in all four stages from heading to late grain filling through correlation analysis. Step 4-2: Process the time-sensitive spectral indices (SIs) using the change amount time-series feature reconstruction method to obtain the time-series spectral indices (SIs) reconstructed by the change amount; the change amount refers to the difference between the SIs obtained using adjacent periods.
8. The method for estimating the thousand-grain weight of winter wheat by combining multi-view UAV imagery with meteorological information according to claim 7, characterized in that, The 15 conventional spectral indices are: Normalized Difference Vegetation Index (NDVI), Normalized Difference Red Edge Index (NDRE), Two-Band Enhanced Vegetation Index (EVI2), and Green Chlorophyll Index (CI). green Red-edged chlorophyll index (CI) red-edge Optimized soil-adjusted vegetation index (OSAVI), blue light normalized difference vegetation index (BNDVI), renormalized difference vegetation index (RDVI), and optimized vegetation index (VI). opt Optimize the nonlinear vegetation index ONLI, nitrogen efficiency index NEI, and DATT. [850,730,675] Three-band vegetation index TBVI [850,730,450] Corrected blue light normalized difference ratio mND blue Atmospheric Impedance Index (VARI) for Green and Visible Light green .
9. The method for estimating the thousand-grain weight of winter wheat by combining multi-view imagery from unmanned aerial vehicles with meteorological information according to claim 1, characterized in that, Step 5 specifically includes: Step 5-1: Calculate the cumulative mean temperature (AMT) from heading to late grain filling by summing the daily average temperatures from the date of acquisition of the H UAV image during the heading stage to the date of acquisition of the LF UAV image during the late grain filling stage. H-LF ; Step 5-2: Reconstruct the time-series spectral indices (SIs) from the changes, and compare them with the cumulative mean temperature (AMT) of the cumulative meteorological characteristics from heading to late grain filling. H-LF The optimal feature combination, along with the corresponding measured thousand-grain weight, is input into the backpropagation neural network (BPNN) algorithm for training, in order to construct a winter wheat thousand-grain weight estimation model and achieve accurate estimation of winter wheat thousand-grain weight. Step 5-3: Using the coefficient of determination R val 2 The root mean square error (RMSE) and relative root mean square error (RRMSE) are used to evaluate the model validation results.
10. The method for estimating the thousand-grain weight of winter wheat by combining multi-view UAV imagery with meteorological information according to claim 9, characterized in that, In step 5-1, for experiments with different sowing dates, the meteorological characteristics are calculated starting from the date of acquisition of the drone image during the heading stage minus the difference between the second and first sowing dates. For the third sowing date, the starting date is the date of acquisition of the drone image during the heading stage plus the difference between the third and second sowing dates.