Crop yield estimation method based on hyperspectral unmanned aerial vehicle

Abstract

The invention discloses a crop yield estimation method based on a hyperspectral unmanned aerial vehicle, and belongs to crop yield estimation, and the method comprises the steps: S100, enabling an unmanned aerial vehicle to carry a hyperspectral camera to obtain a hyperspectral image of a crop, and synchronously collecting a spectral target and actual measurement yield data on the ground; s200, constructing a radiation transmission model in combination with POS data of the unmanned aerial vehicle, eliminating the influence of aerosol scattering by adopting a Monte Carlo atmospheric correction algorithm, and performing radiation correction on the hyperspectral image; s300, feature extraction is carried out on the hyperspectral image after radiation correction, feature extraction results are spliced into a multi-dimensional feature vector, the multi-dimensional feature vector corresponds to actually measured yield data, and a feature set is generated; s400, constructing and training a multi-task space-time Transform prediction model on the basis of the feature set; and S500, crop yield estimation is carried out through the multi-task space-time Transform prediction model.

Description

Technical Field

[0001] The present invention relates to the technical field of crop yield estimation, and more particularly to a crop yield estimation method based on a hyperspectral unmanned aerial vehicle (UAV). Background Art

[0002] In recent years, accurate crop yield estimation has become crucial for agricultural production planning, food security, and market dynamics. Traditional field sampling yield measurement methods are inefficient and limited when used for large-scale estimation. With the rapid development of drones and hyperspectral remote sensing technology, yield estimation methods based on hyperspectral drones have opened up new avenues for achieving high-precision yield estimation over large areas.

[0003] However, existing hyperspectral-based yield estimation methods still have some flaws. Crop yield estimation methods based on statistical models, such as regression analysis and vegetation indices, are simple in principle and easy to implement, but typically require a large amount of measured data for parameter correction and have poor adaptability to complex environments. Furthermore, a single vegetation index often fails to fully reflect the complexity of crop growth conditions. Machine learning-based crop yield estimation methods, such as support vector machines, random forests, and deep learning, can exploit the nonlinear relationship between remote sensing data and yield, improving estimation accuracy.

[0004] Therefore, how to provide a crop yield estimation method based on hyperspectral drones is an urgent problem that needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the present invention provides a crop yield estimation method based on hyperspectral UAV to solve the technical problems existing in the above-mentioned prior art.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A crop yield estimation method based on hyperspectral drones, including:

[0008] S100: The drone is equipped with a hyperspectral camera to acquire hyperspectral images of crops and simultaneously collect spectral targets and measured yield data on the ground;

[0009] S200: Combines drone POS data to build a radiation transfer model, uses a Monte Carlo atmospheric correction algorithm to eliminate the influence of aerosol scattering, and performs radiation correction on hyperspectral images;

[0010] S300: Extract features from the radiometrically corrected hyperspectral image, concatenate the feature extraction results into a multi-dimensional feature vector, and correspond it to the measured yield data to generate a feature set;

[0011] S400: Build and train a multi-task spatiotemporal Transformer prediction model based on the feature set;

[0012] S500: Crop Yield Estimation via a Multi-Task Spatio-Temporal Transformer Prediction Model.

[0013] Furthermore, S100 includes:

[0014] S110: Uses a hyperspectral camera to capture crop reflectance information in different wavelengths;

[0015] S120: Set up the ground synchronous spectral target array and collect data;

[0016] S130: Measure ground crop yields and collect ground crop yield measurement data.

[0017] Furthermore, the radiation transfer model constructed is:

[0018]

[0019] Where L(τ, μ, φ) represents the radiance, L0(τ, μ, φ) represents the initial radiance, τ represents the optical depth, μ represents the cosine of the zenith angle, and φ represents the azimuth angle; ω(τ′) represents the single scattering albedo, P(μ, φ; μ′, φ′) represents the scattering phase function, and e -τ / μ Represents the attenuation factor.

[0020] Furthermore, feature extraction of the radiometrically corrected hyperspectral image includes normalized vegetation index extraction and red edge parameter extraction.

[0021] Furthermore, the architecture of the multi-task spatiotemporal Transformer prediction model includes: an input layer, a spatiotemporal position encoding, an encoder layer, a decoder layer, and a multi-task joint loss function;

[0022] Among them, the input layer converts the feature set into a vector representation of the input, the spatiotemporal position encoding assigns a unique spatial position identifier, and the spatiotemporal position encoding performs spatial encoding and temporal encoding to obtain the total input; the encoder layer includes a multi-head self-attention mechanism, a feedforward neural network, a residual connection and layer normalization, and captures the complex dependency between spectral features and yield through the multi-head self-attention mechanism, and uses the feedforward neural network to perform nonlinear transformation on the attention output to enhance the model expression ability. The multi-head attention mechanism calculates multiple attention subspaces in parallel to capture dependency patterns of different granularities; the decoder layer gradually generates prediction results based on the information output by the encoder, and the multi-task joint loss function jointly optimizes the yield prediction task through weighted summation to output the yield prediction result.

[0023] Furthermore, it also includes: mapping the production values ​​predicted by the multi-task spatiotemporal Transformer prediction model to geographic space to generate a production distribution map;

[0024] The root mean square error between the predicted yield and the measured value is calculated, and the final yield estimation result is obtained based on the root mean square error.

[0025] Furthermore, the formula for calculating the root mean square error between the predicted output and the measured value is:

[0026]

[0027] Where y i is the measured value, is the predicted value, and n is the number of samples.

[0028] It can be seen from the above technical solution that compared with the existing technology, the present invention discloses a crop yield estimation method based on hyperspectral UAV, which combines hyperspectral UAV technology, radiation correction, feature extraction and multi-task spatiotemporal Transformer model, and can achieve more accurate and efficient crop yield estimation, providing strong technical support for precision agriculture. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0030] Figure 1 Schematic diagram of the method of the present invention. DETAILED DESCRIPTION

[0031] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0032] See also Figure 1 The embodiment of the present invention discloses a method for estimating crop yield based on hyperspectral drone, comprising:

[0033] S100: The drone is equipped with a hyperspectral camera to acquire hyperspectral images of crops and simultaneously collect spectral targets and measured yield data on the ground;

[0034] S200: Combines drone POS data to build a radiation transfer model, uses a Monte Carlo atmospheric correction algorithm to eliminate the influence of aerosol scattering, and performs radiation correction on hyperspectral images;

[0035] S300: Extract features from the radiometrically corrected hyperspectral image, concatenate the feature extraction results into a multi-dimensional feature vector, and correspond it to the measured yield data to generate a feature set;

[0036] S400: Build and train a multi-task spatiotemporal Transformer prediction model based on the feature set;

[0037] S500: Crop Yield Estimation via a Multi-Task Spatio-Temporal Transformer Prediction Model.

[0038] Specifically, the S100 provides a wealth of spectral information. Simultaneously, ground-based spectral target and measured yield data are collected, providing an accurate reference for subsequent image correction and model training, ensuring data quality and accuracy, including:

[0039] S110: Uses a hyperspectral camera to capture crop reflectance information in different wavelengths;

[0040] S120: Set up the ground synchronous spectral target array and collect data;

[0041] S130: Measure ground crop yields and collect ground crop yield measurement data.

[0042] Hyperspectral Camera Setup: The hyperspectral camera is fully calibrated in advance to ensure accurate coverage of visible light (wavelength range approximately 400-760nm), near-infrared (760-2500nm), and short-wave infrared (2500-4000nm), with the ability to stably acquire reflectance information from dozens to hundreds of consecutive bands. Different bands have unique value in detecting the physiological characteristics and growth status of crops. For example, the near-infrared band is sensitive to vegetation health and biomass, while the visible light band can be used to identify crop color and morphological characteristics.

[0043] Flight parameter planning: A computer system uses a specific algorithm to calculate and set the drone's flight altitude within a range of 50-200 meters based on key parameters such as the hyperspectral camera's resolution (such as the ground sampling distance (GSD), which determines the actual size of the ground represented by each pixel in the image) and the field of view (FOV, which affects the camera's shooting range). Furthermore, a geographic information system (GIS) technology is used to plan a predetermined flight route, taking into account the geographic boundaries and shape of the crop area to be monitored. This ensures that the flight path evenly covers the target area and that adjacent flight paths have an appropriate overlap rate (generally 60%-80%) to ensure the integrity and continuity of the image data.

[0044] Image acquisition operation: The UAV flies at a stable flight speed (usually 3-10 meters per second, which takes into account both image clarity and acquisition efficiency) above the crop area according to the planned route and set altitude. During this period, the hyperspectral camera continues to work with a set exposure time (determined according to lighting conditions and camera performance, generally between a few milliseconds and tens of milliseconds) and acquisition frequency (for example, 1-5 images are acquired per second to ensure that the displacement between adjacent images does not lose key information) to perform hyperspectral image acquisition operations on the crop area, thereby obtaining high-quality and comprehensive image data, ensuring that the spatial resolution of the image is sufficient to identify individual crops or small crop groups. The image data is stored in the data storage module of the UAV in a specific format (such as the common HDF5 format, which can effectively store the multidimensional information of hyperspectral data) and can be transmitted to the ground computer system for subsequent processing after the flight mission is completed.

[0045] Specifically, the ground-based synchronous spectral target array setup and data acquisition includes:

[0046] Target array arrangement: Researchers carefully designed and evenly arranged ground-based synchronous spectral target arrays made of standard materials with different reflectivity characteristics within the drone flight area, based on the uniformity of crop distribution and the complexity of the terrain. Target materials usually include Teflon, barium sulfate, etc. with known and stable reflectivity, and the values ​​of their reflectivity in different bands can be obtained through standard laboratory measurements. The target shape is generally square or circular, with a side length or diameter between 0.5 and 2 meters, so that it can be clearly identified in hyperspectral images. The layout of the target array must follow certain geometric rules, such as a grid or chessboard arrangement. The distance between adjacent targets is determined according to the drone's flight altitude and image resolution, generally between 5 and 20 meters, to ensure that each target in the image can be independently identified and evenly distributed.

[0047] Synchronous measurement operation: During the flight of the UAV, researchers operate a ground-based spectrometer (such as the ASDFieldSpec series spectrometer, which has high-precision spectral measurement capabilities) to perform synchronous measurements on the target. The spectrometer needs to be calibrated in advance to ensure measurement accuracy. During measurement, the probe of the spectrometer is vertically aligned with the center of the target, and an appropriate measurement distance is maintained (generally 1-2 meters to ensure that the measurement field of view completely covers the target). The actual spectral reflectance data of the target is obtained under the same time and environmental conditions. Each target needs to be measured multiple times (generally 3-5 times), and the average value is taken to improve the accuracy of the data. The measurement data is recorded in real time in the data storage device of the spectrometer, and subsequently matched and associated with the hyperspectral image data collected by the UAV.

[0048] Specifically, the radiation transfer model constructed is:

[0049]

[0050] where L(τ, μ, φ) represents the radiance, L0(τ, μ, φ) represents the initial radiance, τ represents the optical depth, μ represents the cosine of the zenith angle, and φ represents the azimuth angle; ω(τ′) represents the single scattering albedo, P(μ, φ; μ′, φ′) represents the scattering phase function, and e-τ / μ represents the attenuation factor.

[0051] Specifically, the Monte Carlo atmospheric correction algorithm can be used to perform radiation correction on hyperspectral images. The specific steps are as follows:

[0052] Parameter initialization: Set the atmospheric model parameters, including atmospheric composition, aerosol type and concentration, and solar position; determine the number of simulated photons and their initial position (usually based on the sensor position).

[0053] Photon propagation simulation: For each simulated photon, the scattering or absorption behavior during propagation is randomly determined based on the atmospheric scattering and absorption probability model. If scattering occurs, the scattering direction is determined based on the scattering phase function; if absorption occurs, the photon propagation is terminated.

[0054] Statistical calculation: After a large number of photon propagation simulations are completed, the number and energy of photons reaching the ground and returning to the sensor are counted, and the atmospheric radiation and atmospheric transmittance are calculated based on this.

[0055] Radiation correction: Based on the obtained atmospheric parameters, radiation transfer model is used to perform radiation correction on each pixel of the hyperspectral image to obtain the corrected ground reflectance image.

[0056] Through radiation correction, the impact of the sensor itself and environmental factors such as lighting on the image is eliminated, which improves the reliability and consistency of the data and lays the foundation for subsequent feature extraction and model training.

[0057] Specifically, feature extraction of the radiometrically corrected hyperspectral image includes: normalized vegetation index extraction and red edge parameter extraction.

[0058] Specifically, the Normalized Difference Vegetation Index (NDVI) and red edge parameters (such as red edge position REP and red edge amplitude Dr) are key spectral features reflecting crop growth status and are significantly correlated with yield. The multi-task spatiotemporal Transformer model captures the spatiotemporal dependencies between data through a self-attention mechanism. It can simultaneously process spectral features (spatial dimension) and yield data (temporal dimension), enabling feature fusion and yield prediction.

[0059] Specifically, the architecture of the multi-task spatiotemporal Transformer prediction model includes: input layer, spatiotemporal position encoding, encoder layer, decoder layer, and multi-task joint loss function;

[0060] Among them, the input layer converts the feature set into a vector representation of the input, the spatiotemporal position encoding assigns a unique spatial position identifier, and the spatiotemporal position encoding performs spatial encoding and temporal encoding to obtain the total input; the encoder layer includes a multi-head self-attention mechanism, a feedforward neural network, a residual connection and layer normalization, and captures the complex dependency between spectral features and yield through the multi-head self-attention mechanism, and uses the feedforward neural network to perform nonlinear transformation on the attention output to enhance the model expression ability. The multi-head attention mechanism calculates multiple attention subspaces in parallel to capture dependency patterns of different granularities; the decoder layer gradually generates prediction results based on the information output by the encoder, and the multi-task joint loss function jointly optimizes the yield prediction task through weighted summation to output the yield prediction result.

[0061] Specifically, the multi-task spatiotemporal Transformer prediction model input adaptation: NDVI (0-1 range) and red edge parameters (wavelength value or dimensionless) can be mapped to the model input space through normalization processing (such as Z-score).

[0062] Specifically, the task compatibility of the multi-task spatiotemporal Transformer prediction model: the model can be designed to simultaneously predict yield (regression task) and crop type (classification task), share encoder features, and improve generalization.

[0063] The multi-task spatiotemporal Transformer prediction model of this invention can simultaneously learn and predict multiple tasks, leveraging spatiotemporal information to improve prediction accuracy. The Transformer model's powerful feature extraction and modeling capabilities allow it to better capture the complex relationship between crop yield and hyperspectral data, improving the model's generalization and adaptability to diverse crop types and environmental conditions.

[0064] Specifically, the input layer includes:

[0065] The results of feature extraction are spliced ​​into a multidimensional feature vector X spectral ∈R N×D , where N is the number of plots and D is the feature dimension (e.g. 3D corresponds to NDVI, REP, and Dr);

[0066] By linear transformation W embed ∈R D×dmodel Mapping to model dimensions:

[0067] E spectral =X spectral W embed .

[0068] Specifically, the spatiotemporal position coding includes:

[0069] Calculate the spatiotemporal position code E spatial ;

[0070] Through the learnable embedding layer E temporal ∈R T×dmodel The reproductive stage is coded, and T is the total number of stages;

[0071] Get the total input:

[0072] E input =E spectral +E spatial +E temporal .

[0073] Specifically, the complex dependency between spectral features and yield is captured through a multi-head self-attention mechanism, as follows:

[0074] Divide the input into h heads and calculate self-attention in parallel:

[0075]

[0076] Where Q is the query vector Q = E input WQ, K is a key, K=E input WK, V is the value vector, V = E input WV,dk=d model / h.

[0077] Specifically, a feedforward neural network is used to perform nonlinear transformation on the attention output to enhance the model's expressiveness. It consists of two fully connected layers with ReLU as the activation function:

[0078] FFN(x)=ReLU(xW1+b1)W2+b2;

[0079] Where x is the input of the feedforward neural network (FFN), W1 is the weight matrix of the first fully connected layer; b1 is the bias vector of the first fully connected layer; W2 is the weight matrix of the second fully connected layer; b2 is the bias vector of the second fully connected layer.

[0080] Specifically, the multi-head attention mechanism calculates multiple attention subspaces in parallel to capture dependency patterns of different granularities; residual connections and layer normalization (LayerNorm) are added after each sublayer to stabilize the training process.

[0081] Specifically, it also includes: mapping the production values ​​predicted by the multi-task spatiotemporal Transformer prediction model to geographic space to generate a production distribution map;

[0082] The root mean square error between the predicted yield and the measured value is calculated, and the final yield estimation result is obtained based on the root mean square error.

[0083] Specifically, the formula for calculating the root mean square error between the predicted output and the measured value is:

[0084]

[0085] Where y i is the measured value, is the predicted value, and n is the number of samples.

[0086] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0087] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for estimating crop yield based on hyperspectral drones, characterized in that: include: S100: The drone is equipped with a hyperspectral camera to acquire hyperspectral images of crops and simultaneously collect spectral targets and measured yield data on the ground; S2 00: Build a radiation transfer model based on drone POS data, use the Monte Carlo atmospheric correction algorithm to eliminate the influence of aerosol scattering, and perform radiation correction on hyperspectral images; S300: Extract features from the radiometrically corrected hyperspectral image, concatenate the feature extraction results into a multi-dimensional feature vector, and correspond it to the measured yield data to generate a feature set; S400: Build and train a multi-task spatiotemporal Transformer prediction model based on the feature set; S500: Crop Yield Estimation via a Multi-Task Spatio-Temporal Transformer Prediction Model.

2. The method for estimating crop yield based on hyperspectral UAV according to claim 1, characterized in that: S100 includes: S110: Uses a hyperspectral camera to capture crop reflectance information in different wavelengths; S120: Set up the ground synchronous spectral target array and collect data; S130: Measure ground crop yields and collect ground crop yield measurement data.

3. The method for estimating crop yield based on hyperspectral UAV according to claim 1, characterized in that: The radiation transfer model constructed is: where L((τ, μ, φ)) represents the radiance and L0((τ, μ, φ)) represents the initial radiance, where τ represents the optical depth, μ represents the cosine of the zenith angle, and φ represents the azimuth angle; ω(τ′) represents the single scattering albedo, P(μ, φ; μ′, φ′) represents the scattering phase function, and e -τ / μ Represents the attenuation factor.

4. The method for estimating crop yield based on hyperspectral UAV according to claim 1, characterized in that: Feature extraction of radiometrically corrected hyperspectral images includes normalized vegetation index extraction and red edge parameter extraction.

5. The method for estimating crop yield based on hyperspectral UAV according to claim 4, characterized in that: The architecture of the multi-task spatiotemporal Transformer prediction model includes: an input layer, a spatiotemporal position encoding, an encoder layer, a decoder layer, and a multi-task joint loss function; Among them, the input layer converts the feature set into a vector representation of the input, the spatiotemporal position encoding assigns a unique spatial position identifier, and the spatiotemporal position encoding performs spatial encoding and temporal encoding to obtain the total input; the encoder layer includes a multi-head self-attention mechanism, a feedforward neural network, a residual connection and layer normalization, and captures the complex dependency between spectral features and yield through the multi-head self-attention mechanism, and uses the feedforward neural network to perform nonlinear transformation on the attention output to enhance the model expression ability. The multi-head attention mechanism calculates multiple attention subspaces in parallel to capture dependency patterns of different granularities; the decoder layer gradually generates prediction results based on the information output by the encoder, and the multi-task joint loss function jointly optimizes the yield prediction task through weighted summation to output the yield prediction result.

6. The method for estimating crop yield based on hyperspectral UAV according to claim 5, characterized in that: Also includes: Mapping the production values ​​predicted by the multi-task spatiotemporal Transformer prediction model to geographic space to generate a production distribution map; The root mean square error between the predicted yield and the measured value is calculated, and the final yield estimation result is obtained based on the root mean square error.

7. The method for estimating crop yield based on hyperspectral UAV according to claim 6, characterized in that: The formula for calculating the root mean square error between the predicted yield and the measured value is: Where y i is the measured value, is the predicted value, and n is the number of samples.

Images