Citrus leaf nitrogen content estimation method based on fusion of unmanned aerial vehicle and satellite image
Through the fusion of drones and satellite images, combined with convolutional neural networks and random forest models, the time-consuming and labor-intensive monitoring of nitrogen content in traditional citrus leaves is solved, and high-precision nitrogen content estimation is achieved, meeting the real-time needs of orchard management. Improved economic benefits.
Patent Information
- Application Number
- CN202410824666.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-25
- Publication Date
- 2024-11-08
AI Technical Summary
The traditional method of nitrogen content monitoring of citrus leaves is time-consuming and labor-intensive, and it is difficult to achieve rapid detection of large-scale fruit trees, and cannot meet the needs of real-time fertilization management. In addition, the use of single remote sensing data will lose image details, affecting the accurate estimation of nitrogen content.
The fusion of drones and satellite images is adopted, and the vegetation index is constructed through convolutional neural networks, and the nitrogen content is estimated using a random forest regression model, combined with the Pearson correlation coefficient screening sensitivity index to improve the estimation accuracy.
A more accurate estimation of nitrogen content of citrus fruit trees is achieved, the accuracy and accuracy of the estimation model is improved, manpower, time and cost consumption is reduced, and real-time monitoring of fruit tree growth status and scientific fertilization planning support is provided.
Smart Images

Figure CN118918496A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method for estimating nitrogen content in citrus leaves by fusing unmanned aerial vehicles with satellite images, and belongs to the technical field of estimating physical and chemical parameters of agricultural crops. Background Art
[0002] Nitrogen is a key element for citrus growth, and has a significant impact on the growth, development, yield and quality of citrus. Therefore, it is crucial to diagnose the nitrogen content in citrus leaves. Traditional nitrogen content monitoring methods are mainly through laboratory testing, which requires field sample data collection and then obtaining sample nitrogen content information in the laboratory using the Kjeldahl method. Traditional chemical analysis methods for obtaining nitrogen content are time-consuming and laborious, and it is difficult to achieve rapid detection of large-scale fruit trees, which cannot meet the real-time needs of fertilization management and will also cause damage to fruit trees.
[0003] Remote sensing technology can monitor the nutritional elements of crops in real time, effectively making up for the shortcomings of traditional methods and becoming an important part of precision agriculture. However, in the current task of estimating the nitrogen content of crops based on remote sensing technology, usually only a single remote sensing data is used, or different remote sensing data are used to extract features separately and then used in coordination. This method is to use the original multi-source data after abstraction separately, which will lose some of the details of the original image, which is not conducive to the subsequent accurate estimation of nitrogen content. Summary of the invention
[0004] The technical problem to be solved by the present invention is to provide a method for estimating the nitrogen content of citrus leaves by fusing UAV and satellite images, fusing UAV images with satellite images to obtain images with both high spatial resolution and spectral information, and achieving a more accurate estimation of the nutritional elements of citrus trees.
[0005] The present invention adopts the following technical solutions to solve the above technical problems:
[0006] A method for estimating nitrogen content in citrus leaves by fusion of UAV and satellite images comprises the following steps:
[0007] Step 1, obtaining drone images of the entire citrus orchard area at a stage where citrus leaves are mature and physical and chemical parameters are stable, preprocessing the acquired drone images, and constructing a full-color image based on the preprocessed drone images;
[0008] Step 2, obtaining satellite images of the same period as the drone images, and preprocessing them, and using the preprocessed satellite images as multispectral images;
[0009] Step 3, using a panchromatic sharpening method based on a convolutional neural network to fuse the panchromatic image and the multispectral image into one image, and constructing a vegetation index based on the fused image;
[0010] Step 4, selecting several sample points in the entire citrus orchard area, measuring the nitrogen content of the citrus leaves corresponding to each sample point, calculating the Pearson correlation coefficient between each vegetation index constructed in step 3 and the nitrogen content, and screening out the vegetation index corresponding to when the Pearson correlation coefficient exceeds a preset value as the vegetation index sensitive to the nitrogen content of the citrus leaves;
[0011] Step 5, preparing a data set, wherein each sample point in the data set includes a vegetation index that is sensitive to the nitrogen content of citrus leaves, and the sample point label is the corresponding nitrogen content, dividing the data set into a training set and a test set, using the training set to train a random forest regression model, obtaining a citrus leaf nitrogen content estimation model with the vegetation index that is sensitive to the nitrogen content of citrus leaves as input and the nitrogen content of citrus leaves as output, and evaluating the accuracy of the model through the test set;
[0012] Step 6, using the citrus leaf nitrogen content estimation model obtained in step 5 to estimate the leaf nitrogen content of the entire citrus orchard area, and draw a nitrogen content distribution map based on the estimation results.
[0013] As a preferred solution of the present invention, in step 1, the acquired drone image is preprocessed, including image stitching, geometric correction and spectral reflectance calculation.
[0014] As a preferred solution of the present invention, in step 1, a panchromatic image is constructed based on the preprocessed drone image, specifically: for the preprocessed drone image, the average value of the blue, green and red bands is used to obtain the image as a panchromatic image.
[0015] As a preferred solution of the present invention, in step 2, preprocessing the satellite image includes radiation calibration, atmospheric correction and reflectivity calculation.
[0016] As a preferred solution of the present invention, in step 3, the loss function of the panchromatic sharpening method based on the convolutional neural network is The setup is as follows:
[0017] Loss Function Loss of spectral information and spatial information loss The normalized sum is obtained, where the spectral information loss It is a fusion image output by the panchromatic sharpening method based on convolutional neural network The projection in the low-resolution area is calculated and obtained with the multispectral image Ms; spatial information loss Fusion image The sum of the individual space losses is composed of:
[0018]
[0019] 1) Spectral information loss
[0020]
[0021] In the formula, Represents fused image The low-resolution projection of ; T(·) represents the composite function of the downsampling operation, including the low-pass filter operator and decimation; ‖·‖ 1 represents the L1 norm;
[0022] 2) Spatial information loss
[0023] Spatial information loss is caused by spatial information loss at the structural level. and pixel-level spatial information loss composition;
[0024] Let X be a rectangle with a side length of σ at the spatial position (m,n) in the full-color image P, and Y be the fused image The corresponding rectangle in the i-th band has a local spatial loss l σ (m,n,i) is:
[0025] l σ (m,n,i)=1-ρ σ (m,n,i)
[0026] In the formula, ρ σ (m,n,i) is the three-dimensional spatial correlation coefficient of the rectangle, and the spatial information loss at the structural level is the average value of all local space losses;
[0027] The linear regression relationship between the panchromatic image P after downsampling and the multispectral image Ms is expressed as:
[0028]
[0029] Where p represents the image obtained after downsampling the panchromatic image P, Ms i represents the i-th band of the multispectral image Ms, α i is the regression coefficient corresponding to the i-th band, c is random noise, and I represents the number of bands;
[0030] Calculate α using least squares estimation i and c, and applied to the fused image On, get the image
[0031]
[0032] In the formula, Represents fused image The i-th band of
[0033] By minimizing the image The mean square error of the panchromatic image P is used to obtain the spatial information loss at the pixel level.
[0034]
[0035] Will and Normalize them to between 0 and 1 and then add them together to get the spatial information loss
[0036] As a preferred embodiment of the present invention, in step 4, the nitrogen content of the citrus leaves at each sample point is measured by the Kjeldahl method.
[0037] A computer device comprises a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the computer program, the steps of the method for estimating nitrogen content in citrus leaves by fusing unmanned aerial vehicle and satellite images are implemented.
[0038] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method for estimating nitrogen content in citrus leaves by fusing drone and satellite images.
[0039] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:
[0040] 1. The present invention integrates UAV images with satellite images to overcome the limitations of a single data type. The fused images have both the high spatial resolution information of UAV images and the multispectral information of satellite images, which can effectively improve the precision of the nitrogen content estimation model and the accuracy of the estimation, provide reliable data support for orchard management, help managers understand the growth status of fruit trees in real time, formulate scientific fertilization plans, and improve economic benefits.
[0041] 2. The data used in the present invention are drone data and satellite data, which can realize large-scale monitoring of orchards and reduce the manpower, time and cost consumption of traditional chemical detection methods in detecting nutrient elements in citrus leaves. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 It is a flow chart of a method for estimating nitrogen content in citrus leaves by integrating drone and satellite images of the present invention;
[0043] Figure 2 It is the network structure diagram of the PNN model. DETAILED DESCRIPTION
[0044] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.
[0045] like Figure 1 As shown, the present invention proposes a method for estimating nitrogen content in citrus leaves by integrating drone and satellite images, and the specific steps are as follows:
[0046] Step 1: When the leaves of the fruit trees are mature and the physical and chemical parameters are stable, drone images of the entire citrus orchard are collected, and preprocessing such as image stitching, geometric correction, and calculation of spectral reflectance is performed;
[0047] Step 2: construct a panchromatic image based on the drone image, and use the image obtained by the average of the blue, green, and red bands as the panchromatic image (PAN) with full color sharpening;
[0048] Step 3: Select Sentinel-2A images from the same period as the UAV flight mission, perform radiometric calibration, atmospheric correction, and reflectivity calculation, and use the preprocessed satellite images as multispectral (Ms) images in pan-sharpening.
[0049] Step 4: Use the CNN-based Pansharpening (PNN) method to fuse the PAN and Ms images into one image.
[0050] The loss function of the PNN method is set as follows:
[0051] Defining the loss function The loss is obtained by normalizing and adding the loss of spatial information and spectral information preservation ability, where the spectral information loss The output image of the PNN network is The projection of the low-resolution area is compared with the original Ms image to calculate the loss of spatial information. Depend on The loss function is the sum of the individual spatial losses of the 11 bands (excluding the two bands that are not related to vegetation growth). In general, the loss function can be expressed as:
[0052]
[0053] (1) Spectral constraints
[0054] Spectral information loss The calculation formula is as follows:
[0055]
[0056] Output to the network Low-resolution projection Calculated with the original Ms image, where the T(·) operator represents the composite function of the downsampling operation, including the low-pass filter operator and the decimation step, ‖·‖ 1 represents the L1 norm.
[0057] (2) Spatial constraints
[0058] The present invention defines the spatial loss function as the original full-color image P and the output image The correlation coefficient between the spectral bands of the image indicates the degree to which one image can be linearly predicted from another image. If X and Y are single-band images of the same size, and σ XY are their variance and covariance respectively, then the correlation coefficient between the two images can be expressed as:
[0059]
[0060] |ρ XY | = 1 means completely predictable, while ρ XY The closer it is to 0, the less relevant it is.
[0061] A three-dimensional spatial correlation is used instead of a single coefficient to measure spatial consistency. Let X be a rectangle with a side length of σ at the spatial position (m, n) in image P, and Y be the fused image The corresponding rectangle in the i-th band in , the three-dimensional spatial correlation coefficient of the rectangle can be expressed as:
[0062]
[0063] The local space loss of this rectangle is:
[0064] l σ (m,n,i)=1-ρ σ (m,n,i)
[0065] Global space loss is the average of all local losses. The spatial structural similarity of the local area is mainly considered, and the present invention further proposes to consider the spatial consistency from the pixel level.
[0066] According to the panchromatic sharpening algorithm based on the degradation model, it can be concluded that in an ideal high-resolution multispectral image, the linear combination of each band is approximately equal to the grayscale value of the high-resolution panchromatic image.
[0067]
[0068] Among them, M i represents an ideal high-resolution image M 0 The i-th band, α i is the regression coefficient corresponding to each band, and c is the random noise.
[0069] From this formula, we can see that the high-resolution image M 0 The linear combination of each band should be consistent with the panchromatic image P. The study obtains the regression coefficient by calculating it at the original low-resolution scale and applies it to the fine-scale data. s The linear regression relationship between images is expressed as:
[0070]
[0071] Where p is obtained by downsampling the original P image, denoted as T(P), Ms i Indicates the i-th band of the original Ms image. The linear regression model in the present invention uses the least squares estimation to calculate, and the regression coefficient α is obtained i Apply to network output Image on image
[0072]
[0073] The images obtained Compared with the full-color image P, by minimizing The mean square error between the original panchromatic image P and the original panchromatic image P is used to constrain the spatial information loss at the pixel level, denoted as:
[0074]
[0075] The two parts of spatial loss are normalized to between 0 and 1 respectively, and then added together to obtain the final spatial information loss.
[0076] The network structure of the PNN model is as follows Figure 2 As shown, the PNN network uses the original PAN-Ms pairs as input, at their original resolution, without any degradation and thus without losing information.
[0077] Step 5: Based on the fused image Construct vegetation indices and screen various vegetation indices sensitive to leaf nitrogen content based on Pearson correlation coefficients;
[0078] Step 6, using a random forest (RF) regression model to estimate nitrogen content, using the screened vegetation index as input and the nitrogen content of citrus leaves as output for model training, using a systematic sampling method for the experimental data set, dividing it into a training set and a test set in a ratio of 3:1 for model training;
[0079] Step 7, evaluate the accuracy of the model, which can be used to estimate the nitrogen content of citrus leaves in the orchard area, and draw a nitrogen content distribution map based on the estimation results.
[0080] Based on the same inventive concept, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the aforementioned method for estimating nitrogen content in citrus leaves by fusing drone and satellite images are implemented.
[0081] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the aforementioned method for estimating the nitrogen content of citrus leaves by fusing drone and satellite images.
[0082] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0083] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0084] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0085] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0086] The above embodiments are only for illustrating the technical idea of the present invention, and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the present invention.
Claims
1. A method for estimating nitrogen content in citrus leaves by integrating drone and satellite images, characterized in that: The steps include: Step 1, obtaining drone images of the entire citrus orchard area at a stage where citrus leaves are mature and physical and chemical parameters are stable, preprocessing the acquired drone images, and constructing a full-color image based on the preprocessed drone images; Step 2, obtaining satellite images of the same period as the drone images, and preprocessing them, and using the preprocessed satellite images as multispectral images; Step 3, using a panchromatic sharpening method based on a convolutional neural network to fuse the panchromatic image and the multispectral image into one image, and constructing a vegetation index based on the fused image; Step 4, selecting several sample points in the entire citrus orchard area, measuring the nitrogen content of the citrus leaves corresponding to each sample point, calculating the Pearson correlation coefficient between each vegetation index constructed in step 3 and the nitrogen content, and screening out the vegetation index corresponding to when the Pearson correlation coefficient exceeds a preset value as the vegetation index sensitive to the nitrogen content of the citrus leaves; Step 5, preparing a data set, wherein each sample point in the data set includes a vegetation index that is sensitive to the nitrogen content of citrus leaves, and the sample point label is the corresponding nitrogen content, dividing the data set into a training set and a test set, using the training set to train a random forest regression model, obtaining a citrus leaf nitrogen content estimation model with the vegetation index that is sensitive to the nitrogen content of citrus leaves as input and the nitrogen content of citrus leaves as output, and evaluating the accuracy of the model through the test set; Step 6, using the citrus leaf nitrogen content estimation model obtained in step 5 to estimate the leaf nitrogen content of the entire citrus orchard area, and draw a nitrogen content distribution map based on the estimation results.
2. The method for estimating nitrogen content in citrus leaves by integrating drone and satellite images according to claim 1, characterized in that: In the step 1, the acquired drone image is preprocessed, including image stitching, geometric correction and spectral reflectance calculation.
3. The method for estimating nitrogen content in citrus leaves by integrating drone and satellite images according to claim 1, characterized in that: In the step 1, a panchromatic image is constructed based on the preprocessed UAV image, specifically: for the preprocessed UAV image, the average value of the blue, green and red bands is used to obtain the image as a panchromatic image.
4. The method for estimating nitrogen content in citrus leaves by integrating drone and satellite images according to claim 1, characterized in that: In step 2, the satellite image is preprocessed including radiation calibration, atmospheric correction and reflectivity calculation.
5. The method for estimating nitrogen content in citrus leaves by integrating drone and satellite images according to claim 1, characterized in that: In step 3, the loss function of the panchromatic sharpening method based on the convolutional neural network is The settings are as follows: Loss Function Loss of spectral information and spatial information loss The normalized sum is obtained, where the spectral information loss It is a fusion image output by the panchromatic sharpening method based on convolutional neural network The projection in the low-resolution area is calculated and obtained with the multispectral image Ms; the spatial information loss L B Fusion image The sum of the individual spatial losses of each band is composed of: 1) Spectral information loss In the formula, Represents fused image The low-resolution projection of ; T(·) represents the composite function of the downsampling operation, including the low-pass filter operator and decimation; ‖·‖1 represents the L1 norm; 2) Spatial information loss Spatial information loss is caused by spatial information loss at the structural level. and pixel-level spatial information loss composition; Let X be a rectangle with a side length of σ at the spatial position (m,n) in the full-color image P, and Y be the fused image The corresponding rectangle in the i-th band has a local spatial loss l σ (m,n,i) is: l σ (m,n,i)=1-ρ σ (m,n,i) In the formula, ρ σ (m,n,i) is the three-dimensional spatial correlation coefficient of the rectangle, and the spatial information loss at the structural level is the average value of all local space losses; The linear regression relationship between the panchromatic image P after downsampling and the multispectral image Ms is expressed as: Where p represents the image obtained after downsampling the panchromatic image P, Ms i represents the i-th band of the multispectral image Ms, α i is the regression coefficient corresponding to the i-th band, c is random noise, and I represents the number of bands; Calculate α using least squares estimation i and c, and applied to the fused image On, get the image In the formula, Represents fused image The i-th band of By minimizing the image The mean square error of the panchromatic image P is used to obtain the spatial information loss at the pixel level. Will and Normalize them to between 0 and 1 and then add them together to get the spatial information loss 6. The method for estimating nitrogen content in citrus leaves by integrating drone and satellite images according to claim 1, characterized in that: In step 4, the nitrogen content of the citrus leaves at each sample point is measured by the Kjeldahl method.
7. A computer device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that: When the processor executes the computer program, the steps of the method for estimating nitrogen content in citrus leaves by fusing drone and satellite images as described in any one of claims 1 to 6 are implemented.
8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method for estimating nitrogen content in citrus leaves by fusing drone and satellite images as described in any one of claims 1 to 6 are implemented.
