Method for retrieving plant leaf area index and storage medium

Abstract

The application relates to the technical field of hyperspectral remote sensing, in particular to a plant leaf area index inversion method and a storage medium. The plant leaf area index inversion method comprises the following steps: performing fusion processing on spectral features and texture features of unmanned aerial vehicle plant hyperspectral imaging data to obtain fusion image data, each pixel point of the fusion image is a feature vector with the spectral features and the texture features; taking the fusion image data as input of a two-dimensional convolution 2D-CNN leaf blade identification model, so that the output result of the 2D-CNN leaf blade identification model is a binary image comprising leaf blades and a background; and calculating a plant leaf area index based on the output result, thereby accurately inverting the plant leaf area index without damaging the plant and improving accurate judgment on the growth state of the plant.

Description

Technical Field

[0001] This application belongs to the field of hyperspectral remote sensing technology, specifically a method for inverting plant leaf area index and a storage medium. Background Technology

[0002] Leaves are the main organs for photosynthesis, respiration, and transpiration in plants, and leaf area is an important indicator for plant growth and development. Leaf area index (LAI) refers to half of the total leaf area per unit area of ​​land surface. It is often used in precision agriculture for evaluating plant growth, predicting yield, adjusting planting structure, and managing water and fertilizer. The establishment of a hyperspectral inversion model for plant leaf area index is of great significance to agricultural production.

[0003] Leaf area measurement methods are mainly divided into two categories: traditional direct measurement methods and indirect methods. Traditional direct methods are somewhat destructive, requiring manual collection of plant leaves, resulting in low efficiency and the inability to continuously monitor the same plant. Indirect methods use measurement parameters or optical instruments to obtain the leaf area index. Commonly used methods include image-based and remote sensing-based methods. Image-based methods are relatively slow, requiring post-processing of images to calculate parameters and then extrapolate the effective leaf area index. Satellite remote sensing is susceptible to the effects of transit time, cloud cover, fog, and moisture, and has low spatial resolution. Therefore, accurately retrieving the plant leaf area index without damaging the leaves, and thus precisely assessing plant growth status, has become a pressing problem. Summary of the Invention

[0004] The purpose of one or more embodiments in this specification is to provide a method for retrieving plant leaf area index, which can accurately retrieve plant leaf area index without damaging the plant, thereby improving the accuracy of judging plant growth status.

[0005] To solve the above-mentioned technical problems, one or more embodiments of this specification are implemented as follows:

[0006] In a first aspect, a method for inverting plant leaf area index is provided, comprising the following steps: fusing the spectral and texture features of UAV plant hyperspectral imaging data to obtain fused image data, wherein each pixel of the fused image is a feature vector with spectral and texture features; using the fused image data as input to a two-dimensional convolutional 2D-CNN leaf recognition model, so that the output of the 2D-CNN leaf recognition model is a binary image including the leaf and the background; and calculating the plant leaf area index based on the output.

[0007] Secondly, a storage medium is provided for computer-readable storage, the storage medium storing one or more programs, which, when executed by one or more processors, implement the plant leaf area index inversion method as described above.

[0008] As can be seen from the technical solutions provided in one or more embodiments of this specification, the plant leaf area index (LAI) inversion method provided in this invention extracts and fuses the spectral and texture features of UAV plant hyperspectral imaging data as input to a 2D-CNN leaf recognition model. The output of this model includes binary images of the leaves and the background, and finally, the LAI is calculated based on this output. Compared with conventional monitoring data, the LAI inversion method provided in this invention has advantages such as being non-destructive, having high resolution, acquiring more data feature information, being flexible in monitoring locations, enabling large-area continuous dynamic monitoring, and acquiring data in real time. Furthermore, this invention uses an indirect measurement method based on plant images to calculate the LAI, and combines this with labeling the UAV plant hyperspectral imaging data as binary icon data as the modeling standard value, effectively avoiding the problem of damaging the plants. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in one or more embodiments or prior art of this specification, the accompanying drawings used in the description of one or more embodiments or prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a flowchart illustrating a method for retrieving plant leaf area index according to an embodiment of the present invention.

[0011] Figure 2 This is a schematic diagram of a CNN leaf recognition model constructed based on fused image data in the plant leaf area index inversion method provided by an embodiment of the present invention. Detailed Implementation

[0012] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described one or more embodiments are merely some embodiments of this specification, and not all embodiments. All other embodiments obtained by those skilled in the art based on one or more embodiments of this specification without creative effort should fall within the protection scope of this document.

[0013] The plant leaf area index (LAI) inversion method provided in this invention is based on UAV hyperspectral imaging data. Unlike satellite remote sensing, image measurement, and traditional leaf area measurement methods, it offers advantages such as being non-destructive, having high resolution, allowing for flexible monitoring locations, and enabling large-area real-time continuous dynamic monitoring. The plant leaf area index inversion method and its various steps provided in this specification will be described in detail below.

[0014] Example 1

[0015] Reference Figure 1 As shown, the plant leaf area index inversion method provided in this embodiment of the invention includes the following steps: S10: The spectral features and texture features of the UAV plant hyperspectral imaging data are fused to obtain fused image data, and each pixel of the fused image data is a feature vector with spectral features and texture features; S20: The fused image data is used as the input of a two-dimensional convolutional 2D-CNN leaf recognition model to realize that the output result of the 2D-CNN leaf recognition model is a binary image including the leaf and the background; S30: The plant leaf area index is calculated based on the output result.

[0016] The plant leaf area index (LAI) inversion method provided in this invention is based on image processing of UAV hyperspectral imaging data. It enables accurate and real-time inversion monitoring of LAI, allowing for timely and precise assessment of plant growth status. Compared to existing technologies, it offers advantages such as non-destructive operation, high resolution, acquisition of more data feature information, flexible monitoring locations, large-area continuous dynamic monitoring, and real-time data acquisition. UAV hyperspectral remote sensing technology offers advantages such as flexible flight, low cost, high spatial resolution, wide monitoring range, and convenient data acquisition, making it more suitable for agricultural production applications.

[0017] Optionally, in the plant leaf area index inversion method provided in this embodiment of the invention, before S10: fusing the spectral features and texture features of the UAV plant hyperspectral imaging data to obtain fused image data, and each pixel of the fused image data is a feature vector with spectral features and texture features, the method further includes extracting spectral features from the UAV plant hyperspectral imaging data, specifically including: reducing the dimensionality of the UAV plant hyperspectral imaging data and calculating the cumulative contribution rate of the spectral features; when the cumulative contribution rate reaches a set threshold, determining the number of principal components of the spectral features with the highest contribution rates; and reducing the dimensionality of the UAV plant hyperspectral imaging data to the number of principal components.

[0018] Principal component analysis (PCA) was used to reduce the dimensionality of UAV plant hyperspectral imaging data. The cumulative contribution rate of spectral features was calculated. When the cumulative contribution rate reached a set threshold, such as greater than 80%, the top m spectral features by contribution rate were selected as principal components, reducing the UAV plant hyperspectral imaging data from the initial n dimensions to m dimensions, where 0 < m < n. For example, if the original UAV plant hyperspectral data had 300 spectral features, after PCA dimensionality reduction, the top 5 spectral features by contribution rate remained as principal components, and the cumulative contribution rate of these 5 principal components was 98%, thus reducing the dimensionality to 5.

[0019] Optionally, in the plant leaf area index inversion method provided in this embodiment of the invention, before S10: fusing the spectral features and texture features of the UAV plant hyperspectral imaging data to obtain fused image data, and each pixel of the fused image is a feature vector with spectral features and texture features, the method further includes extracting texture features from the UAV plant hyperspectral imaging data. Specifically, this includes: performing grayscale compression on the synthesized color image of the UAV plant hyperspectral imaging data in the 630nm, 520nm, and 460nm bands; converting the grayscale compressed UAV plant hyperspectral imaging data to grayscale and calculating the grayscale co-occurrence matrix; and calculating the mean of the grayscale co-occurrence matrix in multiple directions as texture features.

[0020] The hyperspectral imaging data of plants from UAVs in the 630nm, 520nm, and 460nm bands are synthesized into a color image, which is then converted to grayscale, and the gray-level co-occurrence matrix (GLCM) is calculated. To reduce the computational cost of the GLCM, grayscale compression is performed on the synthesized color image data after grayscale conversion before matrix calculation. Calculating the GLCM involves calculating the probability that a pixel with grayscale level i moves to a pixel with grayscale level j by a fixed positional relationship d = (Δx, Δy), which can be represented by P(i, j), (i, j = 0, 2, ..., L-1), where L represents the grayscale level of the grayscale image after grayscale compression, and the ranges of Δx and Δy are determined by the pixel spacing d and the angle θ. This patent uses energy (ASM), contrast (CON), entropy (ENT), and uniformity (IDM). These are four commonly used statistics of the gray-level co-occurrence matrix, and the formulas are shown below. The mean values ​​of these four commonly used statistics in the four directions of 0°, 45°, 90° and 135° are calculated respectively and used as the final texture features for leaf area index inversion.

[0021] ASM=∑ i ∑ j P 2 (i, j) (1)

[0022] CON = ∑ i ∑ j (ij) 2 P(i,j) (2)

[0023] ENT=-∑ i ∑ j P(i,j)lgP(i,j) (3)

[0024]

[0025] Then, the spectral and texture features of the UAV plant hyperspectral data are fused to obtain fused image data. Each pixel in the fused image data is a feature vector with both spectral and texture features. Here, the spectral and texture features can be represented in vector form, with each pixel having a (m+4)-dimensional feature vector, where m is the number of spectral features and 4 is the number of texture features.

[0026] Optionally, the plant leaf area index inversion method provided in this embodiment of the invention, S20: before using the fused image data as input to the 2D-CNN leaf recognition model and realizing that the output result of the 2D-CNN leaf recognition model is a binary image including the leaf and the background, further includes: constructing a 2D-CNN leaf recognition model based on the fused image data. The spectral features and texture features obtained by processing UAV plant hyperspectral data are fused into fused image data, and the fused image data is input into the 2D-CNN leaf recognition model. Compared with the traditional optical vegetation leaf area index measurement method, this embodiment of the invention not only utilizes the spectral information of the sample but also combines the image information of the sample, obtaining more comprehensive sample feature information, and only using a few specific bands to complete the modeling, thereby calculating the leaf area index and achieving high-precision inversion of the plant leaf area index.

[0027] In the plant leaf area index inversion method provided in this embodiment of the invention, before S10: fusing the spectral and texture features of the UAV plant hyperspectral imaging data to obtain fused image data, where each pixel of the fused image data is a feature vector with spectral and texture features, the method further includes: obtaining a binary image including leaves and background based on the UAV plant hyperspectral imaging data using a plant image indirect measurement method; and using the binary image including leaves and background to annotate the UAV plant hyperspectral imaging data to obtain binary icon annotation data. The UAV plant hyperspectral imaging data is processed into a binary image including leaves and background using a plant image indirect measurement method. The obtained accurate binary image can be used to annotate the leaves and background of the UAV plant hyperspectral imaging data, serving as the standard value for modeling a 2D-CNN leaf recognition model.

[0028] Optionally, in the plant leaf area index inversion method provided in this embodiment of the invention, a binary image including leaves and background is obtained by using the plant image indirect measurement method based on UAV plant hyperspectral imaging data. Specifically, it includes: extracting three band data from the UAV plant hyperspectral imaging data to synthesize a color image; preprocessing the color image; performing morphological processing on the preprocessed color image to initially obtain a binary image; and obtaining the final binary image after correction.

[0029] Optionally, in the plant leaf area index inversion method provided in this embodiment of the invention, before constructing a 2D-CNN leaf recognition model based on fused image data, the method further includes: using binary icon annotation data as the modeling standard value. The 630nm, 520nm, and 460nm bands of the hyperspectral imaging data are extracted as the three channels of RGB to synthesize a color image; the color image is preprocessed, including image filtering and contrast changes; then, the preprocessed color image undergoes morphological processing, threshold segmentation is performed using the OTSU maximum inter-class variance method, and a dilatational erosion algorithm is combined to initially obtain binary images of the leaf and background; the binary image is manually corrected to obtain the final binary image. This accurate binary image is used to annotate the leaves and background of the hyperspectral imaging data to obtain binary icon annotation data as the modeling standard value. The method proposed in this embodiment of the invention uses an indirect measurement method based on plant images to calculate the leaf area index as the standard value, combining plant spectral and texture features for modeling, without damaging the plant leaves, effectively avoiding the problem of plant destruction.

[0030] Optionally, in the plant leaf area index inversion method provided in this embodiment of the invention, a 2D-CNN leaf recognition model is constructed based on fused image data, specifically including: preprocessing the fused image data to make each image in the fused image data cropped into multiple sub-images of a set size, each pixel of the sub-image having a feature vector of spectral features and texture features; and building a 2D-CNN leaf recognition model based on the LeNet network model, with the input layer corresponding to (m+4) image channels.

[0031] Each image in the fused image data is cropped into multiple sub-images, and the sub-images are scaled to a size of 32×32. Each pixel of the sub-image has a (m+4) dimensional feature vector, so the size of each sub-image is (32, 32, m+4).

[0032] See Figure 2As shown, this embodiment of the invention constructs a 2D-CNN leaf recognition model based on the classic LeNet network model by adjusting its parameters. The LeNet network contains basic convolutional layer modules, pooling layer modules, and fully connected layer modules, with a total of 7 layers excluding the input. Since the input sub-image is an image with (m+4) channels, the convolutional kernel size of the C1 layer is 5×5×(m+4), that is, two-dimensional convolution is performed on each channel, and then the (m+4) channels are summed to obtain the two-dimensional convolution output of each pixel. For leaf recognition, the final number of recognition label categories is 2, namely leaf and non-leaf, so the number of nodes in the last fully connected layer is set to 2. This embodiment of the invention proposes to input the fused image data obtained by fusing the spectral feature vector after PCA dimensionality reduction with the texture feature vector extracted by the gray-level co-occurrence matrix method into the 2D-CNN leaf recognition model, which provides richer feature information compared to the traditional vegetation index method.

[0033] Optionally, in the plant leaf area index inversion method provided in this embodiment of the invention, S30: calculating the plant leaf area index based on the output result specifically includes: calculating the plant leaf area index using the finite length averaging method.

[0034]

[0035] Where θ is the solar zenith angle; P(θ) is the gap ratio, which is calculated for each sub-image based on leaf image data and non-leaf image data; G(θ) represents the average projected area of ​​a unit volume of leaf area on a plane perpendicular to the θ direction, which is related to the leaf tilt angle distribution.

[0036] The theoretical basis of the indirect ground-based measurement method of leaf area index is Beer's Law. In this embodiment of the invention, the finite-length averaging method is used to derive the formula for calculating the plant leaf area index:

[0037]

[0038] P(θ) is the gap ratio, which represents the probability that a beam of light passes through the vegetation canopy along the zenith angle θ; P(θ) represents the average projected area of ​​a unit volume of leaf area on a plane perpendicular to the O direction, which is related to the leaf tilt angle distribution, and is generally G(θ) = 0.5; Ω(θ) is the aggregation index, which depends on the spatial distribution of the leaves, Ω(θ) > 1 represents a regular distribution, Ω(θ) = 1 represents a random distribution, and Ω(θ) < 1 represents a clump-like distribution.

[0039]

[0040] The formula for calculating the leaf area index using the finite length averaging method is as follows:

[0041]

[0042] In the formula, P(θ) represents the gap ratio of each sub-image when each image in the fused image data is divided into different sub-images, and the calculation method is as follows:

[0043]

[0044] In summary, based on the output results of the 2D-CNN leaf recognition model, the total number of non-vegetation pixels and the total number of pixels in each sub-image are statistically analyzed to obtain the gap ratio of each sub-image. The solar zenith angle θ and G(θ) = 0.5 measured when capturing the hyperspectral imaging image are input into the calculation formula (7) to obtain the plant leaf area index.

[0045] The above analysis shows that the plant leaf area index (LAI) inversion method provided in this embodiment of the invention uses image technology to extract spectral and texture features from UAV plant hyperspectral imaging data, which are then fused and processed as input to a 2D-CNN leaf recognition model. The output of this model is only a binary image including the leaf and background. Finally, the LAI is calculated based on this output. Compared with conventional methods of obtaining leaf area from monitoring data, the LAI inversion method provided in this embodiment of the invention has advantages such as being non-destructive, having high resolution, acquiring more data feature information, being flexible in monitoring locations, enabling large-area continuous dynamic monitoring, and acquiring data in real time. Furthermore, this embodiment of the invention uses an indirect measurement method based on plant images to calculate the LAI, and by using the binary image labeled with hyperspectral imaging data as a standard value for modeling, it effectively avoids the problem of damaging the plant.

[0046] Example 2

[0047] This invention provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, which, when executed by one or more processors, implement the plant leaf area index inversion method as described above.

[0048] like Figure 1 As shown, the plant leaf area index inversion method provided by this embodiment of the invention includes the following steps: S10: fusion processing of spectral features and texture features of UAV plant hyperspectral imaging data to obtain fused image data, where each pixel of the fused image is a feature vector with spectral and texture features; S20: using the fused image data as input to a 2D-CNN leaf recognition model to achieve an output of a binary image including leaves and background; S30: calculating the plant leaf area index based on the output result.

[0049] The plant leaf area index (LAI) inversion method provided in this invention is based on image processing of UAV hyperspectral imaging data. It enables accurate and real-time monitoring of LAI, allowing for timely and precise assessment of plant growth status. Compared to existing technologies, it offers advantages such as non-destructive operation, high resolution, abundant data feature information, flexible monitoring locations, large-area continuous dynamic monitoring, and real-time data acquisition. UAV hyperspectral remote sensing technology offers advantages such as flexible flight, low cost, high spatial resolution, wide monitoring range, and convenient data acquisition, making it more suitable for agricultural production applications. Furthermore, the LAI inversion method provided in this invention employs an indirect measurement method based on plant images to calculate the LAI, and uses the binary image labeled with UAV hyperspectral imaging data as standard data for modeling, effectively avoiding damage to plants.

[0050] The above analysis shows that the plant leaf area index (LAI) inversion method provided in this embodiment of the invention uses image technology to extract spectral and texture features from UAV plant hyperspectral imaging data, which are then fused and processed as input to a 2D-CNN leaf recognition model. The output of this model is only a binary image including the leaf and background. Finally, the LAI is calculated based on this output. Compared with conventional methods of obtaining leaf area from monitoring data, the LAI inversion method provided in this embodiment of the invention has advantages such as being non-destructive, having high resolution, acquiring more data feature information, being flexible in monitoring locations, enabling large-area continuous dynamic monitoring, and acquiring data in real time. Furthermore, this embodiment of the invention uses an indirect measurement method based on plant images to calculate the LAI, and by using the binary image labeled with hyperspectral imaging data as a standard value for modeling, it effectively avoids the problem of damaging the plant.

[0051] In summary, the above description is merely a preferred embodiment of this specification and is not intended to limit the scope of protection of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of protection of this specification.

[0052] The systems, apparatuses, modules, or units described in one or more of the above embodiments may be implemented by a computer chip or entity, or by a product having a certain function. A typical implementation device is a computer. Specifically, a computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0053] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0054] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

Claims

1. A method for inverting plant leaf area index, characterized in that, Includes the following steps: The spectral and texture features of UAV plant hyperspectral imaging data are fused to obtain fused image data, where each pixel of the fused image data is a feature vector with spectral and texture features. The fused image data is used as input to a 2D-CNN leaf recognition model, so that the output of the 2D-CNN leaf recognition model is a binary image including the leaf and the background. Calculate the plant leaf area index based on the output results; The method further includes fusing the spectral and texture features of the UAV plant hyperspectral imaging data to obtain fused image data, where each pixel of the fused image is a feature vector with both spectral and texture features. Specifically, this involves extracting the spectral features from the UAV plant hyperspectral imaging data. The dimensionality of the plant hyperspectral imaging data from the UAV is reduced, and the cumulative contribution rate of the spectral features is calculated. When the cumulative contribution rate reaches a set threshold, the number of principal components of the spectral features with the highest contribution rates is determined. The dimensionality of the UAV plant hyperspectral imaging data is reduced to the number of principal components; The method further includes fusing the spectral and texture features of the UAV plant hyperspectral imaging data to obtain fused image data, where each pixel of the fused image is a feature vector with both spectral and texture features. Specifically, this involves extracting texture features from the UAV plant hyperspectral imaging data. The plant hyperspectral imaging data from the UAV is compressed to grayscale. The composite color image of the UAV plant hyperspectral imaging data in the 630nm, 520nm, and 460nm bands after grayscale compression is converted to grayscale, and the grayscale co-occurrence matrix is ​​calculated. The mean of the gray-level co-occurrence matrix in multiple directions is calculated as the texture feature.

2. The plant leaf area index inversion method according to claim 1, characterized in that, Before using the fused image data as input to a 2D-CNN leaf recognition model, and before the output of the 2D-CNN leaf recognition model is leaf image data and non-leaf image data, the method further includes: A 2D-CNN leaf recognition model is constructed based on the fused image data.

3. The plant leaf area index inversion method according to claim 2, characterized in that, Before constructing a 2D-CNN leaf recognition model based on the fused image data, the method includes: The binary image, including leaves and background, is obtained by using the plant image indirect measurement method based on the plant hyperspectral imaging data of the UAV. Binary icon annotation data was obtained by annotating the UAV plant hyperspectral imaging data using binary images including leaves and background. The binary icon annotation data is used as the modeling standard value for the 2D-CNN leaf recognition model.

4. The plant leaf area index inversion method according to claim 3, characterized in that, The binary image, including leaves and background, is obtained using the plant image indirect measurement method based on the UAV plant hyperspectral imaging data. Specifically, it includes: A color image is synthesized by extracting three bands of the plant hyperspectral imaging data from the UAV. The color image is preprocessed; The preprocessed color image is subjected to morphological processing to initially obtain the binary image; The final binary image is obtained after correction.

5. The plant leaf area index inversion method according to claim 4, characterized in that, A 2D-CNN leaf recognition model is constructed based on the fused image data, specifically including: The fused image data is preprocessed to change its size, so that each image in the fused image data is cropped into multiple sub-images of a set size, and each pixel of the sub-image is a feature vector with spectral and texture features. The 2D-CNN leaf recognition model is built based on the LeNet network model, and the input layer has an image channel with the same number of feature vectors as each pixel.

6. The plant leaf area index inversion method according to claim 5, characterized in that, The plant leaf area index is calculated based on the output results, specifically including: The leaf area index of the plant was calculated using the finite length averaging method: in, The solar zenith angle; The gap ratio is calculated for each sub-image based on the blade image data and the non-blade image data. The leaf area per unit volume is perpendicular to The average projected area on the directional plane is related to the leaf tilt angle distribution.

7. A storage medium, characterized in that, For computer-readable storage, the storage medium stores one or more programs that, when executed by one or more processors, implement the plant leaf area index inversion method as described in any one of claims 1 to 6.

Images