A method, apparatus and system for calculating leaf area index
By using a dual-input semantic segmentation model, combined with a Transformer segmentation network and an edge detail enhancement branch, the problems of illumination and branch interference in the processing of outdoor canopy images using the hemispherical photograph method were solved, and stable leaf area index calculation was achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NANJING FORESTRY UNIV
- Filing Date
- 2026-03-20
- Publication Date
- 2026-07-17
Smart Images
Figure CN122416239A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of agricultural / forestry ecological monitoring and computer vision technology, and specifically relates to a method, device and system for calculating leaf area index. Background Technology
[0002] Leaf area index (LAI) is an important parameter characterizing vegetation canopy structure and is widely used in crop growth monitoring, evapotranspiration and photosynthesis models, and forest ecological assessment. Current methods for obtaining LAI mainly include destructive sampling, optical instruments (such as LAI analyzers), and canopy structure inversion methods based on hemispherical photographs. Among these, the hemispherical photograph method uses a fisheye lens to acquire the projection information of the canopy onto the sky, and then calculates the porosity (gap fraction) within different zenith angle ranges to invert LAI. This method has advantages such as low cost, portability, and suitability for field measurements.
[0003] However, the key to the hemispherical photographic method lies in the accurate differentiation of "sky gaps / leaf occlusion". Outdoor canopy images are often affected by factors such as lighting variations, overexposure, shadows, branch interference, and fragmented leaf boundaries. Traditional thresholding or manual processing is prone to misclassification, causing statistical bias in porosity and ultimately leading to errors in LAI calculation. In recent years, deep learning semantic segmentation models can automatically learn features, but single CNN or single Transformer structures still have shortcomings in canopy scenes: CNNs are susceptible to scale changes and lack sufficient understanding of the global structure; Transformers have global modeling capabilities but easily lose local texture and edge details, resulting in unstable segmentation of leaf edges, small gaps, and locally occluded areas. Summary of the Invention
[0004] Objective: In order to overcome the shortcomings of the existing technology, the present invention provides a method, device and system for calculating leaf area index.
[0005] Technical solution: To solve the above technical problems, the technical solution adopted by the present invention is as follows: Firstly, a method for calculating leaf area index is provided, including: Obtain raw hemispherical fisheye canopy image data and perform preprocessing; The original hemispherical fisheye canopy photo and the preprocessed hemispherical fisheye canopy photo are input into the trained dual-input semantic segmentation model, and the output is two types of segmentation prediction masks. Calculate the background porosity based on the segmentation prediction mask; The leaf area index is calculated based on the background porosity. The dual-input semantic segmentation model is based on the Transformer segmentation network and has a dual-input structure, including a main input and an auxiliary input. The main input is used to input the original hemispherical fisheye canopy photo to obtain global semantic features, and the auxiliary input is used to input the preprocessed hemispherical fisheye canopy photo to obtain leaf enhancement features.
[0006] The computational method of this invention improves upon the existing Transformer segmentation network by introducing a dual-input structure: the main input is the original RGB image to obtain global semantic features, and the auxiliary input is the leaf enhancement features of the preprocessed image with "only green leaves retained and the rest blacked out". This solves the problem that existing methods easily missegment tree trunks / branches into leaves under complex lighting and branch interference, leading to porosity statistical distortion and thus causing LAI estimation bias.
[0007] In some embodiments, the preprocessing includes: performing color gamut processing on the original hemispherical fisheye canopy image, extracting green leaf pixels in RGB or HSV space, and setting the remaining pixels to black (value 0) to obtain a green-enhanced image that only highlights the green leaves.
[0008] In some embodiments, training the dual-input semantic segmentation model includes: The acquired hemispherical fish-eye canopy photographs were annotated; the annotations included leaf occlusion and background pores. The annotation results are converted into a mask image, and different pixel values are used to represent two classes (e.g., background pores = 0, blade occlusion = 1), which serve as network supervision signals.
[0009] In some embodiments, calculating the background porosity based on the segmentation prediction mask obtained from the output includes: In a hemispherical fisheye canopy image, determine the center and radius of the effective circle and map the pixels to the zenith angle; A ring-shaped region with a zenith angle between 55° and 60° was selected. The total number of background porosity pixels and the total number of blade-occluded pixels within this region were counted, and the background porosity was calculated using the following formula: ; Where P(θ) is the background porosity, θ is the observation angle, and N s N represents the total number of background pore pixels. v This represents the total number of pixels obscured by the leaf.
[0010] In some embodiments, the leaf area index is calculated using the following formula: ; Where LAI is the leaf area index, and G(θ) is the leaf projection function (leaf tilt angle distribution function), representing the average projection ratio of a unit leaf area at the observation angle θ; since G(θ)≈0.5 when θ≈57.5º and is not sensitive to the leaf tilt angle distribution, we take the 55°~60° ring zone to approximate θ=57.5º, and obtain the single-angle method calculation formula: This allows LAI to be directly output from the segmentation prediction mask.
[0011] In some embodiments, the dual-input semantic model further comprises a dual-branch structure, including an edge detail enhancement branch and a spatial context information extraction branch; The edge detail enhancement branch is used to enhance the edge details of the global semantic features extracted from the main input, and is represented as follows: ; ; Among them, E(F) i ) is a feature F i Edge detail enhancement mapping, F i For stage input features, Conv 1×1 Conv 3×3 and Conv 5×5 These are the convolution operations corresponding to the kernel size. Enhance features for edge details, The feature fusion operation is represented; edge detail enhancement addresses the problem that leaf edges and small pore boundaries in images are fragmented and easily smoothed. Multi-scale convolution is used to extract edge and texture details, thereby improving the separation ability of leaves and background pores.
[0012] The spatial context information extraction branch is used to enhance the spatial context information of the auxiliary input blade enhancement features, as shown below: ; in, To enhance spatial context information, SA represents self-attention operation and LN represents normalization. Spatial context information extraction improves the understanding of large-scale canopy structure and occlusion relationships by efficiently modeling long-range dependencies through self-attention, reducing the probability of interference such as tree trunks / branches being misjudged as leaves.
[0013] Each stage of the backbone network of the dual-input semantic segmentation model includes a multi-scale attention fusion module, which is used to fuse edge detail enhancement features and spatial context information enhancement features obtained after edge detail enhancement and spatial context information extraction, as shown below: ; ; Among them, Gi Here, Conv is the gated attention map, and Conv is the convolutional mapping. σ represents element-wise multiplication, σ represents the Sigmoid function, and AFE(F) represents element-wise multiplication. i ) represents the output after attention feature enhancement.
[0014] By improving the feature extraction and multi-scale fusion mechanism, the ability to identify details such as leaf edges and small pores is enhanced, and stable LAI calculation is achieved by combining the single-angle method of 55º~60º annular porosity statistics.
[0015] Secondly, a leaf area index calculation device is provided, comprising: Acquisition module: Used to acquire raw hemispherical fish eye canopy image data and perform preprocessing; Processing module: This module takes the original hemispherical fisheye canopy image and the preprocessed hemispherical fisheye canopy image as inputs into the trained dual-input semantic segmentation model and outputs two-class segmentation prediction masks. Calculation module: Calculates background porosity based on the segmentation prediction mask; Output module: Calculates the leaf area index based on the background porosity; The dual-input semantic segmentation model is based on the Transformer segmentation network and has a dual-input structure, including a main input and an auxiliary input. The main input is used to input the original hemispherical fisheye canopy photo to obtain global semantic features, and the auxiliary input is used to input the preprocessed hemispherical fisheye canopy photo to obtain leaf enhancement features.
[0016] Thirdly, a computer-readable storage medium is provided having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the leaf area index calculation method described in any of the first aspects.
[0017] Fourthly, a computer device / equipment / system is provided, comprising: Memory, used to store computer programs / instructions; A processor for executing the computer program / instructions to implement the steps of the leaf area index calculation method described in any one of the first aspects.
[0018] Beneficial effects: The leaf area index calculation method, device, and system provided by this invention have the following advantages: This invention introduces a dual-input dual-branch structure to enhance leaf detail features while acquiring global semantic features, avoiding the shortcomings of a single CNN or Transformer structure in canopy scenes. It has strong anti-interference ability against illumination and background, improves the accuracy of porosity statistics, and thus improves the accuracy of leaf area index calculation. Attached Figure Description
[0019] Figure 1 This is a schematic diagram of the overall architecture of the dual-input semantic model according to an embodiment of the present invention; Figure 2 This is a comparison chart of the segmentation performance of the dual-input semantic model in this embodiment of the invention and existing semantic segmentation models. Detailed Implementation
[0020] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.
[0021] The present invention will be further described below with reference to specific embodiments.
[0022] Example 1: This example provides a method for calculating leaf area index, including: Obtain raw hemispherical fisheye canopy image data and perform preprocessing; The original hemispherical fisheye canopy photo and the preprocessed hemispherical fisheye canopy photo are input into the trained dual-input semantic segmentation model, and the output is two types of segmentation prediction masks. Calculate the background porosity based on the segmentation prediction mask; The leaf area index is calculated based on the background porosity. The dual-input semantic segmentation model is based on the Transformer segmentation network and has a dual-input structure, including a main input and an auxiliary input. The main input is used to input the original hemispherical fisheye canopy image to obtain global semantic features, and the auxiliary input is used to input the preprocessed hemispherical fisheye canopy image to obtain leaf enhancement features, such as... Figure 1 The diagram shows the overall architecture of the model.
[0023] The preprocessing includes: color gamut processing of the original hemispherical fisheye canopy photo, extracting green leaf pixels in RGB or HSV space, and setting the remaining pixels to black (value 0) to obtain a green-enhanced photo that only highlights the green leaves.
[0024] The training of the dual-input semantic segmentation model includes: The acquired hemispherical fish-eye canopy photographs were annotated; the annotations included leaf occlusion and background pores. The annotation results are converted into a mask image, and different pixel values are used to represent two classes (e.g., background pores = 0, blade occlusion = 1), which serve as network supervision signals.
[0025] The calculation of background porosity based on the segmentation prediction mask obtained from the output includes: In a hemispherical fisheye canopy image, determine the center and radius of the effective circle and map the pixels to the zenith angle; A ring-shaped region with a zenith angle between 55° and 60° was selected. The total number of background porosity pixels and the total number of blade-occluded pixels within this region were counted, and the background porosity was calculated using the following formula: ; Where P(θ) is the background porosity, θ is the observation angle, and N s N represents the total number of background pore pixels. v This represents the total number of pixels obscured by the leaf.
[0026] In some embodiments, the leaf area index is calculated using the following formula: ; Where LAI is the leaf area index, and G(θ) is the leaf projection function (leaf tilt angle distribution function), representing the average projection ratio of a unit leaf area at the observation angle θ; since G(θ)≈0.5 when θ≈57.5º and is not sensitive to the leaf tilt angle distribution, we take the 55°~60° ring zone to approximate θ=57.5º, and obtain the single-angle method calculation formula: This allows LAI to be directly output from the segmentation prediction mask.
[0027] The dual-input semantic model also has a dual-branch structure, including an edge detail enhancement branch and a spatial context information extraction branch; The edge detail enhancement branch is used to enhance the edge details of the global semantic features extracted from the main input, and is represented as follows: ; ; Among them, E(F) i ) is a feature F i Edge detail enhancement mapping, F i For stage input features, Conv 1×1 Conv 3×3 and Conv 5×5 These are the convolution operations corresponding to the kernel size. Enhance features for edge details, The feature fusion operation is represented; edge detail enhancement addresses the problem that leaf edges and small pore boundaries in images are fragmented and easily smoothed. Multi-scale convolution is used to extract edge and texture details, thereby improving the separation ability of leaves and background pores.
[0028] The spatial context information extraction branch is used to enhance the spatial context information of the auxiliary input blade enhancement features, as shown below: ; in, To enhance spatial context information, SA represents self-attention operation and LN represents normalization. Spatial context information extraction improves the understanding of large-scale canopy structure and occlusion relationships by efficiently modeling long-range dependencies through self-attention, reducing the probability of interference such as tree trunks / branches being misjudged as leaves.
[0029] Each stage of the backbone network of the dual-input semantic segmentation model includes a multi-scale attention fusion module, which is used to fuse edge detail enhancement features and spatial context information enhancement features obtained after edge detail enhancement and spatial context information extraction, as shown below: ; ; Among them, G i Here, Conv is the gated attention map, and Conv is the convolutional mapping. σ represents element-wise multiplication, σ represents the Sigmoid function, and AFE(F) represents element-wise multiplication. i ) represents the output after attention feature enhancement.
[0030] By improving the feature extraction and multi-scale fusion mechanism, the ability to identify details such as leaf edges and small pores is enhanced, and stable LAI calculation is achieved by combining the single-angle method of 55º~60º annular porosity statistics.
[0031] To demonstrate the necessity of annular band selection and auxiliary input parameters, hemispherical canopy images were acquired at different shooting times and under different lighting conditions, and the stability of LAI results under different settings was compared: the annular band range was changed (e.g., 50º~55º, 55º~60º, 60º~65º) and LAI was calculated respectively. The error was compared with the reference LAI (manually labeled porosity or instrument measurement value) to verify the stability of the 55º~60º annular band for the single-angle method; the green threshold parameter of the auxiliary input "green leaf enhancement image" was adjusted to maintain the segmentation and LAI error consistency across different time data, thereby verifying the necessity and adjustability of the threshold parameter setting.
[0032] Figure 2 The image visually demonstrates the comparison of segmentation prediction masks for different models on the same hemispherical canopy image. For example... Figure 2 As shown in Figure a, the prediction results of the SegFormer model, from left to right, are the input hemispherical canopy image and the corresponding binary segmentation prediction mask. It can be observed that SegFormer exhibits many scattered misclassification phenomena in the annular region, and is particularly susceptible to interference from non-leaf structures such as tree trunks / branches, resulting in discontinuous noise and erroneous responses in the prediction mask. Figure 2Figures b to d show a comparison of the prediction results of EDAFormer, DeepLabV3+, and the method proposed in this embodiment under the same conditions. The visualization clearly shows that EDAFormer and DeepLabV3+ also suffer from misclassification and boundary breakage issues caused by trunk interference to varying degrees. However, the method proposed in this embodiment can more effectively eliminate the interference response from the trunk / branch, resulting in a cleaner and more continuous prediction mask. It also provides a more complete depiction of the leaf structure and pore details in the annular region, thus exhibiting higher accuracy and stability in the leaf / background segmentation task.
[0033] Compared to the existing models mentioned above, the dual-input semantic model provided in this embodiment improves overall performance by 4.50% to 14.60% in mIoU and 2.70% to 11.40% in F1. Furthermore, in suppressing the key issue of "tree trunks / branches being misclassified as leaves," the present invention shows a more significant improvement in precision, ranging from 2.90% to 11.80%, while improving recall by 2.50% to 11.00%. This indicates that the method in this embodiment can maintain the complete detection of leaf regions while reducing false positives, thereby providing more reliable segmentation results for subsequent porosity statistics and LAI estimation.
[0034] This invention introduces an edge detail enhancement branch, auxiliary input detail compensation, and a multi-scale attention fusion module, and verifies their necessity through ablation comparison. Experimental results show that when the edge detail enhancement branch is removed, mIoU decreases by 6.4% and precision decreases by 6.1%; when the auxiliary input is removed, mIoU decreases by 4.4%; and when the multi-scale attention fusion module is removed, mIoU decreases to 81.6% (a decrease of 5.3%). These results demonstrate that each structure plays a significant role in leaf edge refinement, detail compensation, and adaptive fusion of primary and secondary features, effectively reducing trunk misclassification and boundary adhesion problems.
[0035] Example 2: This example provides a leaf area index calculation device, including: Acquisition module: Used to acquire raw hemispherical fish eye canopy image data and perform preprocessing; Processing module: This module takes the original hemispherical fisheye canopy image and the preprocessed hemispherical fisheye canopy image as inputs into the trained dual-input semantic segmentation model and outputs two-class segmentation prediction masks. Calculation module: Calculates background porosity based on the segmentation prediction mask; Output module: Calculates the leaf area index based on the background porosity; The dual-input semantic segmentation model is based on the Transformer segmentation network and has a dual-input structure, including a main input and an auxiliary input. The main input is used to input the original hemispherical fisheye canopy photo to obtain global semantic features, and the auxiliary input is used to input the preprocessed hemispherical fisheye canopy photo to obtain leaf enhancement features.
[0036] Example 3: This example provides a computer-readable storage medium storing a computer program / instructions thereon. When the computer program / instructions are executed by a processor, they implement the steps of any of the leaf area index calculation methods described in Example 1.
[0037] Example 4: This example provides a computer device / equipment / system, including: Memory, used to store computer programs / instructions; A processor is configured to execute the computer program / instructions to implement the steps of the leaf area index calculation method described in any one of Embodiment 1.
[0038] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0039] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0040] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0041] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0042] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. A method for calculating leaf area index, characterized in that, include: Obtain raw hemispherical fisheye canopy image data and perform preprocessing; The original hemispherical fisheye canopy photo and the preprocessed hemispherical fisheye canopy photo are input into the trained dual-input semantic segmentation model, and the output is two types of segmentation prediction masks. Calculate the background porosity based on the segmentation prediction mask; The leaf area index is calculated based on the background porosity. The dual-input semantic segmentation model is based on the Transformer segmentation network and has a dual-input structure, including a main input and an auxiliary input. The main input is used to input the original hemispherical fisheye canopy photo to obtain global semantic features, and the auxiliary input is used to input the preprocessed hemispherical fisheye canopy photo to obtain leaf enhancement features.
2. The leaf area index calculation method according to claim 1, characterized in that, The preprocessing includes: color gamut processing of the original hemispherical fisheye canopy photo, extracting green leaf pixels in RGB or HSV space, and setting the remaining pixels to black to obtain a green-enhanced photo.
3. The leaf area index calculation method according to claim 1, characterized in that, The training of the dual-input semantic segmentation model includes: The acquired hemispherical fish-eye canopy photographs were annotated; the annotations included leaf occlusion and background pores. The annotation results are converted into a mask image, which serves as a network supervision signal.
4. The leaf area index calculation method according to claim 1, characterized in that, The calculation of background porosity based on the segmentation prediction mask obtained from the output includes: In a hemispherical fisheye canopy image, determine the center and radius of the effective circle and map the pixels to the zenith angle; A ring-shaped region with a zenith angle between 55° and 60° was selected. The total number of background porosity pixels and the total number of blade-occluded pixels within this region were counted, and the background porosity was calculated using the following formula: ； Where P(θ) is the background porosity, θ is the observation angle, and N s N represents the total number of background pore pixels. v This represents the total number of pixels obscured by the leaf.
5. The leaf area index calculation method according to claim 1, characterized in that, The formula for calculating the leaf area index is as follows: ； Where LAI is the leaf area index, P(θ) is the background porosity, θ is the observation angle, and G(θ) is the leaf projection function.
6. The leaf area index calculation method according to claim 1, characterized in that, The dual-input semantic model also has a dual-branch structure, including an edge detail enhancement branch and a spatial context information extraction branch; The edge detail enhancement branch is used to enhance the edge details of the global semantic features extracted from the main input, and is represented as follows: ； ； Among them, E(F) i ) is a feature F i Edge detail enhancement mapping, F i For stage input features, Conv 1×1 Conv 3×3 and Conv 5×5 These are the convolution operations corresponding to the kernel size. Enhance features for edge details, Indicates feature fusion operation; The spatial context information extraction branch is used to enhance the spatial context information of the auxiliary input blade enhancement features, as shown below: ； in, To enhance features with spatial context information, SA represents self-attention operation and LN represents normalization; Each stage of the backbone network of the dual-input semantic segmentation model includes a multi-scale attention fusion module, which is used to fuse edge detail enhancement features and spatial context information enhancement features obtained after edge detail enhancement and spatial context information extraction, as shown below: ； ； Among them, G i Here, Conv is the gated attention map, and Conv is the convolutional mapping. σ represents element-wise multiplication, σ represents the Sigmoid function, and AFE(F) represents element-wise multiplication. i ) represents the output after attention feature enhancement.
7. A leaf area index calculation device, characterized in that, include: Acquisition module: Used to acquire raw hemispherical fish eye canopy image data and perform preprocessing; Processing module: This module takes the original hemispherical fisheye canopy image and the preprocessed hemispherical fisheye canopy image as inputs into the trained dual-input semantic segmentation model and outputs two-class segmentation prediction masks. Calculation module: Calculates background porosity based on the segmentation prediction mask; Output module: Calculates the leaf area index based on the background porosity; The dual-input semantic segmentation model is based on the Transformer segmentation network and has a dual-input structure, including a main input and an auxiliary input. The main input is used to input the original hemispherical fisheye canopy photo to obtain global semantic features, and the auxiliary input is used to input the preprocessed hemispherical fisheye canopy photo to obtain leaf enhancement features.
8. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the leaf area index calculation method according to any one of claims 1 to 6.
9. A computer device / equipment / system, characterized in that, include: Memory, used to store computer programs / instructions; A processor for executing the computer program / instructions to implement the steps of the leaf area index calculation method according to any one of claims 1 to 6.