A method for estimating carbon storage of urban street trees at the individual tree level

Abstract

The present invention relates to the technical field of carbon storage estimation for urban street trees, and specifically discloses a method for estimating the carbon storage of urban street trees at the individual tree level. The method comprises the following steps: A01: establishing a multi-task street tree individual tree recognition network based on street view images; A02: constructing a tree structure solution method for panoramic street view images; and A03: establishing a street tree spatial position solution network. The method can directly obtain tree height, diameter at breast height (DBH), and spatial position information based on individual tree recognition results from street view images, and finally further analyze the carbon storage distribution of urban street trees.

Description

Technical Field

[0001] The present application relates to the technical field of carbon storage estimation of street trees, and specifically discloses a method for estimating carbon storage of urban street trees at the single-tree level. Background Art

[0002] Street trees are important carbon sinks in urban ecosystems. Establishing an inventory of street trees at the individual tree level, recording structural information such as tree height and diameter at breast height, is conducive to understanding the growth and carbon storage status of urban trees at multiple scales, such as individual trees, streets, and urban areas. The traditional method of establishing a city-wide street tree inventory through manual field sampling consumes a lot of manpower, material resources, and time costs. There is an urgent need for a low-cost, automatic, and efficient method for measuring tree structural information. The accumulation of massive amounts of free street view imagery and the successful application of convolutional neural networks (CNNs) in the image field provide a solution for measuring street tree information.

[0003] There are various methods for estimating tree carbon stocks. The allometric equation is one of the most widely used methods for estimating individual tree carbon stocks. This method uses tree height and diameter at breast height (DBH) as parameters to construct a carbon stock fitting equation. Traditional direct measurement of DBH requires significant labor, physical, and time resources. Therefore, indirect estimation methods have been developed based on multi-source datasets. Currently, four main types of datasets are used for indirect estimation of tree carbon stocks: optical remote sensing data, photogrammetry data, synthetic aperture radar (SAR), and LiDAR data. Optical remote sensing and photogrammetry data cannot capture vertical structural information about trees. Due to the high spatial heterogeneity of trees, estimating biomass at the individual tree scale provides more accurate tree carbon stock data and is fundamental to understanding the dynamics of carbon storage. SAR data are difficult to measure at the individual tree level. While LiDAR data can capture three-dimensional structural information at the individual tree level, large-scale measurements are difficult due to their high cost.

[0004] Street view images, as a freely available data resource, can save a lot of manpower and material costs, and can provide a large amount of ground vertical information, and have been effectively applied in many fields. Based on street view images combined with deep learning networks, efficient street tree information extraction can be achieved, such as urban street tree detection, street tree classification and street tree geographic location encoding. The instance segmentation network can achieve multi-task learning, improve the efficiency and performance of the model, and help to carry out refined identification and analysis of trees, but the direct application of the network to the identification of street trees in urban scenes will have the problem of tree occlusion. The present invention combines the multi-task convolutional neural network instance segmentation model with the mapping algorithm of street view images, and realizes the automatic and efficient extraction of street tree structural information (including tree height and diameter at breast height) at the single tree level, effectively solving the problem of tree occlusion, and establishing a street tree position measurement method based on panoramic street view images and depth estimation algorithms. The street tree inventory list based on geographic coordinate encoding is obtained, and the spatial distribution of urban street tree carbon storage can be quickly obtained;

[0005] At present, most tree carbon storage measurements choose to use estimation methods that combine remote sensing images and lidar point cloud data. These methods are costly and cannot achieve large-scale single-tree-level measurements. Although tree information extraction combined with convolutional neural networks has achieved efficient tree detection and segmentation, it is necessary to further extract detailed structural information of trees (tree height and diameter at breast height). At the same time, although street view images have the advantages of easy data acquisition and rich information, due to the problems of fisheye lens and spliced ​​pixel distortion, it is difficult to achieve street tree structural information measurement and cannot be directly applied to street tree carbon storage estimation. Therefore, there is currently a lack of a low-cost, automatic and efficient street tree carbon storage estimation method based on street view images using convolutional neural networks;

[0006] This paper proposes a multi-task convolutional neural network for single-tree identification based on Baidu panoramic images, which realizes low-cost urban street tree count and single-tree structural information measurement, and estimates the carbon storage of street trees. This method effectively solves the problem of tree shading, helps to gain a deeper understanding of the carbon sequestration capacity of urban street trees, and provides new methods and means for urban ecosystem carbon sink management. Summary of the Invention

[0007] The purpose of the present invention is to construct a technical system suitable for estimating the carbon storage of urban street trees based on panoramic street view images combined with a multi-task convolutional neural network method. First, in order to solve the tree occlusion problem and make the network more suitable for urban street tree detection and segmentation, a multi-task convolutional neural network is constructed based on the YOLO v8 network to realize the recognition of street trees in street view images at the level of single trees. Then, in order to calculate the carbon storage of single street trees, a panoramic image tree structure solution method is proposed. This method can directly obtain the height and diameter of single trees from the image based on the single tree recognition results. Finally, in order to further analyze the carbon storage distribution of urban street trees, a panoramic image single tree position measurement method is implemented.

[0008] In order to achieve the above object, the present invention provides the following basic scheme:

[0009] A method for estimating carbon storage of street trees at the individual tree level comprises the following steps:

[0010] A01: Building a multi-task street tree recognition network for street view images;

[0011] The construction steps are as follows:

[0012] a1: Acquisition of panoramic street view data: Generate sampling point data based on OSM road network data and obtain panoramic street view images through Baidu API;

[0013] a2: Panoramic image roadside tree annotation dataset: Use Labelme to annotate the acquired panoramic images and divide them into training and test sets:

[0014] a3: Model Improvement: A multi-dimensional dynamic convolutional attention mechanism was added to the YOLO v8 network backbone model to form a new multi-task network model, namely the street view image multi-task street tree recognition network;

[0015] A02: Construction of a tree structure calculation method for panoramic street view images;

[0016] The construction steps are as follows:

[0017] b1: Tree crown and trunk structure identification: For tree and crown trunk separation identification, the Fstats function is used to estimate the mutation point to separate and identify the crown and trunk of the tree. The row where the mutation point is located is regarded as the row where the bottom baseline of the tree is located.

[0018] b2: Panoramic Cylindrical Projection: This uses the camera's height and orientation, as well as the image size, vertical and horizontal fields of view, to graphically construct the tree structure in the panoramic image.

[0019] b3: Tree structure calculation: In order to measure the distortion of pixels at different positions caused by the fisheye lens and stitching, the vertical (C v,i) and level (C h,i ) correction factor;

[0020] A03: Construction of a network for calculating the spatial position of urban street trees;

[0021] The construction steps are as follows:

[0022] c1: extract tree benchmarks;

[0023] c2: Generate a depth map of the panoramic image and calculate the depth values ​​of the tree benchmarks;

[0024] c3: tree position measurement;

[0025] A04: Draw a carbon storage distribution map of street trees.

[0026] The principles and effects of this basic solution are:

[0027] 1. Compared with the existing technology, the technical solution proposed in this invention is a multi-task convolutional neural network street tree carbon storage estimation method based on Baidu Street View imagery. It can directly identify street trees and measure the structure and position information of individual trees based on free panoramic urban images, greatly improving the efficiency of street tree measurement and investigation. The proposed multi-task convolutional neural network YOLO v8 instance segmentation model combining ODConv and WI oU can effectively solve the problem of tree occlusion, reduce tree omissions, and improve the confidence level of individual tree identification. By comparing with measured data, the street tree structure and position measurement method based on panoramic images of this invention has achieved good results and successfully estimated the carbon storage of street trees in a certain urban area. This method has high application value in low-cost large-area street tree carbon storage estimation and provides a reference for the coordinated management of urban carbon sink resources.

[0028] 1. Further, in step a3 of step A01, the multi-dimensional dynamic convolutional attention mechanism uses a parallel strategy to learn the complementary attention of the convolution kernel in all four dimensions of the kernel space (i.e., spatial size, number of input channels, number of output channels, and number of convolution kernels) to improve the occlusion problem in urban street tree detection. A dynamic non-monotonic focusing loss function (WIoU) is established, which replaces the complete intersection-over-union (CIoU) loss function of the bounding box to reduce the impact of low-quality annotation boxes on the results and improve the model detection effect.

[0029]

[0030] Furthermore, after step a3 in step A01, the established multi-task street tree recognition network for street view images needs to be evaluated. Precision, recall, and average intersection-over-union (IoU) are used to evaluate the network. Higher scores on these metrics indicate better model performance. The details are as follows:

[0031] Each indicator can be calculated based on the four prediction results in the confusion matrix: true positive (TP), false positive (FP), true negative (TN), and false negative (FN). TP indicates that a true positive example is also predicted as a positive example; FP indicates that a true negative example is predicted as a positive example; TN indicates that a true negative example is predicted as a negative example; and FN indicates that a true positive example is predicted as a negative example. Precision focuses on the proportion of predicted positive examples that are actually positive examples, reflecting the accuracy of the model's predictions and more in line with practical needs. The specific formula is as follows:

[0032]

[0033] Recall rate focuses on how many positive examples in the real samples are predicted. The specific formula is as follows:

[0034]

[0035] The mean intersection over union (mIoU) is a standard metric in semantic segmentation tasks. It focuses on the average of the ratio of the intersection of all predicted positive classes and the true positive classes to the union of the two. The specific formula is as follows:

[0036]

[0037] Furthermore, in step b1 of step A02, for each potential change point, the F statistic (Chow test statistic) is calculated. The larger the value of the F statistic, the more significant the parameter difference between the intervals, and thus the more likely there is a mutation point. The calculation formula of the F statistic is as follows:

[0038]

[0039] Furthermore, in b2 of step A02, the tree structure in the panoramic image is constructed based on the following assumptions: X1: The panoramic image is captured in a plane space, and the influence of the undulating terrain is not considered; X2: The pitch angle of the camera is 0 (that is, the angle between the camera lens and the horizontal plane) and remains horizontal; X3: The trees are planted on the shooting ground; X4: The 360° panoramic image is projected from a sphere to a cylinder using spherical cylindrical projection.

[0040] 7. Further, in step b3 of step A02, the correction factors are: when a pixel moves away from the center line, the vertical correction factor increases from 1 to infinity, and the horizontal correction factor decreases from 1 to 0, and correction is performed;

[0041] After mapping the image to the cylinder, the vertical angle (θ i ) is expressed as:

[0042]

[0043] Based on the assumption, the reference pixel size W can be calculated by dividing the pixel distortion from the baseline to the midline by the camera height (VH) r , that is, the actual size of the pixel without distortion;

[0044]

[0045] Where n is the number of pixels from the bottom of the tree to the midline, C v,i is the vertical scaling factor of the i-th pixel, W r is the reference pixel height without distortion;

[0046] Select trees that are close to the camera and evenly distributed on both sides of the center line to calculate W r , calculated W r The value is 0.03m. Finally, the height and diameter of the street trees can be expressed as:

[0047]

[0048] Where k and n are the number of rows of vertical tree pixels at the bottom and top of the tree, respectively;

[0049]

[0050] Where k and j are the vertical row numbers of the tree bottom and trunk top respectively; N i is the number of pixels in row i that are classified as trees, and the curly brackets in the equation indicate the order of indexes from bottom to top. 10 Indicates taking the 10th quantile.

[0051] Furthermore, in c1 in step A03, the bottom of the street tree is regarded as the reference point, and the latitude and longitude coordinates of the street tree are calculated by obtaining the depth value of the reference point on the depth estimation image. The coordinates of the lower left and lower right corners of the street tree bounding box detected by the network are (Xleft, Ybottom) and (Xright, Ybottom), then the tree reference point is (Xleft+Xright) / 2.

[0052] Furthermore, in c2 of step A03, depth estimation estimates depth information through disparity (d). Disparity is the offset of the same object in two images. The MonoDepth monocular depth estimation network can solve the disparity field without the need for ground truth depth. The input image is transformed by predicting the disparity to reconstruct the image on the other side. Given the baseline distance b between the cameras and the camera focal length f, the depth can be recovered from the predicted disparity. The calculation formula is as follows:

[0053]

[0054] Use MonoDepth pre-trained on the Cityscape dataset and fine-tuned using the KITTY visual benchmark dataset to predict Baidu panoramic images and generate disparity maps. Since these pre-trained parameters are set for datasets such as KITTI and Cityscapes, the image size difference between the input image (b1) and the original Cityscapes and KITTY datasets (b0) needs to be corrected during post-processing. The distance between the street trees in the image and the camera position when the image was captured is the pixel value (in meters) of the corrected depth estimate image. The correction formula is as follows:

[0055]

[0056] Furthermore, in c3 of step A03, in order to realize the position measurement of the street trees in the panoramic image, the panoramic image is divided into multiple areas, and the depth and yaw angle of the reference point are calculated respectively according to the area where the street tree reference point is located.

[0057] Furthermore, in step A04, a single-tree allometric growth equation can be established based on tree structural information to obtain tree biomass. Tree biomass consists of aboveground biomass (such as trunk, bark, branches, and leaves) and underground biomass. The carbon storage of trees can be calculated by multiplying the biomass by the carbon content. The carbon storage estimation formula for street trees is as follows:

[0058] W B =0.0104(D 2 H) 0.9994 #(17)

[0059] W L =0.0188(D 2 H) 0.8024 #(18)

[0060] W T =W S +W P +W B +W L #(19)

[0061] W R =0.0197(D 2 H) 0.8963 #(20)

[0062] W=W T +W R #(twenty one)

[0063] C=W×F#(22)

[0064] Where W S represents the trunk biomass, W Prepresents bark biomass, W B represents the branch biomass, W L represents the leaf biomass, W T represents the total aboveground biomass, W R represents the underground biomass, D represents the average diameter at breast height of the plot, H represents the average tree height of the plot, W represents the total biomass of the trees, and W represents the average tree height. T represents the total aboveground biomass, W R represents the underground biomass, C is the carbon storage, and F is the carbon content. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0066] Figure 1 A flowchart of a method for estimating carbon storage at the single-tree level for urban roadside trees using a multi-task convolutional neural network based on Baidu Street View images is shown;

[0067] Figure 2 A diagram illustrating the multi-task network structure in a single-tree-level urban street tree carbon storage estimation method proposed in an embodiment of the present application is shown;

[0068] Figure 3 A schematic diagram showing the comparison of the improved STV-MNet and baseline models in a single-tree-level urban street tree carbon storage estimation method proposed in an embodiment of the present application is shown, where a is the baseline model and b is the STV-MNet;

[0069] Figure 4 A diagram showing the effect of tree mutation point detection in a single-tree-level urban street tree carbon storage estimation method proposed in an embodiment of the present application is shown;

[0070] Figure 5 A cylindrical projection diagram of a method for estimating carbon storage of urban street trees at the single-tree level proposed in an embodiment of the present application is shown;

[0071] Figure 6 A schematic diagram of panoramic image information in a method for estimating carbon storage of urban roadside trees at the single-tree level proposed in an embodiment of the present application is shown;

[0072] Figure 7 A schematic diagram showing depth estimation results in a single-tree-level urban street tree carbon storage estimation method proposed in an embodiment of the present application is shown;

[0073] Figure 8 A schematic diagram of calculating the position of street trees in a panoramic image in a method for estimating the carbon storage of urban street trees at the single-tree level proposed in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0074] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the specific implementation methods, structures, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0075] Implementation example Figure 1 、 Figure 2 、 Figure 3 、 Figure 4 、 Figure 5 、 Figure 6 、 Figure 7 and Figure 8 As shown:

[0076] A method for estimating carbon storage of street trees at the individual tree level comprises the following steps:

[0077] A01: Building a multi-task street tree recognition network for street view images;

[0078] The construction steps are as follows:

[0079] a1: Acquisition of panoramic street view data: Generate sampling point data based on OSM road network data and obtain panoramic street view images through Baidu API;

[0080] a2: Panoramic image roadside tree annotation dataset: Use Labelme, an online annotation tool software, to annotate the acquired panoramic images and divide them into training and test sets.

[0081] When dividing the training set and test set, the training set and test set are divided in a ratio of 8:2 for model training and testing;

[0082] a3: Model Improvement: A multi-dimensional dynamic convolutional attention mechanism was added to the YOLO v8 network backbone model to form a new multi-task network model, namely the street view image multi-task street tree recognition network. The YOLO v8 network backbone model is an existing deep learning network structure.

[0083] Specifically: Based on the YOLO v8 network backbone model, a multi-dimensional dynamic convolutional attention mechanism is added. This mechanism uses a parallel strategy to learn the complementary attention of convolution kernels in all four dimensions of the kernel space (i.e., spatial size, number of input channels, number of output channels, and number of convolution kernels) to improve the occlusion problem in urban street tree detection. Figure 2The network structure diagram is shown in Figure 2. In addition, the bounding box loss function complete intersection over union (CIoU) is replaced by the loss function of the dynamic non-monotonic focusing mechanism (WIoU) to reduce the impact of low-quality annotation boxes on the results and improve the model detection effect. The CIoU and WIoU formulas are as follows:

[0084]

[0085]

[0086] Specifically: After step a3 in step A01, the established street view image multi-task street tree recognition network needs to be evaluated using precision, recall, and average intersection-over-union (IoU) ratio.

[0087] Specifically, each indicator is calculated based on the four prediction results in the confusion matrix: true positive (TP), false positive (FP), true negative (TN), and false negative (FN). TP indicates that a true positive example is also predicted as a positive example; FP indicates that a true negative example is predicted as a positive example; TN indicates that a true negative example is predicted as a negative example; and FN indicates that a true positive example is predicted as a negative example. Precision focuses on the proportion of predicted positive examples that are actually positive examples, reflecting the accuracy of the model's predictions and more in line with practical needs. The specific formula is as follows:

[0088]

[0089] Recall rate focuses on how many positive examples in the real samples are predicted. The specific formula is as follows:

[0090]

[0091] The mean intersection over union (mIoU) is a standard metric in semantic segmentation tasks. It focuses on the average of the ratio of the intersection of all predicted positive classes and the true positive classes to the union of the two. The specific formula is as follows:

[0092]

[0093] Specifically: Ablation experiments are used to verify the effect of the new model:

[0094] The initial network model YOLO v8n-seg, YOLO v8n-seg-ODConv with multi-dimensional dynamic convolution attention, YOLO v8n-seg-WIoU with improved Loss and the model proposed in this paper are trained with the same training hyperparameters to compare the model performance. Table 1 and Table 2 are the comparison results of various indicators in the ablation experiment. Figure 3 Improve the model:

[0095] Table 1. Semantic segmentation index scores of each model in the ablation experiment

[0096]

[0097]

[0098] Table 2 Ablation experiment target detection index scores of each model

[0099]

[0100] It is obvious that the new model built based on the YOLO v8 network backbone model and the addition of a multi-dimensional dynamic convolutional attention mechanism has better detection results.

[0101] A02: Construction of a tree structure calculation method for panoramic street view images;

[0102] The construction steps are as follows:

[0103] b1: Tree crown and trunk structure identification: For tree and crown trunk separation identification, the Fstats function is used to estimate the mutation point to separate and identify the crown and trunk of the tree. The row where the mutation point is located is regarded as the row where the bottom baseline of the tree is located.

[0104] Specifically: When identifying a single tree, the network uniformly identifies the trunk and crown as a tree. In order to automatically measure the tree diameter at breast height through images, the crown and trunk of the detected single tree need to be separated. The pixel width of each row in the tree segmentation result is quite different between the crown and the trunk. Therefore, the present invention uses the Fstats function in the strucchange package of the R software to estimate the mutation point to separate the crown and the trunk of the tree, and regards the row where the identified mutation point is located and the row where the bottom baseline of the tree is located as the trunk part. For each potential change point, the F statistic (Chow test statistic) is calculated. The larger the value of the F statistic, the more significant the parameter difference between the intervals, and thus the more likely the existence of a mutation point. The calculation formula of the F statistic is as follows:

[0105]

[0106] like Figure 4 As shown, Figure 4 It is the effect of mutation point detection;

[0107] b2: Panoramic Cylindrical Projection: This uses the camera's height and orientation, as well as the image size, vertical and horizontal fields of view, to graphically construct the tree structure in the panoramic image.

[0108] b3: Tree structure calculation: In order to measure the distortion of pixels at different positions caused by the fisheye lens and stitching, the vertical (C v,i ) and level (C h,i ) correction factor;

[0109] Specifically, the tree structure in the panoramic image is analyzed from a graphical perspective by using the camera's height and direction as well as the image size, vertical and horizontal fields of view, and other characteristics of the panoramic image. This paper is based on the following assumptions: (1) The panoramic image is captured in a flat space, without considering the influence of the terrain; (2) The camera's pitch angle is 0 (i.e., the angle between the camera lens and the horizontal plane) and remains horizontal; (3) The trees are planted on the ground; (4) The spherical cylindrical projection is used to project the 360° panoramic image from a sphere to a cylinder. Figure 5 This is a cylindrical projection diagram. Based on assumptions 1 and 2, it can be concluded that the camera's viewpoint can be regarded as the center line of the image, which is the same height as the camera height. According to assumption 3, the distance from the tree base to the center line pixel represents the camera height (VH). Assumption 4 shows that the pixel size will change with the distance of the pixel from the center line. Figure 6 is a schematic diagram of panoramic image information. Figure 5 Schematic diagram of cylindrical projection (where θ i is the vertical angle of the pixel, D i is the vertical distance from the ith pixel to the center line, and H / 2 is half the image height, that is, the distance from the center line to the top. ).

[0110] Specifically: In order to measure the distortion of pixels at different positions caused by the fisheye lens shooting and stitching, the angle (θ i ), the present invention calculates the vertical (C v,i ) and level (C h,i ), when a pixel moves away from the center line, the vertical correction factor increases from 1 to infinity and the horizontal correction factor decreases from 1 to 0 to perform correction.

[0111] After mapping the image to the cylinder, the vertical angle (θ i ) can be expressed as:

[0112]

[0113] Based on the assumption, the reference pixel size W can be calculated by dividing the pixel distortion from the baseline to the midline by the camera height (VH) r , that is, the actual size of the pixel without distortion.

[0114]

[0115] Where n is the number of pixels from the bottom of the tree to the midline, C v,i is the vertical scaling factor of the i-th pixel, W r is the reference pixel height without distortion.

[0116] Select trees that are close to the camera and evenly distributed on both sides of the center line to calculate W r , calculated W r The value is 0.03m. Finally, the height and diameter of the street trees can be expressed as:

[0117]

[0118] Where k and n are the number of rows of vertical tree pixels at the bottom and top of the tree, respectively.

[0119]

[0120] Where k and j are the vertical row numbers of the tree bottom and trunk top respectively; N i is the number of pixels in row i that are classified as trees, and the curly brackets in the equation indicate the order of indexes from bottom to top. 10 Indicates taking the 10th quantile.

[0121] A03: Construction of a network for calculating the spatial position of urban street trees;

[0122] The construction steps are as follows:

[0123] c1: extract tree benchmarks;

[0124] Specifically, this method considers the base of a street tree as a reference point and calculates the tree's latitude and longitude coordinates by obtaining its depth value on the depth estimation image. The coordinates of the lower left and lower right corners of the tree's bounding box detected by the network are (Xleft, Ybottom) and (Xright, Ybottom), respectively. Therefore, the tree's reference point is (Xleft + Xright) / 2.

[0125] c2: Generate a depth map of the panoramic image and calculate the depth values ​​of the tree benchmarks;

[0126] Specifically: Depth estimation estimates depth information through disparity (d), which is the offset of the same object in two images. The MonoDepth monocular depth estimation network can solve the disparity field without the need for ground truth depth. It transforms the input image by predicting the disparity to reconstruct the other side image. Given the baseline distance b between the cameras and the camera focal length f, the depth can be recovered from the predicted disparity. The calculation formula is as follows:

[0127]

[0128] This paper uses MonoDepth pre-trained on the Cityscape dataset and fine-tuned using the KITTY visual benchmark dataset to predict Baidu panoramic images and generate disparity maps. Since these pre-trained parameters are set for datasets such as KITTI and Cityscapes, the image size difference between the input image (b1) and the original Cityscapes and KITTY datasets (b0) needs to be corrected during post-processing. The distance between the street trees in the image and the camera position when the image was captured is the pixel value (in meters) of the corrected depth estimate image. The correction formula is as follows:

[0129]

[0130] The camera parameters and image information involved are shown in Table 3:

[0131] Table 3 Parameter information used in the present invention

[0132]

[0133] The results generated by MonoDepth show that for example Figure 7 ;

[0134] c3: tree position measurement;

[0135] Specifically: To measure the position of street trees in panoramic images, the panoramic image is divided into five regions. According to the input data format requirements of the MonoDepth network, the depth estimation image size is transformed to (256, 512). The spans of each region are 0-64, 64-192, 192-320, 320-448, 448-512, and the center line of each region is 0, 128, 256, 384, and 512. According to the region where the street tree benchmark point is located, the depth and yaw angle of the benchmark point are calculated respectively. Figure 8 Schematic diagram of position measurement method;

[0136] Specifically: According to the principle of similar triangles, the angle θ' can be calculated:

[0137]

[0138] Where a is the distance from the bottom of the tree to the center line of the photo, and w is the width of the photo.

[0139] The latitude and longitude information of trees can be obtained by the following formula:

[0140] X=X'+depth×cosθ#(24)

[0141] Y=Y'+depth×sinθ#(25)

[0142] where (x', y') are the camera coordinates; θ is the yaw angle between the camera and the base of the tree, calculated according to the following formula:

[0143] θ=θ'+θ0#(26)

[0144] θ0 is the default SVI tile yaw angle, and θ' is the yaw angle in depth image space.

[0145] A04: Draw a spatial distribution map of carbon storage of urban street trees.

[0146] Specifically: Using tree structural information (tree height and diameter at breast height), we can establish a single-tree allometric growth equation to obtain tree biomass. Tree biomass consists of aboveground biomass (such as trunk, bark, branches, and leaves) and underground biomass. Multiplying biomass by carbon content can be used to estimate tree carbon storage. The carbon storage estimation formula for hardwood broadleaf street trees is as follows:

[0147] W B =0.0104(D 2 H) 0.9994 #(17)

[0148] W L =0.0188(D 2 H) 0.8024 #(18)

[0149] W T =W S +W P +W B +W L #(19)

[0150] W R =0.0197(D 2 H) 0.8963 #(20)

[0151] W=W T +W R #(twenty one)

[0152] C=W×F#(22)

[0153] Where W S represents trunk biomass; W P represents bark biomass; W B represents the biomass of branches; W L represents the leaf biomass; W T represents the total aboveground biomass; W R represents the underground biomass; D represents the average diameter at breast height of the plot; H represents the average tree height of the plot; W represents the total biomass of the trees; W T represents the total aboveground biomass; WR represents the underground biomass; C is the carbon storage; and F is the carbon content.

[0154] The purpose of the present invention is to construct a technical system suitable for estimating the carbon storage of urban street trees based on panoramic street view images combined with a multi-task convolutional neural network method. First, in order to solve the tree occlusion problem and make the network more suitable for urban street tree detection and segmentation, a multi-task convolutional neural network is constructed based on the YOLO v8 network to realize the recognition of street trees in street view images at the level of single trees. Then, in order to calculate the carbon storage of single street trees, a panoramic image tree structure solution method is proposed. This method can directly obtain the height and diameter of single trees from the image based on the single tree recognition results. Finally, in order to further analyze the carbon storage distribution of urban street trees, a single tree position measurement method of panoramic street view images is realized.

[0155] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the present invention has been disclosed as a preferred embodiment as above, it is not intended to limit the present invention. Any person skilled in the art can make some changes or modifications to equivalent embodiments using the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A method for estimating carbon storage of urban street trees at the individual tree level, characterized by: The following steps are involved: A01: Building a multi-task street tree recognition network for street view images; The construction steps are as follows: a1: Acquisition of panoramic street view data: Generate sampling point data based on OSM road network data and obtain panoramic street view images through Baidu API; a2: Panoramic image roadside tree annotation dataset: Use Labelme to annotate the acquired panoramic images and divide them into training and test sets; a3: Model Improvement: A multi-dimensional dynamic convolutional attention mechanism was added to the YOLO v8 network backbone model to form a new multi-task network model, namely the street view image multi-task street tree recognition network; The multi-dimensional dynamic convolutional attention mechanism uses a parallel strategy to learn the complementary attention of convolution kernels in all four dimensions of the kernel space, namely the spatial size, the number of input channels, the number of output channels, and the number of convolution kernels, to improve the occlusion problem in urban street tree detection. The bounding box loss function is completely replaced by the intersection over union (CIoU) loss function to the dynamic non-monotonic focusing mechanism loss function WIoU to reduce the impact of low-quality annotation boxes on the results and improve the model detection effect. A02: Construction of a tree structure calculation method for panoramic street view images; The construction steps are as follows: b1: Tree crown and trunk structure identification: For tree and crown trunk separation identification, the Fstats function is used to estimate the mutation point to separate and identify the crown and trunk of the tree. The row where the mutation point is located is regarded as the row where the bottom baseline of the tree is located. b2: Panoramic Cylindrical Projection: This uses the camera's height and orientation, as well as the image size, vertical and horizontal field of view, to graphically construct the tree structure in the panoramic image. b3: Tree structure calculation: To measure the distortion of pixels at different positions caused by the fisheye lens and stitching, calculate the vertical correction factor C v,i and horizontal correction factor C h,i ; A03: Construction of a network for calculating the spatial position of urban street trees; The construction steps are as follows: c1: extract tree benchmarks; c2: Generate a depth map of the panoramic image and calculate the depth values ​​of the tree benchmarks; c3: tree position measurement; A04: Drawing of carbon storage distribution map of urban street trees.

2. The method for estimating carbon storage of urban roadside trees at the single tree level according to claim 1 is characterized in that: In step a3 of step A01, the bounding box loss function complete intersection over union (CIoU) and the loss function WIoU of the dynamic non-monotonic focusing mechanism are expressed as follows:

3. The method for estimating carbon storage of urban roadside trees at the single tree level according to claim 2 is characterized in that: After step a3 in step A01, the established multi-task street tree recognition network for street view images needs to be evaluated. Precision, recall, and average intersection-over-union (IoU) are used to evaluate the network. Higher scores on these metrics indicate better model performance. The details are as follows: Based on the four prediction results in the confusion matrix: true positive TP, false positive FP, true negative TN and false negative FN, various indicators can be calculated. TP is the prediction of a true positive example as a positive example, FP is the prediction of a true negative example as a positive example, TN is the prediction of a true negative example as a negative example, and FN is the prediction of a true positive example as a negative example. Precision focuses on how many of the predicted positive examples are true positive examples, reflecting the accuracy of the model's prediction and more in line with actual demands. The specific formula is as follows: Recall rate focuses on how many positive examples in the real samples are predicted. The specific formula is as follows: The mean intersection over union (mIoU) is a standard metric in semantic segmentation tasks. It focuses on the average of the ratio of the intersection of all predicted positive classes and the true positive classes to the union of the two. The specific formula is as follows:

4. The method for estimating carbon storage of roadside trees at the individual tree level according to claim 1, characterized in that: In step b1 of step A02, for each potential change point, the F statistic is calculated. The larger the value of the F statistic, the more significant the parameter difference between the intervals, and thus the more likely there is a mutation point. The calculation formula of the F statistic is as follows:

5. The method for estimating carbon storage of roadside trees at the individual tree level according to claim 4 is characterized in that: In b2 of step A02, the construction of the tree structure in the panoramic image is based on the following assumptions: X1: The panoramic image is captured in a plane space, and the influence of the undulating terrain is not considered; X2: The pitch angle of the camera is 0, that is, the angle between the camera lens and the horizontal plane remains horizontal; X3: The trees are planted on the shooting ground; X4: The 360° panoramic image is projected from a sphere to a cylinder using spherical cylindrical projection.

6. The method for estimating carbon storage of urban roadside trees at the single tree level according to claim 4 is characterized in that: In step b3 of step A02, the correction factor is as follows: when a pixel moves away from the center line, the vertical correction factor increases from 1 to infinity, and the horizontal correction factor decreases from 1 to 0, and correction is performed; After mapping the image to the cylinder, the vertical angle θ of each pixel i Expressed as: Based on the assumption, the reference pixel size W can be calculated by dividing the pixel distortion from the baseline to the midline by the camera height VH r , that is, the actual size of the pixel without distortion; Where nh is the number of pixels from the bottom of the tree to the midline, C v,i is the vertical correction factor of the i-th pixel, W r is the reference pixel height without distortion; Select trees that are close to the camera and evenly distributed on both sides of the center line to calculate W r , calculated W r Taking the value as 0.03m, the height and diameter at breast height of the street trees can be expressed as: Where nb and nt are the number of rows of pixels in the vertical direction of the tree bottom and tree top respectively; Where nb and j are the number of vertical pixel rows at the bottom and top of the tree respectively; N i is the number of pixels in row i that are classified as trees. The curly brackets in the equation indicate the order of indexes from bottom to top. 10 Indicates taking the 10th quantile.

7. A method for estimating carbon storage of urban roadside trees at the single tree level according to any one of claims 1 to 6, characterized in that: In c1 of step A03, the bottom of the street tree is regarded as the reference point, and the latitude and longitude coordinates of the street tree are calculated by obtaining the depth value of the reference point on the depth estimation image. The coordinates of the lower left and lower right corners of the street tree bounding box detected by the network are (Xleft, Ybottom) and (Xright, Ybottom), so the tree reference point is (Xleft+Xright) / 2.

8. The method for estimating carbon storage of roadside trees at the individual tree level according to claim 7, characterized in that: In step A03, in c2, depth estimation estimates depth information through disparity d. Disparity is the offset of the same object in two images. The MonoDepth monocular depth estimation network can solve the disparity field without the need for ground truth depth. It transforms the input image by predicting the disparity to reconstruct the other side image. Given the baseline distance b between the cameras and the camera focal length f, the depth can be recovered from the predicted disparity. The calculation formula is as follows: Use MonoDepth pre-trained on the Cityscape dataset and fine-tuned with the best pre-trained weights on the KITTY visual benchmark dataset to predict Baidu panoramic images and generate disparity maps. Since these pre-trained parameters are set for the KITTI and Cityscapes datasets, it is necessary to correct for the image size difference between the input image b1 and the original Cityscapes and KITTY datasets b0 during post-processing. The distance between the street trees in the image and the camera position when the image was captured is the pixel value of the corrected depth estimate image, in meters. The correction formula is as follows:

9. The method for estimating carbon storage of urban roadside trees at the single tree level according to claim 8, characterized in that: In step c3 of step A031, in order to realize the position measurement of the street trees in the panoramic street view image, the panoramic image is divided into multiple areas, and the depth and yaw angle of the reference points are calculated respectively according to the areas where the street tree reference points are located.

10. The method for estimating carbon storage of urban roadside trees at the single tree level according to claim 9, characterized in that: In step A04, the tree structure information is used to establish a single tree allometric growth equation to obtain tree biomass. Tree biomass consists of aboveground biomass (trunk, bark, branches, and leaves) and underground biomass. The tree carbon storage can be calculated by multiplying the biomass by the carbon content. The carbon storage estimation formula for street trees is as follows: W B =0.0104(D 2 H) 0.9994 (17); W L =0.0188(D 2 H) 0.8024 (18); IN T =In S +W P +W B +W L (19); W R =0.0197(D 2 H) 0.8963 (20); W=W T +W R (21); C = W × F (22); Where W S represents the trunk biomass, W P represents bark biomass, W B represents the branch biomass, W L represents the leaf biomass, W T represents the total aboveground biomass, W R represents the underground biomass, D represents the average diameter at breast height of the plot, H represents the average tree height of the plot, W represents the total biomass of the trees, and W represents the average tree height. T represents the total aboveground biomass, W R represents the underground biomass, C is the carbon storage, and F is the carbon content.

Images