A method for estimating regional vegetation three-dimensional green volume based on reconstructed point cloud

Abstract

The present invention provides a method for estimating the three-dimensional green volume of regional vegetation based on reconstructed point clouds, and belongs to the technical field of three-dimensional green volume estimation. The method analyzes the problems existing in the existing urban three-dimensional green volume measurement model, and is designed in combination with the inherent characteristics of the three-dimensional reconstructed point cloud itself and the requirements of model versatility. The method performs three-dimensional reconstruction based on unmanned aerial vehicle remote sensing images to obtain three-dimensional reconstructed point cloud data, focusing on the octree-voxel theory, and innovatively adds a scan line filling optimization method to obtain an improved model for three-dimensional green volume measurement, thereby completing the measurement of the three-dimensional green volume of the city. The method of the present invention effectively solves the problem of inaccurate measurement in the existing three-dimensional green volume quantitative measurement method.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional green volume estimation, and in particular relates to a method for estimating the three-dimensional green volume of regional vegetation based on reconstructed point clouds. Background Art

[0002] As an emerging ecological quantitative indicator, Living Vegetation Volume (LVV) can reflect the changes in the greening effect of the measured area with spatial distribution, and can be more directly related to other ecological indicators such as biomass. Therefore, it is widely used in urban ecological environment assessment and urban planning. At the same time, three-dimensional point cloud data is widely used in remote sensing and surveying and mapping, but is rarely directly used in the estimation of three-dimensional green volume of vegetation in urban areas. With the increasing maturity of UAV remote sensing technology and three-dimensional reconstruction technology, the use of UAV remote sensing images to carry out real-scene three-dimensional reconstruction technology is rapidly emerging. This cross-research field provides a faster response, lower cost, and richer data collection method for three-dimensional green volume estimation in urban areas. The point cloud data generated by three-dimensional reconstruction has a more detailed surface texture and has more advantages in recording vegetation canopy characteristics than other forms of point cloud data. However, through investigation, the following problems exist in related research:

[0003] (1) Existing three-dimensional green volume quantitative model methods, such as the study by Fang Huang et al. (Fang Huang*, Shuyi ng Peng, Shengyi Chen, Hongxia Cao, Ning Ma. VO-LVV—A novel urban regional living vegetation volume quantitative estimation model based on the voxel measurement method and an octree data structure[J]. Remote Sensing, 2022, 14(4):855.), are mainly based on the measurement of growth parameters of a single tree species. Different sets of model parameters are required to deal with different types of tree species, making it difficult to achieve universality of the measurement model;

[0004] (2) The existing three-dimensional green volume quantification model method is mainly based on single tree modeling, which cannot be applied to conditions such as a large number of tree species and complex forest layer structure in a region, and is not suitable for large-scale application in urban areas;

[0005] (3) The existing three-dimensional reconstruction based on UAV remote sensing images is time-consuming and inefficient. In addition, due to the complex texture information of vegetation and other objects and the limited photography angle of UAV remote sensing, the vegetation reconstruction effect is poor during the three-dimensional reconstruction process and the lack of understory information makes it difficult to directly calculate the three-dimensional green volume in urban areas based on three-dimensional point cloud data.

[0006] Therefore, how to conduct convenient, fast and universal quantitative analysis of three-dimensional green space in urban areas based on three-dimensional reconstructed point cloud data has become a problem to be solved. Summary of the invention

[0007] In view of the problems existing in the background technology, the purpose of the present invention is to provide a method for estimating the three-dimensional green volume of regional vegetation based on reconstructed point clouds. The method collects UAV remote sensing data and uses high-performance computing technology to achieve rapid three-dimensional reconstruction of regional vegetation scenes to obtain three-dimensional reconstructed point cloud data; and based on the three-dimensional reconstructed point cloud data, the PointNet++ network model is used to complete the extraction of regional vegetation point clouds, and finally a dense point cloud model reflecting the spatial distribution of green plants is obtained based on the three-dimensional reconstruction technology, and an octree and voxel quantization model suitable for three-dimensional green volume is established, and finally the quantitative estimation of three-dimensional green volume is completed.

[0008] To achieve the above object, the technical solution of the present invention is as follows:

[0009] A method for estimating the three-dimensional green volume of regional vegetation based on reconstructed point clouds comprises the following steps:

[0010] Step 1. Select an urban area with typical vegetation as the area to be measured, and use the UAV oblique photogrammetry system to perform oblique photogrammetry on the area to obtain oblique image data;

[0011] Step 2. Based on the open source 3D reconstruction algorithm, the oblique image data set obtained in step 1 is sparsely reconstructed and densely reconstructed to obtain the 3D reconstruction point cloud data of the scene in the measured area;

[0012] Step 3. For the scene 3D reconstruction point cloud data obtained in step 2, use the point cloud semantic segmentation neural network model to extract vegetation point cloud data;

[0013] Step 4. Construct a regional three-dimensional green volume quantitative model for the vegetation point cloud data obtained in step 3, specifically: first establish a voxel model under the octree search structure, and then use the scan line filling algorithm to correct the voxel structure of the regional vegetation, so as to obtain a regional three-dimensional green volume quantitative model;

[0014] Step 5. Input the vegetation point cloud data extracted in step 3 into the regional three-dimensional green volume quantification model obtained in step 4 to estimate the three-dimensional green volume.

[0015] Furthermore, the typical vegetation in step 1 refers to vegetation with a canopy of regular geometric shape, and the three-dimensional green volume true value can be calculated using the "crown diameter-crown height" equation.

[0016] Furthermore, the UAV oblique photogrammetry operation system in step 1 is composed of DJI M300 RTK and Zenmuse P1 image sensor; wherein the image sensor is fixed on the UAV, and the UAV related aerial survey parameters are set, including altitude h=60m, speed v=3m / s, heading overlap rate of 80%, lateral overlap rate of 70%, etc.

[0017] Furthermore, in step 1, 3-4 image control points are arranged in the area to be measured, and their spatial position information is measured by the RTK base station; each image control point should be captured by as many oblique images as possible, and should be selected at an easily identifiable corner position.

[0018] Furthermore, the specific process of step 2 is:

[0019] Step 2.1. Based on the oblique image dataset obtained in step 1, the scale-invariant feature transform (SIFT) algorithm is used to extract feature points, and then the extracted feature points are matched to obtain feature matching points;

[0020] Step 2.2. Based on the feature matching points, the sparse point cloud is reconstructed using the Structure from Motion (SfM) algorithm to obtain sparse point cloud data;

[0021] Step 2.3. Based on the sparse point cloud, the clustering and patch model-based multi-view dense matching (Clustering Multiple View Stereo & Patch-based Multi-View Stereo, CMVS+PMVS) algorithm is used to reconstruct the dense point cloud to obtain dense point cloud data; the dense point cloud data is the scene 3D reconstruction point cloud data.

[0022] Furthermore, the point cloud semantic segmentation neural network model in step 3 includes an encoder and a decoder, wherein the encoder is used to obtain feature extraction results, and the decoder is used to obtain vegetation point cloud data with the same amount of original point cloud data;

[0023] The encoder consists of a sampling layer and a combination layer. The sampling layer represents the dense point cloud data into a high-dimensional space composed of coordinate values ​​and eigenvalues. The farthest point sampling method is used to select the N1 point clouds with the longest spacing from the N input point clouds in the metric space, and this group of point clouds is used as a group of regional center points. In the combination layer, the nearest k points of each regional center point are selected to represent the region and obtain the feature extraction result.

[0024] The decoder consists of several groups of decoding units, each of which consists of an insertion layer and a PointNet unit layer. After the encoder completes feature extraction, the model will access the PointNet layer as a feature extractor to aggregate the features of k neighbors through trainable weights.

[0025] The insertion layer first completes the point cloud upsampling through the distance weighted features of the k nearest neighbor points, as shown in formula (1):

[0026]

[0027] Among them, d(x,x i ) is the Euclidean distance from point x to the i-th neighbor point, P represents the power of the Euclidean distance, f i (j) represents the i-th neighbor point of the j-th layer, a total of k, f (j) (x) represents the output of point x in the jth layer, and C is the total number of layers.

[0028] Furthermore, the specific process of step 3 is as follows:

[0029] Step 3.1. Screen the 3D reconstructed point cloud and remove the data with poor reconstruction quality (referring to the vegetation point cloud data with incomplete canopy morphology due to the lack of photogrammetry perspective, especially the vegetation point cloud data with large-scale holes on the lower surface of the canopy);

[0030] Step 3.2. Cut the obtained data into blocks and manually annotate each block with semantic labels. The vegetation information of interest in this study is marked as one attribute class, and other ground objects are marked as another class;

[0031] Step 3.3. Divide the labeled data into a training set and a test set according to a certain ratio (approximately 5:1); the training set data is used for training the point cloud semantic segmentation neural network model parameters, and the test set is used to detect the actual effect of the network after training and obtain the network model parameters;

[0032] Step 3.4. Based on the trained point cloud semantic segmentation neural network model, input the scene 3D reconstructed point cloud data to obtain vegetation point cloud data.

[0033] Furthermore, the specific process of step 4 is as follows:

[0034] Step 4.1. Count the maximum and minimum values ​​of vegetation point cloud data in the three coordinate directions of X, Y, and Z, recorded as (x min ,x max ),(y min ,y max ),(z min ,z max ); with (x min ,y min ,z min ) and (x max ,y max ,z max ) to construct a three-dimensional bounding box of vegetation point cloud data as the partition space of the voxel model and octree;

[0035] Step 4.2. According to the set voxel resolution, a voxel model is constructed, and on this basis, an octree index is established for each vegetation point cloud;

[0036] Step 4.3. Sort all non-empty voxels according to the voxel center coordinates, first stratify the voxel set according to the size of the Z-axis coordinate; in each layer of non-empty voxel subsets, first sort according to the size of the X-axis coordinate, then sort according to the Y-axis, and traverse the voxels according to the sorting results; when a voxel is at the same X-axis coordinate as the previous voxel, it constitutes a scan line;

[0037] Step 4.4. Preset a scan line length threshold and determine the canopy part to which each scan line belongs: if the scan line is too long, it is considered that this section belongs to the interval between two adjacent trees, otherwise it can be considered that this section is part of the canopy;

[0038] Step 4.5. For the scan line inside the tree crown, calculate the volume according to its length and the cross-sectional area of ​​the voxel, and obtain the cross-sectional volume of the canopy on this scan line;

[0039] Step 4.6. Using the same method, repeat steps 4.3 to 4.5 for each voxel along the contour in the X and Y directions to obtain the canopy volume of each layer, and then add up the canopy volume of each layer to obtain the total canopy volume V of the three-dimensional green volume to be measured.

[0040] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0041] The present invention analyzes the problems existing in the existing urban three-dimensional green volume measurement model, and designs the method of the present invention in combination with the inherent characteristics of the three-dimensional reconstruction point cloud and the requirements of model versatility. The method performs three-dimensional reconstruction based on drone remote sensing images to obtain three-dimensional reconstruction point cloud data, and focuses on the octree-voxel theory. The scanning line filling optimization method is innovatively added to obtain an improved model for three-dimensional green volume measurement, thereby completing the measurement of the three-dimensional green volume of the city. The method of the present invention effectively solves the problem of inaccurate measurement in the existing three-dimensional green volume quantitative measurement method. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a schematic diagram of the flow chart of the method for estimating the three-dimensional green volume of regional vegetation according to the present invention.

[0043] Figure 2 Schematic diagram of the process of constructing a 3D dense point cloud based on UAV images.

[0044] Figure 3 The present invention is a flow chart of a three-dimensional green volume estimation method based on the octree-voxel method in the prior art.

[0045] Figure 4 This is a flow chart of the three-dimensional green volume measurement method based on scan line filling according to the present invention. DETAILED DESCRIPTION

[0046] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below in conjunction with the implementation modes and the accompanying drawings.

[0047] A method for estimating the three-dimensional green volume of regional vegetation based on reconstructed point cloud. The flow chart of the method of the present invention is as follows: Figure 1 As shown, the following steps are included:

[0048] Step 1. Select an urban area with typical vegetation as the area to be measured. The typical vegetation refers to the vegetation with regular geometric canopy shape, which can use the "crown diameter-crown height" equation to calculate the true value of the three-dimensional green volume. Use the UAV oblique photogrammetry system to perform oblique photogrammetry on the measurement area to obtain oblique image data. The specific process is as follows:

[0049] Step 1.1. Select DJI M300 RTK and Zenmuse P1 image sensor to form the UAV oblique photogrammetry system of the present invention;

[0050] Step 1.2. Mount the image sensor on the drone and turn on the drone. In the operating software that comes with the drone, connect the drone to the wireless hotspot and set the relevant aerial survey parameters, including altitude h = 60m, speed v = 3m / s, heading overlap rate of 80%, lateral overlap rate of 70%, etc.

[0051] Step 1.3. In the operation software, define a quadrilateral area that can completely cover the area to be measured as the operation area, and automatically plan the route according to the aerial survey parameters set in step 1.2: Generally, the flight direction is the direction parallel to the longest side in the operation area. The drone will return and scan along this direction in the operation area until it has completed the operation area.

[0052] Step 1.4. To improve the accuracy of 3D reconstruction in the office, 3-4 image control points are set up in the survey area, and the spatial position information of the image control points is determined by the RTK base station; each image control point should be captured by as many oblique images as possible and should be set at an easily identifiable corner position;

[0053] Step 2. Based on the open source 3D reconstruction algorithm, the oblique image data set obtained in step 1 is sparsely reconstructed and densely reconstructed to obtain the 3D reconstruction point cloud data of the scene in the measured area. The flow chart of step 2 is as follows: Figure 2 As shown, the specific process is:

[0054] Step 2.1. Based on the oblique image dataset obtained in step 1, the SIFT algorithm is used to extract feature points, and then the extracted feature points are matched in multiple images to obtain feature matching points. The specific matching process is as follows:

[0055] Find the point p closest to each ray in three-dimensional space, which corresponds to the camera P j =K j [R j |t j ] The position of the observed feature point on the image {x j}, where K j is the internal parameter matrix, [R j |t j ] is an external parameter, and the camera ray can be expressed as formula (1):

[0056]

[0057] Among them, q is the point on the ray, c j is the camera position, d is the length coefficient, is the imaging direction vector;

[0058] Let q be the point on each ray closest to point p j , then q j It is necessary to satisfy formula (2) to obtain the minimum:

[0059]

[0060] According to the mathematical model, the solution of each parameter is as follows:

[0061]

[0062] Among them, r j 2 For p and q j The 2-norm between j is the minimum value of d in formula (2), r j For p and q j The distance between them, j is the camera number in the photogrammetry process;

[0063] The solved p is the feature matching point. The solution of point p can be obtained by accumulating r j 2 Solve the optimal value of point p as a least squares problem or by minimizing the residual of the measurement equation;

[0064] Step 2.2. Based on the feature matching points, the sparse point cloud is reconstructed using the Structure from Motion (SfM) algorithm to obtain sparse point cloud data;

[0065] Step 2.3. Based on the sparse point cloud, the Clustering Multiple View Stereo & Patch-based Multi-View Stereo (CMVS+PMVS) algorithm is used to reconstruct the dense point cloud to obtain dense point cloud data. That is, based on the sparse point cloud, the patch is divided and the patch structure is expanded by patch diffusion to obtain a dense point cloud. The patch is a local tangent plane that approximates the surface of the object, usually determined by the center point c(P), the normal vector n(P), and the associated reference image set V(P); the obtained dense point cloud data is the scene 3D reconstruction point cloud data;

[0066] Step 3. For the scene 3D reconstruction point cloud data obtained in step 2, use the point cloud semantic segmentation neural network model to train and extract vegetation point cloud data. The specific process is as follows:

[0067] Step 3.1. Screen the 3D reconstructed point cloud and remove the data with poor reconstruction quality (referring to the vegetation point cloud data with incomplete canopy morphology due to the lack of photogrammetry perspective, especially the vegetation point cloud data with large-scale holes on the lower surface of the canopy);

[0068] Step 3.2. Cut the obtained data into blocks and manually annotate each block with semantic labels. The vegetation information of interest in this study is marked as one attribute class, and other ground objects are marked as another class;

[0069] Step 3.3. Divide the labeled data into a training set and a test set according to a certain ratio (approximately 5:1); the training set data is used for training the point cloud semantic segmentation neural network model parameters, and the test set is used to detect the actual effect of the network after training and obtain the network model parameters;

[0070] Step 3.4. Based on the trained point cloud semantic segmentation neural network model, input the scene 3D reconstructed point cloud data to obtain vegetation point cloud data.

[0071] In the prior art, the octree-voxel method is directly used to estimate the three-dimensional green volume based on the extracted vegetation point cloud data. The specific process is as follows: Figure 3 As shown, the process is:

[0072] Step 4.1. For the extracted vegetation point cloud data, construct a voxel model, specifically: count the maximum and minimum values ​​of the vegetation point cloud data in the three coordinate directions of X, Y, and Z, recorded as (x min ,x max ),(y min ,y max ),(z min ,z max ); with (x min ,y min ,z min ) and (x max ,y max ,z max ) to determine a maximum bounding box that encloses all point clouds and obtain the root node of the 0th level of the octree;

[0073] In the octree structure, each point cloud space is divided equally from the three coordinate directions into 8 parts. In theory, there are three possibilities for each subspace:

[0074] (1) The subspace is completely filled with data, but for point cloud data, there is no concept of volume, so there is no such thing as complete filling;

[0075] (2) There is no data in the subspace, that is, it is an empty node. In this case, there is no need to continue dividing in this subspace;

[0076] (3) There is data distribution in the subspace, that is, it is a non-empty node. At this time, the point cloud space is further divided into 8 sub-cubes, and the subdivision is performed recursively in sequence.

[0077] Repeat the above determination and division operation for each point cloud space in the octree until there is no data point in the current point cloud space, or the current space volume has reached the preset minimum value, then stop the loop iteration and obtain the voxel model;

[0078] Step 4.2. Establish an octree search structure for the point cloud data in the voxel model. In order to facilitate the voxel neighbor search, a unified octree node position information representation method is introduced in the octree, specifically:

[0079] Assuming that the sub-nodes divided in a point cloud space node can be numbered 0-7, the sub-node number oct i It can be expressed as formula (5),

[0080] oct i =x i +2y i +4z i (5)

[0081] where x i ,y i , z i is a 0-1 binary variable, whose geometric meaning is that the child node is located in the front / back half, left / right half, top / bottom half of the parent node being divided; the position of a node can be represented by a series of octal numbers, where each bit represents the child node number of an ancestor node of this node in the parent node of this ancestor; based on this node numbering method, the position of the node in the entire octree space can be expressed as (6):

[0082]

[0083] For the point cloud to be searched, by solving the spatial position code of the point, the voxel space where the point is located can be located, and all the point clouds in the voxel space are used as the basis for determining the belonging of the newly added point, thereby completing the voxel neighbor search;

[0084] Step 4.3. Traverse all non-empty voxels separately. At the same time, in order to avoid too few point clouds in the divided voxels, which will result in too large green volume values ​​in the final calculation, m i The point cloud contained in the voxel with centered on must meet the preset density condition. Assuming that the target voxel density threshold under the octree partition is t, the vegetation voxel suitable for consideration must meet formula (7):

[0085]

[0086] Among them, V i is the volume of a single voxel;

[0087] Step 4.4. When the density is lower than this threshold, it is considered that the vegetation information in the voxel is sparse and should be discarded to avoid an overly large estimate;

[0088] Step 4.5. When the density is greater than or equal to a certain threshold, the voxel is considered to be completely covered by the vegetation point cloud; the total three-dimensional green volume estimation volume is obtained by accumulating according to different levels, as shown in formula (8);

[0089]

[0090] Among them, M is the set of all voxel centers associated with the vegetation point cloud, H(m i ) is the node level;

[0091] Step 4.6. According to the observed original data, the data collection method, and the characteristics of the vegetation type, the derived three-dimensional green volume calculation formula is corrected by adding information completion parameters, morphology completion parameters, etc.; a more accurate corrected three-dimensional green volume value is obtained, as shown in formula (9);

[0092]

[0093] Among them, c(P) is the information completion parameter, and c(Q) is the morphological completion parameter. For c(P), due to the missing information in the three-dimensional point cloud data, c(P) is defined as the ratio of the complete vegetation point cloud information to the actual vegetation point cloud information, so as to correct the measurement results. By observing the vegetation point cloud data, it can be seen that for the data collected by airborne methods (airborne LiDAR / airborne three-dimensional reconstruction), only the upper half of the vegetation canopy information can be obtained, which can be approximately considered to be only half of the complete canopy model, that is, c(P) = 2; while for the vehicle-mounted LiDAR data, it can be considered that the back side of the tree canopy information is missing, that is, the amount of data obtained is only 0.75 times the original data, at this time c(P) = 4 / 3.

[0094] For c(Q), we assume that the true three-dimensional green volume is solved by the "crown diameter-crown height-volume" equation. If the theoretical value obtained by this formula method is to be taken as the true value, there is a major premise: assuming that the vegetation grows uniformly in all directions, its canopy satisfies isotropy, that is, the crown diameter values ​​measured from all directions are equal. Since this formula simulates the plant as a regular perfect circle in each cross section, but the cross sections of each layer of the plant actually measured are usually elliptical, the calculation method of the morphological completion parameter c(Q) is shown in formula (10).

[0095] in, is the maximum crown radial quantity measured at the maximum cross section of the vegetation point cloud, For the cross section vertical crown radial volume;

[0096] Step 4.7. Establish a polynomial function relationship between the corrected measured value and the true value, preset multiple fitting functions, solve multiple polynomial fitting functions of different highest orders at the same time, and then use indicators such as correlation coefficient and variance to evaluate the results, and select a better fitting function. Through parameter adjustment and fitting of the fitting function, the final corrected three-dimensional green quantity measurement value is obtained.

[0097] However, the green volume value obtained by the above three-dimensional green volume estimation method does not match the actual green volume value. Therefore, the present invention proposes an improved three-dimensional green volume measurement method based on octree-voxel filling of scan lines. This method still constructs the octree partitioning structure of the vegetation point cloud according to the set voxel resolution. The difference is that the measurement of the three-dimensional green volume is no longer considered to be a simple accumulation of the voxels where the detected vegetation point cloud is located, but it is necessary to fill the voxels of the internal hollow part into the result; the flowchart of the improved method of the present invention is as follows Figure 4 As shown, the specific process is:

[0098] Step 5.1. Construct a 3D bounding box of the vegetation point cloud data, which will serve as the partition space of the voxel model and the octree; construct the voxel model according to the set voxel resolution, and on this basis, establish an octree index for each vegetation point cloud;

[0099] Step 5.2. Sort all non-empty voxels according to the voxel center coordinates, and first stratify the voxel set according to the size of the Z-axis coordinate; according to the idea of ​​stratified measurement, divide the vegetation voxels in the space along the Z-axis. For each layer of voxel space, it can be considered as a slice of the vegetation point cloud at the same horizontal height. By accumulating the three-dimensional green volume of the above vegetation slices, the accurate measurement value of the three-dimensional green volume of regional vegetation in the region can be finally obtained;

[0100] Step 5.3. In each layer of non-empty voxel subsets, first sort according to the size of the X-axis coordinates, then sort according to the Y-axis, and traverse the voxels according to the sorting results; when a voxel is at the same X-axis coordinate as the previous voxel, a scan line is formed;

[0101] Step 5.4. Preset a scan line length threshold and determine the canopy part to which each scan line belongs: if the scan line is too long, it is considered that this section belongs to the interval between two adjacent trees, otherwise it is considered that this section is part of the canopy;

[0102] Step 5.5. For the scan line inside the tree canopy, calculate the canopy cross-sectional volume on this scan line based on its length and voxel cross-sectional area, specifically:

[0103] The key to measuring each layer of point cloud slices is to calculate the cross-sectional area. According to the scan line filling algorithm, a layer of voxel space is scanned along the X-axis or Y-axis direction to fill the hollow area of ​​the vegetation canopy slice. For example, for scanning in the X-axis direction, a starting voxel is picked near the lower left corner of the contour, and a line is traced from the starting voxel to the voxel on the other side of the contour in the positive direction of the X-axis; the first voxel is equivalent to the minimum value, and the last voxel is equivalent to the maximum X-axis coordinate value along the scan line, and all empty voxels between the starting and ending voxels are marked, and the canopy cross-sectional volume on this scan line is obtained, as shown in formula (11).

[0104]

[0105] Among them, x j max represents the x value at the larger end of the j-th scan line, x j min represents the x value at the smaller end of the j-th scan line, and is the y-axis and z-axis side length of a single non-empty child node at level i along the three-dimensional direction;

[0106] Step 5.6. Repeat steps 5.2 to 5.5 for each voxel along the contour in the X and Y directions in the same way, thereby obtaining the canopy volume of each layer, and then accumulating the canopy volume of each layer to obtain the total canopy volume V of the three-dimensional green volume to be measured, as shown in formula (12);

[0107]

[0108] Example 1

[0109] The same ellipsoidal vegetation in an experimental area was selected, and three point cloud data were obtained at the same time phase to compare three different 3D green volume measurement methods. Using the pre-processed single plant LAS point cloud data, based on the actual experience of 3D green volume measurement, the voxel resolution was set to 0.21m and the density threshold was set to 500 points / m 3 , the scan line length threshold is set to 4.0m, and the measurement results are shown in Table 1:

[0110] Table 1 Comparison of results of different three-dimensional green quantity measurement methods (ellipsoidal single plant)

[0111]

[0112] By testing this vegetation point cloud sample, we can draw the following experimental conclusions in single vegetation point cloud data:

[0113] (1) The three-dimensional green volume calculation method implemented in this study uses an octree organization method, which is faster than the convex hull method of constructing a triangular mesh. The acceleration ratio can generally reach about 54:1--196:1, showing good calculation efficiency;

[0114] (2) Especially when processing reconstructed point cloud data, the absolute value of the measurement error rate of this method is 11.2%, which has better measurement accuracy than the absolute value of the error rate of 26.8% of the traditional triangulated grid-convex hull method.

[0115] (3) For the improved octree-voxel measurement algorithm of this study, in addition to maintaining the high measurement efficiency brought by the octree and voxel model (the acceleration ratio generally reaches about 34:1--166:1), on the one hand, the measurement results of vegetation point cloud data obtained under different acquisition methods are relatively stable; on the other hand, the three-dimensional green volume measurement values ​​of this method on the three types of data are generally closer to the true value. Especially in the measurement task of three-dimensional reconstruction point cloud, the absolute value of the error rate is only 2.6% compared with the true value.

[0116] In order to verify whether the model has a certain universality in the measurement of vegetation with other crown forms, another vegetation with a cylindrical canopy was selected, and the "crown diameter-crown height-volume" equation was used to verify the model under the condition that other test conditions remained unchanged. The experimental results are shown in Table 2:

[0117] Table 2 Comparison of results of different three-dimensional green quantity measurement methods (cylindrical single plant)

[0118]

[0119] By testing another type of single vegetation point cloud sample, the following conclusions can be drawn: (1) Compared with the convex hull method, the octree-voxel-based 3D green volume measurement method in this paper has smaller variance in measurement results, better stability and versatility when facing point cloud data of different types of tree species collected in different ways; (2) Among the three types of point clouds, the three methods compared in this paper are the best in the measurement of 3D reconstructed point clouds; (3) Compared with the convex hull method, the measurement result deviation of the improved octree-voxel measurement method is slightly larger, but it has an absolute advantage in measurement speed.

[0120] The above description is only a specific implementation mode of the present invention. Any feature disclosed in this specification, unless otherwise stated, can be replaced by other alternative features that are equivalent or have similar purposes; all the disclosed features, or all the steps in the methods or processes, except for mutually exclusive features and / or steps, can be combined in any way.

Claims

1. A method for estimating the three-dimensional green volume of regional vegetation based on reconstructed point clouds, characterized in that: The following steps are involved: Step 1. Select an urban area with typical vegetation as the area to be measured, and use the UAV oblique photogrammetry system to perform oblique photogrammetry on the area to obtain oblique image data; Step 2. Based on the open source 3D reconstruction algorithm, the oblique image data set obtained in step 1 is sparsely reconstructed and densely reconstructed to obtain the 3D reconstruction point cloud data of the scene in the measured area; Step 3. For the scene 3D reconstruction point cloud data obtained in step 2, use the point cloud semantic segmentation neural network model to extract vegetation point cloud data; Step 4. Construct a regional three-dimensional green volume quantitative model for the vegetation point cloud data obtained in step 3, specifically: first establish a voxel model under the octree search structure, and then use the scan line filling algorithm to correct the voxel structure of the regional vegetation, so as to obtain a regional three-dimensional green volume quantitative model; The specific process is: Step 4.

1. Count the maximum and minimum values ​​of vegetation point cloud data in the three coordinate directions of X, Y, and Z, recorded as (x min ,x max ),(y min ,y max ),(z min ,z max ); with (x min ,y min ,z min ) and (x max ,y max ,z max ) to construct a three-dimensional bounding box of vegetation point cloud data as the partition space of the voxel model and octree; Step 4.

2. According to the set voxel resolution, a voxel model is constructed, and on this basis, an octree index is established for each vegetation point cloud; Step 4.

3. Sort all non-empty voxels according to the voxel center coordinates, and first stratify the voxel set according to the size of the Z-axis coordinate; In each layer of non-empty voxel subsets, first sort according to the size of the X-axis coordinates, then sort according to the Y-axis, and traverse the voxels according to the sorting results; when a voxel is at the same X-axis coordinate as the previous voxel, a scan line is formed; Step 4.

4. Preset a scan line length threshold and determine the canopy part to which each scan line belongs: if the scan line is too long, it is considered that this section belongs to the interval between two adjacent trees, otherwise it is considered that this section is part of the canopy; Step 4.

5. For the scan line inside the tree crown, calculate the volume according to its length and the cross-sectional area of ​​the voxel, and obtain the cross-sectional volume of the canopy on this scan line; Step 4.

6. Repeat steps 4.3 to 4.5 for each voxel along the contour in the X and Y directions in the same way to obtain the canopy volume of each layer, and then add up the canopy volume of each layer to obtain the total canopy volume V of the three-dimensional green volume to be measured; Step 5. Input the vegetation point cloud data extracted in step 3 into the regional three-dimensional green volume quantification model obtained in step 4 to estimate the three-dimensional green volume.

2. The method for estimating the three-dimensional green volume of regional vegetation based on reconstructed point cloud according to claim 1, characterized in that: The typical vegetation in step 1 refers to the vegetation with a canopy of regular geometric shape, and the three-dimensional green volume true value can be calculated using the "crown diameter-crown height" equation.

3. The method for estimating the three-dimensional green volume of regional vegetation based on reconstructed point cloud according to claim 1, characterized in that: The UAV oblique photogrammetry system in step 1 is composed of DJI M300 RTK and Zenmuse P1 image sensor. The image sensor is fixed on the UAV, and the relevant aerial survey parameters of the UAV are set, including altitude h = 60m, speed v = 3m / s, heading overlap rate of 80%, and lateral overlap rate of 70%.

4. The method for estimating the three-dimensional green volume of regional vegetation based on reconstructed point cloud according to claim 1, characterized in that: In step 1, 3-4 image control points are set in the area to be measured, and their spatial position information is measured by the RTK base station; each image control point should be captured by as many oblique images as possible and should be selected at an easily identifiable corner position.

5. The method for estimating the three-dimensional green volume of regional vegetation based on reconstructed point cloud according to claim 1, characterized in that: The specific process of step 2 is: Step 2.

1. Based on the oblique image data set obtained in step 1, a scale-invariant feature transformation algorithm is used to extract feature points, and then the extracted feature points are matched to obtain feature matching points; Step 2.

2. Based on the feature matching points, the sparse point cloud is reconstructed using the motion recovery structure algorithm to obtain sparse point cloud data; Step 2.

3. Based on the sparse point cloud, a multi-view dense matching algorithm based on clustering and patch model is used to reconstruct the dense point cloud to obtain dense point cloud data; the dense point cloud data is the scene three-dimensional reconstruction point cloud data.

6. The method for estimating the three-dimensional green volume of regional vegetation based on reconstructed point cloud according to claim 1, characterized in that: In step 3, the point cloud semantic segmentation neural network model includes an encoder and a decoder, wherein the encoder is used to obtain feature extraction results, and the decoder is used to obtain vegetation point cloud data with the same amount of original point cloud data; The encoder consists of a sampling layer and a combination layer. The sampling layer represents the dense point cloud data into a high-dimensional space composed of coordinate values ​​and eigenvalues. The farthest point sampling method is used to select the N1 point clouds with the longest spacing from the N input point clouds in the metric space, and this group of point clouds is used as a group of regional center points. In the combination layer, the nearest k points of each regional center point are selected to represent the region and obtain the feature extraction result. The decoder consists of several groups of decoding units, each of which consists of an insertion layer and a PointNet unit layer. After the encoder completes feature extraction, the model will access the PointNet layer as a feature extractor to aggregate the features of k neighbors through trainable weights. The insertion layer first completes the point cloud upsampling through the distance weighted features of the k nearest neighbor points, as shown in formula (1): Among them, d(x,x i ) is the Euclidean distance from point x to the i-th neighbor point, P represents the power of the Euclidean distance, f i (j) represents the i-th neighbor point of the j-th layer, a total of k, f (j) (x) represents the output of point x in the jth layer, and C is the total number of layers.

7. The method for estimating the three-dimensional green volume of regional vegetation based on reconstructed point cloud according to claim 1, characterized in that: The specific process of step 3 is as follows: Step 3.

1. Screen the 3D reconstructed point cloud data and remove the data with poor reconstruction quality. The data with poor reconstruction quality refers to the vegetation point cloud data with incomplete canopy morphology due to the lack of photogrammetry perspective, especially the vegetation point cloud data with large-scale holes on the lower surface of the canopy; Step 3.

2. Cut the obtained data into blocks and manually annotate each block with semantic labels. The vegetation information of interest in this study is marked as one attribute class, and other ground objects are marked as another class; Step 3.

3. Divide the labeled data into a training set and a test set according to a certain ratio; the training set data is used for training the point cloud semantic segmentation neural network model parameters, and the test set is used to detect the actual effect of the network after training and obtain the network model parameters; Step 3.

4. Based on the trained point cloud semantic segmentation neural network model, input the scene 3D reconstructed point cloud data to obtain vegetation point cloud data.

Images