Method and system for measuring diameter at any height of standing tree

Abstract

The present application relates to a kind of arbitrary height diameter measurement method of standing tree, comprising: in augmented reality scene, user clicks the arbitrary height point of the standing tree to be measured, obtains current camera image and corresponding pixel coordinates;The left and right edge segmentation point of the standing tree to be measured is separated by carrying out semantic segmentation to current image, and the depth information of each pixel in the standing tree region to be measured is obtained;Tangent constraint loss function describing standing tree cylindrical surface is constructed, and space point surface constraint loss function is constructed;The direction vector of a point on standing tree cylindrical center line and its preliminary estimation;Switch the observation angle of the same standing tree to be measured, to obtain multiple sets of geometric parameter estimation value;The diameter value of the standing tree of the height selected by user is finally estimated;Measurement result is superimposed in real time in augmented reality scene.The present application also relates to a kind of standing tree arbitrary height diameter measurement system.The present application can complete measurement directly in field site and judge data reliability in real time through artificial interaction, and it has both convenience and accuracy simultaneously.

Description

Technical Field

[0001] This invention relates to a method and system for measuring the diameter of a standing tree at any height. Background Technology

[0002] High-precision data on the diameter of standing trees at any height (e.g., diameter at breast height, ground diameter) is crucial for accurately estimating forest stock volume, carbon sequestration capacity, and biomass. Traditionally, obtaining such data has relied primarily on methods such as felling measurements or total station measurements. These methods are not only cumbersome and damaging to trees but also difficult to implement. In recent years, lidar technology has been increasingly applied to forest plot surveys. However, static lidar scanners are typically bulky, have slow data processing, and are easily affected by forest canopy obstruction, making it difficult to obtain complete standing tree point clouds. While mobile lidar partially solves the obstruction problem through motion scanning, its data accuracy heavily depends on point cloud registration, and its high equipment cost limits its large-scale application. In contrast, lightweight measurement solutions based on smartphones have gained widespread attention due to their low cost and portability. In particular, augmented reality (AR) technology combined with visual SLAM enables smartphones to become a field tool with real-time measurement and visual verification capabilities.

[0003] Augmented reality (AR) is a technology that overlays virtual information onto real-world scenes, with its core technology being Simultaneous Localization and Mapping (SLAM). SLAM technology relies solely on observation and motion sensors to achieve relative positioning and environmental map construction, without requiring Global Navigation Satellite System (GNSS) signal support. In the field of forest surveys, SLAM-based AR technology exhibits unique advantages: First, it can directly estimate and visualize tree measurement factors in the real-world scene, achieving "what you see is what you get" measurement and result synchronization; second, by fusing measurement results into the real-world view in real time, it facilitates intuitive "supervision" of the estimation results by operators, effectively identifying and avoiding gross errors. Currently, portable devices such as smartphones generally possess high-performance AR capabilities. Compared to traditional forest survey methods, this technology offers the advantages of portability, efficiency, and high precision; compared to other remote sensing technologies, it features real-time performance, reliability, and on-site verification. Therefore, AR technology, especially in forest plot surveys requiring precise on-site verification, is playing an increasingly important role.

[0004] Traditional tree measurement methods mainly include felling measurement, total station optical measurement, and various specialized tree measuring tools. Falling measurement, as the most primitive method, involves directly felling trees for measurement. It offers the highest accuracy and is often considered the benchmark, but it is completely destructive and unsuitable for monitoring living trees. Total station measurement represents a significant advancement. Utilizing non-cooperative target ranging and side-measuring capabilities, it achieves non-contact, high-precision (centimeter-level) measurement of the diameter of standing trees at any height, offering high efficiency. However, its application is limited by expensive equipment, the need for specialized operation, and visibility between trees. Furthermore, various altimeters or upper diameter gauges (such as the Bruegel altimeter) rely on indirect calculations based on geometric principles such as trigonometry. While lightweight and low-cost, their accuracy is limited, and they cannot directly obtain the diameter at height. Overall, traditional methods have laid a foundation in accuracy and reliability, but generally suffer from limitations in efficiency, cost, or requirements regarding trees and the environment.

[0005] LiDAR technology revolutionizes standing tree surveying by actively emitting lasers and receiving echoes to acquire high-density 3D point cloud data of targets. The core process involves scanning to acquire the point cloud, followed by denoising, registration, and segmentation to extract individual tree point clouds. Then, algorithms such as cylinder fitting or circle fitting are used to invert the diameter of the trunk at any height. Depending on the platform, it is mainly divided into ground-based LiDAR and mobile LiDAR (such as vehicle-mounted and handheld). Ground-based LiDAR scans at fixed points, achieving the highest point cloud quality with accuracy ranging from millimeters to centimeters. Mobile LiDAR significantly improves scanning efficiency through platform movement, making it suitable for large-scale surveys. However, this advanced technology comes with significant challenges: equipment and processing costs are extremely high, data processing is complex, and both static and dynamic scanning are susceptible to shading by branches and leaves, leading to missing point clouds in the upper part of the trunk, often requiring inference and repair using algorithmic models. While LiDAR represents the forefront of high-precision, digital, and 3D measurement, its cost and complexity limit its widespread adoption in routine surveys. Furthermore, data processing methods often rely on post-processing, making it impossible to obtain results on-site and monitor the accuracy of the measurements.

[0006] Overall, the shortcomings of the existing solutions include: It is impossible to measure the diameter of a standing tree at any height on-site. The measurement results cannot be visualized and monitored on-site.

[0007] Specifically, while traditional felling measurement and total station methods can obtain relatively accurate data, the former is destructive, requiring the felling of trees to measure diameters at any height, while the latter, although allowing measurements on visible parts of the tree trunk, involves bulky equipment, complex operation, and requires specialized technicians to set up a control network. Furthermore, measurements can only be taken through instrument readings, making it impossible to visually monitor the location of measurement points and the reliability of the results on-site. LiDAR solutions (including static terrestrial 3D laser scanners and mobile LiDAR) can acquire high-precision 3D point cloud data to extract diameters at any height, but their equipment is expensive and inconvenient to carry. Data processing heavily relies on post-processing software, making it impossible to generate measurement results in real-time in the field and perform interactive visualization verification. Close-range photogrammetry and structure-of-motion (SfM) methods require the establishment of control points in the sample plot, are highly dependent on tree trunk texture, and have lengthy data processing flows, similarly failing to achieve real-time on-site measurement and visualization verification. While emerging augmented reality (AR) measurement applications (such as ForestScanner) offer portability and some visualization capabilities, their functionality is limited to diameter at breast height (DBH) measurement. They cannot meet the on-site measurement needs for the diameter of standing trees at any height. Furthermore, their visualization interface only provides numerical feedback and lacks the ability to monitor the real-time positioning and fitting process of the tree trunk cross-section.

[0008] In summary, none of the existing technical solutions have been able to simultaneously solve the two core problems of "completing on-site measurement of the diameter of standing trees at any height" and "on-site visual monitoring of measurement results," resulting in a lack of a standing tree diameter measurement method that combines high precision, non-destructiveness, portability, and real-time interactivity in forest resource surveys. Summary of the Invention

[0009] In view of this, it is necessary to provide a method and system for measuring the diameter of standing trees at any height, which can be directly measured in the field and the reliability of the data can be judged in real time through human interaction, while being both convenient and accurate.

[0010] This invention provides a method for measuring the diameter of a standing tree at any height. The method includes: Step S1, in an augmented reality scene, the user clicks on any height point of the standing tree to be measured via touch interaction, simultaneously acquiring the current camera image and the corresponding pixel coordinates; Step S2, using a SAM model to perform semantic segmentation on the current image, separating the left and right edge segmentation points of the standing tree based on the mean and covariance matrix of the pixel coordinates, and combining the real-time acquired depth map to obtain the depth information of each pixel within the area of ​​the standing tree; Step S3, constructing tangent constraint loss functions describing the cylindrical surface of the standing tree using the left and right edge point sets respectively, and combining the depth of each pixel... The information constructs a spatial point-surface constraint loss function; by optimizing and solving the above two loss functions, a point on the center line of the standing tree cylinder and its direction vector are initially estimated; in step S4, the observation perspective of the same standing tree to be measured is switched, and steps S1 to S3 are repeated to obtain multiple sets of geometric parameter estimates; by combining parameters including semantic segmentation boundaries, optimized standing tree center line points and direction vectors, the diameter value of the standing tree at any height selected by the user is finally estimated through geometric projection and calculation; in step S5, the measurement results including the diameter estimate and the standing tree center line are superimposed and displayed in real time in the augmented reality scene for the user to perform intuitive verification and manual supervision.

[0011] Step S1 includes: First, real-time camera pose estimation and sparse point cloud map construction are completed through monocular SLAM; then, based on keyframe and depth estimation mechanisms, dense 3D reconstruction is achieved through multi-view stereo vision (MVS) technology, providing high-precision geometric data support for the positioning of cross-sections and diameter fitting of standing trees.

[0012] The step of performing real-time camera pose estimation and sparse point cloud map construction using monocular SLAM includes: First, feature points are extracted from the image. Observation equations are constructed through feature matching and epipolar geometry constraints, and motion equations are constructed using IMU angular velocity and angular velocity data. The mobile device state information is estimated by jointly solving the equations. Simultaneously, a local map is maintained, and the matched feature points are projected into three-dimensional space using triangulation to generate a sparse three-dimensional point cloud map. Backend optimization is used to globally optimize the pose and map points of historical frames to reduce accumulated errors, and loop closure detection technology is used to eliminate drift, ultimately achieving stable real-time positioning and sparse environment reconstruction.

[0013] The method of using multi-view stereo vision (MVS) technology to achieve dense 3D reconstruction, providing high-precision geometric data support for the positioning of cross-sections and diameter fitting of standing trees, includes: First, for each keyframe, the depth map at that viewpoint is calculated pixel by pixel using stereo matching of adjacent keyframes, through photometric consistency cost calculation and semi-global matching or a deep learning-based depth estimation algorithm. Then, erroneous depth values ​​are eliminated through consistency checks and reprojection error filtering. Finally, the effective depth points of each keyframe are fused into a unified global coordinate system to generate a dense 3D point cloud model covering the surface of the standing tree trunk.

[0014] Step S2 includes: SLAM output position with posture Current image and multi-view stereo vision to obtain depth map As input, an augmented reality scene is constructed. Through user interaction, clicking on the tree trunk to be tested within the augmented reality scene obtains the coordinates of the previous pixel in the tree trunk image. The process yields the following dataset: , in, The total number of observations of the standing trees to be measured. Not less than 1 , For the first The images and depth maps obtained from imaging the standing tree to be tested. , for The rotation matrix and translation vector of the image in the world coordinate system during imaging. For the first Secondary imaging is achieved through augmented reality scene acquisition. The coordinates of one pixel on the tree trunk to be tested.

[0015] Step S3 includes: First, based on single-frame data processing, the estimated values ​​of the tree position and normal vector are obtained. Then, data fusion is performed to obtain a globally consistent estimate. It is assumed that the tree trunk grows in the same direction at any height and the diameter of the tree at different heights is different, but it basically conforms to the cylindrical shape. Based on this assumption, fitting is performed to obtain the estimated values ​​of the tree position and normal vector, thereby further obtaining the diameter at any height.

[0016] Step S4 includes: Given the height to be measured Then, project the height point on the axis onto the first... The frame image space coordinate system is then projected onto the image plane coordinate system; based on the coordinates of the projected pixels, the set of left edge pixels, the set of right edge pixels, and the set of pixels with depth are indexed near that height; a loss function is then constructed to achieve [the desired effect]. The height and diameter are estimated.

[0017] Step S5 includes: After completing the arbitrary height diameter estimation, the necessary view matrix and projection matrix are calculated based on the current pose, and the arbitrary height diameter is expressed in the augmented reality scene to supervise the accuracy of the measurement results.

[0018] This invention provides a system for measuring the diameter of a standing tree at any height. The system includes an acquisition module, a segmentation module, a vector estimation module, a diameter estimation module, and an augmented reality construction module. Specifically: the acquisition module allows a user to interact with the tree in an augmented reality scene by clicking on any height point of the standing tree, simultaneously acquiring the current camera image and its corresponding pixel coordinates; the segmentation module performs semantic segmentation on the current image using a SAM model, separating the left and right edge segmentation points of the standing tree based on the mean and covariance matrix of the pixel coordinates, and combining this with a real-time depth map to obtain the depth information of each pixel within the area of ​​the standing tree; the vector estimation module constructs tangents describing the cylindrical surface of the standing tree using the left and right edge point sets respectively. The system employs a constraint loss function and a spatial point-plane constraint loss function, incorporating depth information from each pixel. By optimizing and solving these two loss functions, a point on the centerline of the standing tree cylinder and its direction vector are initially estimated. The diameter estimation module switches the observation perspective of the same standing tree under test, repeating steps S1 to S3 to obtain multiple sets of geometric parameter estimates. By integrating parameters including semantic segmentation boundaries, the optimized centerline point of the standing tree, and its direction vector, and through geometric projection and calculation, the diameter of the standing tree at any height selected by the user is finally estimated. The augmented display construction module overlays the measurement results, including the diameter estimate and the centerline of the standing tree, onto the augmented reality scene in real time, allowing users to perform intuitive verification and manual supervision.

[0019] This application leverages the powerful zero-shot segmentation capability of the SAM model to identify standing tree outlines in real time on mobile devices. Combining AR interaction with depth information, an optimized algorithm fits a cylindrical model of the standing tree, ultimately calculating the diameter at a user-specified height. This result can be visualized and manually verified within an AR scene. This application simultaneously addresses two core issues: "completing on-site diameter measurement of standing trees at arbitrary heights" and "on-site visual monitoring of measurement results." Furthermore, it provides a novel solution for forest resource surveys and monitoring that is high-precision, non-destructive, portable, and offers real-time interactive capabilities. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the basic configuration of the augmented reality device according to an embodiment of the present invention; Figure 2 This is a flowchart of the method for measuring the diameter of a standing tree at any height according to the present invention; Figure 3 This is a schematic diagram of the 3D reconstruction process of monocular SLAM and multi-view stereo vision fusion according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the interactive interface of an embodiment of the present invention, showing a user clicking on a point on a tree trunk in an augmented reality scene; Figure 5 This is a schematic diagram of the SAM semantic segmentation model processing flow according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the processing results of semantic segmentation and depth acquisition at each stage in an embodiment of the present invention. (a) is the semantic segmentation result (yellow area in the figure), (b) is the depth map mapping result, (c) is the edge extraction and left and right edge segmentation result (red and green lines in the figure are left and right edges, red is the arithmetic mean point of edge points, blue arrow is the direction of the main axis of the trunk obtained by covariance matrix decomposition), and (d) is the segmentation result. Figure 7 This is a schematic diagram of the tree position and normal vector estimation process according to an embodiment of the present invention; Figure 8 This is a schematic diagram of the arbitrary height diameter estimation process according to an embodiment of the present invention; Figure 9 This is a hardware architecture diagram of the standing tree arbitrary height and diameter measurement system of the present invention. Detailed Implementation

[0021] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0022] This embodiment adopts the basic configuration of an augmented reality device as follows: Figure 1 As shown, the front of the device is a display screen used to build augmented reality scenes; the back houses an augmented reality camera for observing the scene; the IMU (Integrated Measurement Unit) acquires acceleration and angular velocity data during the device's dynamic processes; and the CPU and GPU process the IMU and camera data in real time, completing arbitrary height diameter measurements and augmented reality scene construction. The core technology of augmented reality is real-time SLAM (Simultaneous Localization and Mapping), whose real-time localization and mapping data can be used not only for building augmented reality scenes but also as a data source for arbitrary height measurements.

[0023] See Figure 2 The diagram shown is a flowchart of a preferred embodiment of the method for measuring the diameter of an arbitrary height of a standing tree according to the present invention.

[0024] Step S1: Monocular SLAM and Multi-View Stereo Vision. That is, in an augmented reality scenario, the user interacts with the device by clicking on any height point of the standing tree under test, simultaneously acquiring the current camera image and its corresponding pixel coordinates. Please refer to [link / reference needed]. Figure 3 Specifically: Monocular SLAM (Simultaneous Localization and Mapping) and Multi-View Stereo Vision (MVS) are the core technologies underlying this invention for achieving 3D perception of standing trees and measurement of diameter at arbitrary heights. The technical process can be summarized as follows: First, monocular SLAM is used to complete real-time camera pose estimation and sparse point cloud map construction; then, based on keyframes and depth estimation mechanisms, MVS technology is used to achieve dense 3D reconstruction, providing high-precision geometric data support for cross-sectional positioning and diameter fitting of standing trees.

[0025] Furthermore, in the monocular SLAM stage, image sequences of the standing tree and its surrounding environment are continuously acquired using a monocular camera on a mobile device (such as a smartphone or AR glasses), and angular velocity sequences are obtained using inertial navigation. First, feature points such as ORBs are extracted from the images. Observation equations are constructed through feature matching and epipolar geometry constraints, and motion equations are constructed using IMU angular velocity data. Based on the joint solution of these equations, the position, rotation, velocity, and other state information of the mobile device are estimated. Simultaneously, a local map is maintained, and the matched feature points are projected into 3D space using triangulation methods to generate a sparse 3D point cloud map. Backend optimization (such as bundle adjustment) is used to globally optimize the pose and map points of historical frames, reducing accumulated errors. Loop closure detection technology is used to eliminate drift, ultimately achieving stable real-time localization and sparse environment reconstruction.

[0026] Building upon this foundation, to obtain complete geometric information of the standing tree surface to meet the requirements of arbitrary height and diameter measurement, multi-view stereo vision (MVS) technology is introduced for dense reconstruction. This process uses keyframes output from SLAM and their corresponding high-precision camera poses as input, employing an MVS method based on depth map fusion: First, for each keyframe, using stereo matching of adjacent keyframes, a depth map is calculated pixel-by-pixel from that viewpoint through photometric consistency cost calculation and semi-global matching (SGM) or a deep learning-based depth estimation algorithm; then, erroneous depth values ​​are eliminated through consistency checks and reprojection error filtering; finally, the effective depth points of each keyframe are fused into a unified global coordinate system to generate a dense 3D point cloud model covering the surface of the standing tree trunk. This dense point cloud can accurately depict the actual morphology of the tree trunk cross-section (such as irregular contours, local protrusions, etc.), providing a high-fidelity 3D geometric foundation for subsequent trunk region extraction based on semantic segmentation models, arbitrary height cross-section localization, and accurate diameter fitting.

[0027] Step S2, Edge Segmentation and Depth Acquisition. That is, the SAM model is used to perform semantic segmentation on the current image. Based on the mean and covariance matrix of the pixel coordinates of the tree under test, the left and right edge segmentation points of the tree are separated. Combined with the real-time acquired depth map, the depth information of each pixel within the area of ​​the tree under test is obtained. Specifically, this includes: The following steps in this embodiment are based on the SLAM output position. with posture Current image and multi-view stereo vision to obtain depth map As input, an augmented reality scene is first constructed, and the user interacts by clicking on the tree trunk to be tested within the augmented reality scene. Figure 4 ), to obtain the coordinates of a pixel in the tree trunk image. The process yields the following dataset: , in, This represents the total number of observations of the standing trees to be measured (this value is not less than 1). , For the first The images and depth maps obtained from imaging the standing tree to be tested. , for The rotation matrix and translation vector of the image in the world coordinate system during imaging. For the first Secondary imaging is achieved through augmented reality scene acquisition. The coordinates of one pixel on the tree trunk to be tested.

[0028] This embodiment uses the Segment Anything (SAM) model to segment the flower imaging region. SAM is a landmark work published by Meta AI in 2023. Its core breakthrough lies in upgrading the segmentation task from "specific category recognition" to "general visual understanding," achieving "what you see is what you get" segmentation interaction through cue engineering. It is considered an important step in the evolution of computer vision towards a foundational model. The segmentation process is as follows: Figure 5 As shown, the SAM model uses a large image encoder to generate an image embedding representation, and then performs efficient queries through multiple input prompts to generate a target mask (the set of all pixel coordinates of the target in the image) in real time. Figure 4 The image shows the segmentation results of the tree trunk under test based on the SAM model. Figure 6 As shown in Figure a. middle After processing the SAM model, the resulting dataset is: , in, for The set of pixel coordinates of the tree trunk to be tested.

[0029] Let the depth image be during the imaging process Compared with reference image The mapping relationship is For depth images Each effective depth point Its corresponding depth value Using mapping relationships The coordinates of this point in the reference image coordinate system are calculated as follows: , Will Coordinate rounding ( After the operation, if ,but A point on the tree trunk at a depth of This point constitutes the dataset. ( Figure 6 (b) The dataset after this step is: , The tree trunk edge extraction algorithm takes the semantic segmentation result of the tree trunk as input and is implemented based on the digital binary image topology analysis method proposed by Suzuki and Abe in 1985. First, the algorithm scans the input binary image line by line, identifying boundary starting points by detecting changes in pixel values. Specifically, when a transition from a zero pixel to a non-zero pixel occurs during the scan, it is determined as the starting point of the outer boundary; when a transition from a non-zero pixel to a zero pixel occurs, it is determined as the starting point of the hole boundary. Second, starting from each boundary starting point, the algorithm traces adjacent boundary pixels along the boundary direction, forming a closed boundary path. During the tracing process, the algorithm assigns a unique identifier to each newly discovered boundary to distinguish different boundaries and record their containment relationships. Finally, the algorithm establishes a hierarchical structure based on the surrounding relationships between boundaries: outer boundaries correspond to connected regions with a pixel value of 1, and hole boundaries correspond to regions with a pixel value of 0. Through this encoding method, the algorithm can completely represent the topology of all connected regions and their internal holes in the binary image. The result after this processing is: , in, for The set of pixel coordinates of the tree trunk edge to be tested.

[0030] Tree trunk edge pixel set The Middle The pixel coordinates of the point are The mean point of this set of points have: , , in for The number of pixels in the set. The covariance matrix of this point set is: , right Perform eigenvalue decomposition to obtain two eigenvalues. , ( ) and the corresponding feature vectors , .in, It is the main feature vector, pointing in the direction of maximum variance of the point set, corresponding to the main axis direction of the trunk (i.e. the extension direction of the trunk). This is the secondary eigenvector, perpendicular to the trunk direction, pointing horizontally (widthwise) towards the trunk. Eigenvalues and These represent the degree of dispersion along the primary and secondary directions, respectively. Based on the main axis of the trunk... and horizontal Define the left and right side division rules for each edge point. have: , like If so, then the edge point is determined to be located on the right side of the tree trunk; if If so, then the edge point is determined to be located on the left side of the tree trunk. Figure 6 (c) After this processing, the dataset is: , in, , These are the sets of pixels on the left and right edges of the tree trunk, respectively. Figure 6 (d).

[0031] Step S3, Estimation of the standing tree position and normal vector: That is, constructing tangent constraint loss functions describing the surface of the standing tree cylinder using the left and right edge point sets respectively, and constructing a spatial point-surface constraint loss function by combining the depth information of each pixel; by optimizing and solving the above two loss functions, a point on the center line of the standing tree cylinder and its direction vector are initially estimated. Specifically: To avoid the impact of SLAM pose drift on the consistency of multiple data acquisitions, we first process single-frame data to obtain estimated values ​​for the tree position and normal vector, and then perform data fusion to obtain a globally consistent estimate. This embodiment assumes that the tree trunk grows in the same direction at any height, and that the diameter of trees at different heights varies, but generally conforms to a cylindrical shape. Based on this assumption, we construct an equation using the above dataset, fit it, and obtain estimated values ​​for the tree position and normal vector, thereby further obtaining the diameter at any height. Specifically, as follows... Figure 7 As shown.

[0032] In single-frame data processing, assume the cylinder's axis lies in a plane: the origin of the image space coordinate system is a point in this plane, and the normal vector of this plane is: , in This is the camera intrinsic parameter matrix. From this, we obtain the... Two points on the frame axis are: , , in, , For pixels , The depth.

[0033] With the The description of the nearest left and right boundary points of a depth pixel in a frame is as follows: , , in Due to the left boundary point Right boundary point and If the three points are at relatively similar heights, then the cylindrical tree trunks at those three points are assumed to have the same diameter. Furthermore, if we assume the diameter at that height is... Then the left and right side points , As the point of tangency exists, the equation is: , , Based on this equation, the tangent loss function can be constructed; the equation for the existence of pixels with depth is: , Based on this, a point loss function for the cylinder surface is constructed. By constructing the loss function using all depth pixels, the value of two points on the cylinder's axis can be estimated. , depth , Thus, the coordinates of two points on the axis are obtained. , This forms a dataset: .

[0034] In SLAM pose estimation, rotational drift is relatively small, therefore the normal vector of the standing tree axis is directly calculated based on the average of multiple frames: .

[0035] To calculate the location of the standing tree, the center position of the tree's diameter at ground level is first obtained from different frames of data. Specifically, the closest points on the axis of the standing tree's cylinder and its left and right edge pixels in each frame that form a straight line with the origin of the image space coordinate system are transformed to the world coordinate system. The point on the axis with the smallest Z-axis value is taken as the location point of the standing tree. The position of the frame is: , in , , , For dataset Number of elements. Due to significant pose drift in visual SLAM within a forest, the position of the standing tree in the first frame is taken as the tree's position, i.e., the tree's position is: , Based on the aforementioned calculations of the standing tree positions in other frames, pose correction and multi-frame data fusion are achieved, where the first... The frame offset correction is as follows: , This results in the following dataset: , This embodiment uses this dataset to estimate the diameter at arbitrary heights.

[0036] Step S4: Multi-view data fusion and arbitrary height diameter estimation. That is, switching the observation perspective of the same standing tree and repeating steps S1 to S3 to obtain multiple sets of geometric parameter estimates, improving the robustness and accuracy of subsequent calculations. Combining semantic segmentation boundaries, optimized tree centerline points, and direction vectors, the diameter of the standing tree at any height selected by the user is finally estimated through geometric projection and calculation. Specifically: The arbitrary height diameter estimation process in this embodiment is as follows: Figure 8 As shown. At a given height to be measured... Then, project the height point on the axis onto the first... Frame space coordinate system: , Then project it onto the image plane coordinate system: , Based on the coordinates of the projected pixels, the set of pixels on the left edge, the set of pixels on the right edge, and the set of pixels with depth are indexed near this height: , , , If the diameter of this height is Then the three sets of points have an equation: , , , in, , , Based on this equation, a loss function can be constructed to achieve [the desired outcome]. The height and diameter are estimated.

[0037] Step S5, Augmented Reality Scene Construction. This involves overlaying and displaying measurement results such as the estimated diameter and the centerline of the standing tree in real-time within the augmented reality scene, allowing users to visually verify and provide manual supervision. Specifically: After completing the arbitrary height diameter estimation, the necessary view matrix and projection matrix are calculated based on the current pose, and the arbitrary height diameter is expressed in the augmented reality scene to supervise the accuracy of the measurement results.

[0038] See Figure 9 The diagram shown is a hardware architecture diagram of the standing tree arbitrary height diameter measurement system 10 of the present invention. The system includes: an acquisition module 101, a segmentation module 102, a vector estimation module 103, a diameter value estimation module 104, and an enhanced display construction module 105.

[0039] The acquisition module 101 is used to acquire the current camera image and the corresponding pixel coordinates simultaneously when the user clicks on any height point of the standing tree to be measured through touch interaction in an augmented reality scene.

[0040] The segmentation module 102 is used to perform semantic segmentation on the current image using the SAM model. Based on the mean and covariance matrix of the pixel coordinates of the tree to be tested, the left and right edge segmentation points of the tree to be tested are separated. Combined with the depth map acquired in real time, the depth information of each pixel in the area of ​​the tree to be tested is obtained.

[0041] The vector estimation module 103 is used to construct tangent constraint loss functions describing the surface of the standing cylinder using the left and right edge point sets respectively, and to construct spatial point-surface constraint loss functions by combining the depth information of each pixel; by optimizing and solving the above two loss functions, a point on the center line of the standing cylinder and its direction vector are initially estimated.

[0042] The diameter estimation module 104 is used to switch the observation perspective of the same standing tree under test and repeat steps S1 to S3 to obtain multiple sets of geometric parameter estimates, thereby improving the robustness and accuracy of subsequent calculations. By combining parameters such as semantic segmentation boundaries, optimized tree centerline points, and direction vectors, the diameter of the standing tree at any height selected by the user is finally estimated through geometric projection and calculation.

[0043] The augmented display construction module 105 is used to overlay and display measurement results such as diameter estimation and tree centerline in real time in the augmented reality scene, so that users can perform intuitive verification and manual supervision.

[0044] Although the present invention has been described with reference to the present preferred embodiments, those skilled in the art should understand that the above preferred embodiments are only used to illustrate the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for measuring the diameter of a standing tree at any height, characterized in that, The method includes: Step S1: In the augmented reality scenario, the user clicks on any height point of the tree to be tested through touch interaction, and simultaneously obtains the current camera image and the corresponding pixel coordinates; Step S2: Use the SAM model to perform semantic segmentation on the current image. Based on the mean and covariance matrix of the pixel coordinates of the tree to be tested, separate the left and right edge segmentation points of the tree to be tested. Combine with the depth map acquired in real time, obtain the depth information of each pixel in the area of ​​the tree to be tested. Step S3: Construct tangent constraint loss functions describing the surface of the standing cylinder using the left and right edge point sets respectively, and construct spatial point-surface constraint loss functions by combining the depth information of each pixel; by optimizing and solving the two loss functions, a point on the center line of the standing cylinder and its direction vector are initially estimated. Step S4: Switch the observation perspective of the same tree to be measured, and repeat steps S1 to S3 to obtain multiple sets of geometric parameter estimates; combine the parameters including semantic segmentation boundary, optimized tree centerline point and direction vector, and estimate the tree diameter at any height selected by the user through geometric projection and calculation. Step S5 involves overlaying the measurement results, including the estimated diameter and the centerline of the standing tree, onto the augmented reality scene in real time for users to visually verify and manually supervise; wherein: Step S3 includes: First, based on single-frame data processing, the estimated values ​​of the tree position and normal vector are obtained. Then, data fusion is performed to obtain a globally consistent estimate. It is assumed that the tree trunk grows in the same direction at any height and the diameter of the tree at different heights is different, but it basically conforms to the cylindrical shape. Based on this assumption, fitting is performed to obtain the estimated values ​​of the tree position and normal vector, thereby further obtaining the diameter at any height. Step S4 includes: Given the height to be measured Then, project the height point on the axis onto the first... The frame image space coordinate system is then projected onto the image plane coordinate system; based on the coordinates of the projected pixels, the set of left edge pixels, the set of right edge pixels, and the set of pixels with depth are indexed near that height; a loss function is then constructed to achieve [the desired effect]. The height and diameter are estimated.

2. The method as described in claim 1, characterized in that, Step S1 includes: First, real-time camera pose estimation and sparse point cloud map construction are completed through monocular SLAM; then, based on keyframe and depth estimation mechanisms, dense 3D reconstruction is achieved through multi-view stereo vision (MVS) technology, providing high-precision geometric data support for the positioning of cross-sections and diameter fitting of standing trees.

3. The method as described in claim 2, characterized in that, The real-time camera pose estimation and sparse point cloud map construction using monocular SLAM includes: First, feature points are extracted from the image. Observation equations are constructed through feature matching and epipolar geometry constraints, and motion equations are constructed using IMU angular velocity and acceleration data. Based on the joint solution of the equations, the state information of the mobile device is estimated. Simultaneously, a local map is maintained, and the matched feature points are projected into three-dimensional space using triangulation methods to generate a sparse three-dimensional point cloud map. Backend optimization is used to globally optimize the pose and map points of historical frames to reduce accumulated errors, and loop closure detection technology is used to eliminate drift, ultimately achieving stable real-time positioning and sparse environment reconstruction.

4. The method as described in claim 3, characterized in that, The method of using multi-view stereo vision (MVS) technology to achieve dense 3D reconstruction, providing high-precision geometric data support for the positioning of cross-sections and diameter fitting of standing trees, includes: First, for each keyframe, the depth map at that viewpoint is calculated pixel by pixel using stereo matching of adjacent keyframes, through photometric consistency cost calculation and semi-global matching or a deep learning-based depth estimation algorithm. Then, erroneous depth values ​​are eliminated through consistency checks and reprojection error filtering. Finally, the effective depth points of each keyframe are fused into a unified global coordinate system to generate a dense 3D point cloud model covering the surface of the tree trunk.

5. The method as described in claim 4, characterized in that, Step S2 includes: Position of SLAM output with posture Current image Depth map obtained from multi-view stereo vision As input, an augmented reality scene is constructed. Through user interaction, clicking on the tree trunk to be tested within the augmented reality scene obtains the coordinates of the previous pixel in the tree trunk image. The obtained dataset is: ， in, The total number of observations of the standing trees to be measured. Not less than 1 , For the first The images and depth maps obtained from imaging the standing tree under test. , for The rotation matrix and translation vector of the image in the world coordinate system during imaging. For the first Secondary imaging is achieved through augmented reality scene acquisition. The coordinates of one pixel on the tree trunk to be tested.

6. The method as described in claim 5, characterized in that, Step S5 includes: After completing the arbitrary height diameter estimation, the necessary view matrix and projection matrix are calculated based on the current pose, and the arbitrary height diameter is expressed in the augmented reality scene to supervise the accuracy of the measurement results.

7. A system for measuring the diameter of a standing tree at any height using the method for measuring the diameter of a standing tree at any height according to claim 1, characterized in that, The system includes an acquisition module, a segmentation module, a vector estimation module, a diameter estimation module, and an enhanced display construction module, wherein: The acquisition module is used to acquire the current camera image and the corresponding pixel coordinates simultaneously when the user clicks on any height point of the standing tree to be measured through touch interaction in an augmented reality scene. The segmentation module is used to perform semantic segmentation on the current image using the SAM model. Based on the mean and covariance matrix of the pixel coordinates of the tree to be tested, the left and right edge segmentation points of the tree to be tested are separated. Combined with the depth map acquired in real time, the depth information of each pixel in the area of ​​the tree to be tested is obtained. The vector estimation module is used to construct tangent constraint loss functions describing the surface of the standing cylindrical tree using the left and right edge point sets respectively, and to construct spatial point-surface constraint loss functions by combining the depth information of each pixel; by optimizing and solving the two loss functions, a point on the center line of the standing cylindrical tree and its direction vector are initially estimated. The diameter estimation module is used to switch the observation perspective of the same tree under test and repeat steps S1 to S3 to obtain multiple sets of geometric parameter estimates. By combining parameters including semantic segmentation boundaries, optimized tree centerline points and direction vectors, the diameter of the tree at any height selected by the user is finally estimated through geometric projection and calculation. The augmented display module is used to overlay and display the measurement results, including the diameter estimate and the center line of the standing tree, in real time in the augmented reality scene, allowing users to perform intuitive verification and manual supervision.

Images