Method for generating high-fidelity tree grid model from multi-view image and related equipment

Abstract

The embodiment of the invention provides a method for generating a high-fidelity tree grid model from a multi-view image and related equipment, and the method comprises the steps: obtaining a plurality of three-dimensional Gaussian points of a target tree, and carrying out the extraction of trunk points of the plurality of three-dimensional Gaussian points, and obtaining a plurality of trunk Gaussian points; based on the shortest path length between each trunk Gaussian point and the root node, clustering the plurality of trunk Gaussian points to obtain a plurality of trunk clusters, each trunk cluster comprising a plurality of trunk Gaussian points; calculating the mass center of each trunk cluster, and carrying out refined connection processing based on a plurality of mass centers to obtain a tree skeleton; generating a tree trunk model based on the plurality of three-dimensional Gaussian points and the tree skeleton; according to the method, on the basis of the connection direction between each three-dimensional Gaussian point and the tree trunk model, crown screening is carried out on the multiple three-dimensional Gaussian points to obtain multiple crown points, and the tree model of the target tree is generated on the basis of the multiple crown points and the tree trunk model, so that the overall trueness of the generated tree model is remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a method for generating a high-fidelity tree mesh model from multi-view images and related equipment. BACKGROUND

[0002] In the fields of digital twin city, virtual geographic environment, fine management of forestry resources, and ecological simulation, the construction of a high-fidelity three-dimensional tree model is a basic and key technology. An existing method for generating a high-fidelity tree mesh model from multi-view images mainly relies on laser scanning or multi-view image reconstruction. Among them, the method based on multi-view images has attracted widespread attention due to its low device cost and flexible data acquisition. The general process is to generate a three-dimensional point cloud of the target tree from a series of two-dimensional images through Structure from Motion (SfM) and other technologies. Then, technical personnel will process the three-dimensional point cloud using various algorithms, such as through point cloud segmentation, skeleton extraction, and surface reconstruction steps, to finally generate a complete three-dimensional mesh model containing the tree trunk and crown, to meet the visualization and analysis requirements in different scenarios.

[0003] However, when generating a tree model using the existing image-based method, due to the complex geometric shape and high self-occlusion characteristics of the tree itself, the generated three-dimensional point cloud data often has problems such as noise, sparsity, and incompleteness. On the one hand, this makes it difficult for traditional algorithms to accurately distinguish between the trunk and branches when extracting the tree trunk skeleton, and it is easy to produce topological structure errors or discontinuous skeletons due to data loss, directly affecting the geometric accuracy of the trunk model. On the other hand, for the tree crown part with complex structure and severe data loss, it is almost impossible to reconstruct the internal fine branch structure completely relying on data-driven methods, resulting in a distorted shape and missing details of the generated crown model, causing the generated tree model to have low fidelity. SUMMARY

[0004] The method for generating a high-fidelity tree mesh model from multi-view images and related equipment provided by the embodiments of the present application can improve the fidelity of the generated tree model.

[0005] To achieve the above-mentioned purpose, the first aspect of the embodiments of the present application proposes a method for generating a high-fidelity tree mesh model from multi-view images, which comprises:

[0006] Obtaining a plurality of three-dimensional Gaussian points of a target tree, and extracting trunk points from the plurality of three-dimensional Gaussian points to obtain a plurality of trunk Gaussian points;

[0007] cluster the plurality of stem Gaussian points based on a shortest path length between each of the stem Gaussian points and the root node, to obtain a plurality of stem clusters, each of the stem clusters comprising a plurality of the stem Gaussian points;

[0008] calculate a centroid of each of the stem clusters, and perform a refined connection process based on a plurality of the centroids, to obtain a tree skeleton;

[0009] generate a tree stem model based on the plurality of three-dimensional Gaussian points and the tree skeleton;

[0010] perform a tree crown screening on the plurality of three-dimensional Gaussian points based on a connection direction between each of the three-dimensional Gaussian points and the tree stem model, to obtain a plurality of tree crown points, and generate a tree model of the target tree based on the plurality of tree crown points and the tree stem model.

[0011] In some embodiments, the stem point extraction on the plurality of three-dimensional Gaussian points to obtain a plurality of stem Gaussian points comprises:

[0012] determine a plurality of neighboring Gaussian points corresponding to each of the three-dimensional Gaussian points;

[0013] obtain a neighboring point difference value based on a difference between the neighboring Gaussian points and a centroid of the plurality of neighboring Gaussian points, and obtain a covariance matrix of the three-dimensional Gaussian point based on an average value of all the neighboring point difference values;

[0014] perform an eigenvalue decomposition on the covariance matrix to obtain a plurality of eigenvalues, and calculate a non-scattering degree of each of the three-dimensional Gaussian points based on the plurality of eigenvalues;

[0015] select the stem Gaussian points from the plurality of three-dimensional Gaussian points based on the non-scattering degrees of the three-dimensional Gaussian points.

[0016] In some embodiments, the selection of the stem Gaussian points from the plurality of three-dimensional Gaussian points based on the non-scattering degrees of the three-dimensional Gaussian points comprises:

[0017] select the three-dimensional Gaussian points with a non-scattering degree greater than a preset scattering degree from the plurality of three-dimensional Gaussian points as initial stem Gaussian points;

[0018] perform a clustering process on the plurality of initial stem points based on a Euclidean distance between each of the initial stem Gaussian points, to obtain a plurality of initial stem point clusters;

[0019] select an initial stem point cluster with a cluster point quantity greater than a preset cluster point quantity from the plurality of initial stem point clusters as a target stem point cluster, and take the plurality of initial stem Gaussian points in the target stem point cluster as the stem Gaussian points.

[0020] In some embodiments, the clustering of the plurality of trunk Gaussian points based on the shortest path length between each of the trunk Gaussian points and the root node comprises:

[0021] generating an undirected weighted graph based on the plurality of trunk Gaussian points, and calculating the shortest path length between each of the trunk Gaussian points and the root node based on a shortest path fast algorithm;

[0022] the clustering of the plurality of trunk Gaussian points based on the shortest path length between each of the trunk Gaussian points and the root node comprises:

[0023] In some embodiments, the refinement connection processing based on the plurality of centroids comprises:

[0024] connecting the plurality of centroids to obtain an initial tree skeleton of the target tree, the initial tree skeleton comprising a plurality of vertices, each two adjacent vertices forming a skeleton line segment;

[0025] calculating a vertex weight corresponding to each of the vertices based on a length of a sub-tree connected to the vertex;

[0026] selecting a refinement vertex from the plurality of vertices based on the vertex weight and a distance between the skeleton line segments;

[0027] obtaining the tree skeleton based on the plurality of refinement vertices.

[0028] In some embodiments, the generating of the tree trunk model based on the plurality of three-dimensional Gaussian points and the tree skeleton comprises:

[0029] generating a cylindrical trunk for each of the trunk clusters in the tree skeleton;

[0030] obtaining a distance weight of each three-dimensional Gaussian point based on a ratio between a straight-line distance of the three-dimensional Gaussian point from the cylindrical trunk and a maximum straight-line distance;

[0031] generating a distance sum objective function based on an accumulation of products of the distance weight and the straight-line distance corresponding to all the three-dimensional Gaussian points;

[0032] solving the distance sum objective function based on a least square method to obtain an optimal cylindrical radius, an optimal end point and an optimal direction vector corresponding to the trunk cluster, and updating the cylindrical trunk based on the optimal cylindrical radius, the optimal end point and the optimal direction vector;

[0033] generating the tree trunk model based on all the updated cylindrical trunks.

[0034] In some embodiments, the tree crown screening on the plurality of three-dimensional Gaussian points based on the connection direction between each of the three-dimensional Gaussian points and the tree trunk model obtains a plurality of tree crown points, including:

[0035] For each branch skeleton point in the tree trunk model, based on the branch skeleton point, the parent node of the branch skeleton point, and the branch child node of the branch skeleton point, a branch angle of each branch skeleton point is calculated;

[0036] Based on the numerical relationship between the branch angle and the preset branch threshold value, a branch angle interval is obtained.

[0037] Based on the connection direction between each of the three-dimensional Gaussian points and each of the branch skeleton points, a growth direction of each of the three-dimensional Gaussian points is calculated.

[0038] From the plurality of three-dimensional Gaussian points, the three-dimensional Gaussian points corresponding to the angle in the branch angle interval are selected as the tree crown points.

[0039] In some embodiments, the plurality of three-dimensional Gaussian points of the target tree are obtained, including:

[0040] Obtaining a plurality of multi-view images of a target scene, the target scene including the target tree;

[0041] Using three-dimensional point cloud technology, three-dimensional Gaussian extraction is performed on the plurality of multi-view images to obtain a plurality of scene Gaussian points;

[0042] Based on the low-dimensional feature of each of the scene Gaussian points, from the plurality of scene Gaussian points, the scene Gaussian points with the same low-dimensional feature as the tree feature are selected as the three-dimensional Gaussian points.

[0043] To achieve the above-mentioned purpose, a second aspect of the embodiment of the present application proposes an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the method for generating a high-fidelity tree grid model from multi-view images as described in the first aspect when executing the computer program.

[0044] To achieve the above-mentioned purpose, a third aspect of the embodiment of the present application proposes a storage medium, the storage medium is a computer readable storage medium, the storage medium stores a computer program, and the computer program is executed by a processor to implement the method for generating a high-fidelity tree grid model from multi-view images as described in the first aspect.

[0045] A method for generating a high-fidelity tree mesh model from a multi-view image and related equipment are provided in the embodiments of the present application. The method comprises the following steps: firstly, a plurality of three-dimensional Gaussian points of a target tree are obtained, and a plurality of main stem Gaussian points are obtained by performing main stem point extraction on the plurality of three-dimensional Gaussian points; then, the plurality of main stem Gaussian points are clustered based on the shortest path length between each main stem Gaussian point and a root node to obtain a plurality of main stem clusters, and each main stem cluster comprises a plurality of main stem Gaussian points; next, the centroid of each main stem cluster is calculated, and a refined connection process is performed based on the plurality of centroids to obtain a tree skeleton; subsequently, a tree stem model is generated based on the plurality of three-dimensional Gaussian points and the tree skeleton; finally, a plurality of crown points are obtained by performing crown screening on the plurality of three-dimensional Gaussian points based on the connection direction between each three-dimensional Gaussian point and the tree stem model, and a tree model of the target tree is generated based on the plurality of crown points and the tree stem model. The embodiments of the present application can overcome the noise and breakpoints in the original data by performing main stem point extraction on the three-dimensional Gaussian points and using a clustering algorithm based on the shortest path length and a skeleton refinement process, and can generate a tree stem skeleton with correct topological structure and accurate geometric shape, thereby effectively solving the technical problem of stem model distortion caused by data quality problems in the prior art. For the crown part with serious data loss, a screening strategy based on the connection direction is used. The strategy implicitly includes the morphological rules of plant growth, can intelligently screen out crown points that meet the natural growth pattern from sparse point clouds, effectively makes up for the defect that pure data-driven methods cannot reconstruct fine branch structures, and thus the finally generated complete tree model not only has clear and accurate stem structure, but also has full and realistic crown shape, thereby significantly improving the overall realism.

[0046] Other features and advantages of the present application will be set forth in the following description, and in part will be apparent from the description, or can be learned by practice of the present application. The objects and other advantages of the present application will be realized and attained by the structure particularly pointed out in the description and claims of the present application. Medical diagnosis system data processing method, system, device and storage medium BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 is a flowchart of a method for generating a high-fidelity tree mesh model from a multi-view image provided by an embodiment of the present application.

[0048] Figure 2 is Figure 1 a flowchart of step 101 in the method.

[0049] Figure 3 is a schematic diagram of a segmentation process of a three-dimensional tree Gaussian point provided by another embodiment of the present application.

[0050] Figure 4 is Figure 1Another flowchart of step 101.

[0051] Figure 5 is Figure 4 A flowchart of step 404.

[0052] Figure 6 is a schematic diagram of the trunk Gaussian point extraction provided by another embodiment of the present application.

[0053] Figure 7 is Figure 1 A flowchart of step 102.

[0054] Figure 8 is Figure 1 A flowchart of step 103.

[0055] Figure 9 is a schematic diagram of the main skeleton refinement provided by another embodiment of the present application.

[0056] Figure 10 is a schematic diagram of the generation and refinement of the main skeleton provided by another embodiment of the present application.

[0057] Figure 11 is Figure 1 A flowchart of step 104.

[0058] Figure 12 is a schematic diagram of the cylinder fitting trunk provided by an embodiment of the present application.

[0059] Figure 13 is a schematic diagram of the generation process of the tree trunk grid model provided by an embodiment of the present application.

[0060] Figure 14 is Figure 1 A flowchart of step 105.

[0061] Figure 15 is a schematic diagram of the bifurcation angle provided by an embodiment of the present application.

[0062] Figure 16 is a schematic diagram of the bifurcation angle distribution in the tree skeleton provided by an embodiment of the present application.

[0063] Figure 17 is a schematic diagram of the canopy generation fusing morphological self-similarity and spatial competition provided by an embodiment of the present application.

[0064] Figure 18 is a flowchart of the tree model generation provided by an embodiment of the present application.

[0065] Figure 19is a simulation result schematic diagram of an embodiment of the present application, which adopts a method for generating a high-fidelity tree grid model from a multi-view image.

[0066] Figure 20 is a comparative simulation result schematic diagram of a first tree model generation according to another embodiment of the present application.

[0067] Figure 21 is a simulation data schematic table according to another embodiment of the present application.

[0068] Figure 22 is a comparative simulation result schematic diagram of a second tree model generation according to another embodiment of the present application.

[0069] Figure 23 is a simulation data schematic table according to another embodiment of the present application.

[0070] Figure 24 is a schematic diagram of three-dimensional Gaussian point segmentation of multiple trees according to another embodiment of the present application.

[0071] Figure 25 is a comparative simulation result schematic diagram of different search radii according to another embodiment of the present application.

[0072] Figure 26 is a comparative simulation result schematic diagram with or without self-similarity angle constraint according to another embodiment of the present application.

[0073] Figure 27 is a hardware structure schematic diagram of an electronic device according to another embodiment of the present application. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not intended to limit the present application.

[0075] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a manner different from the module division in the device or the order in the flowchart.

[0076] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0077] In the fields of digital twin city, virtual geographic environment, fine management of forestry resources and ecological simulation, the construction of high-fidelity three-dimensional tree model is a basic and key technology. A method of generating high-fidelity tree mesh model from multi-view images mainly relies on laser scanning or multi-view image reconstruction. Among them, the method based on multi-view images is widely concerned because of its low equipment cost and flexible data acquisition. Its general process is to generate a three-dimensional point cloud of the target tree from a series of two-dimensional images through Structure from Motion (SfM) and other technologies. Then, technicians will use various algorithms to process the three-dimensional point cloud, such as through point cloud segmentation, skeleton extraction and surface reconstruction, etc. Finally, a complete three-dimensional mesh model containing the trunk and crown of the tree is generated to meet the visualization and analysis requirements in different scenarios.

[0078] However, when generating a tree model by using the existing image-based method, due to the complex geometric shape and high self-occlusion characteristics of the tree itself, the generated three-dimensional point cloud data often has problems such as noise, sparsity and incompleteness. On the one hand, this makes it difficult for traditional algorithms to accurately distinguish the trunk from the branches and leaves when extracting the trunk skeleton, and the topology structure error or discontinuous skeleton is easily caused by data loss, which directly affects the geometric accuracy of the trunk model. On the other hand, for the crown part with complex structure and serious data loss, it is almost impossible to reconstruct the internal fine branch structure by relying entirely on data-driven methods, resulting in a generated crown model with distorted shape and missing details, which leads to a situation that the generated tree model is low in reality.

[0079] In order to improve the reality of the generated tree model, the trunk points are extracted from the three-dimensional Gaussian points in the embodiments of the present application, and the clustering algorithm based on the shortest path length and the skeleton refinement processing are used, which can overcome the noise and breakpoints in the original data, generate a tree trunk skeleton with correct topology structure and accurate geometric shape, effectively solve the technical problem that the trunk model is distorted due to data quality problems in the prior art, and for the crown part with serious data loss, a filtering strategy based on the connection direction is adopted. The strategy implies the morphological law of plant growth, can intelligently filter out the crown points that meet the natural growth pattern from the sparse point cloud, effectively makes up for the defect that the fine branch structure cannot be reconstructed by the pure data-driven method, so that the finally generated complete tree model not only has clear and accurate trunk structure, but also has full and realistic crown shape, and the overall reality is significantly improved.

[0080] A method of generating a high-fidelity tree mesh model from multi-view images and related equipment provided by the present application will be further described below. First, a method of generating a high-fidelity tree mesh model from multi-view images in the embodiments of the present application is specifically described. Referring to Figure 1An optional flowchart of a method for generating a high-fidelity tree mesh model from multi-view images according to an embodiment of the present application, Figure 1 The method in the above embodiment can include, but is not limited to, steps 101-105. It should be understood that the order of steps 101-105 in the above embodiment is not limited, and the order of steps can be adjusted, or some steps can be reduced or added according to actual needs. The method for generating a high-fidelity tree mesh model from multi-view images provided by the present application can be applied to any server or intelligent terminal with processing and computing capabilities. Figure 1 The order of steps 101-105 in the above embodiment is not limited, and the order of steps can be adjusted, or some steps can be reduced or added according to actual needs. The method for generating a high-fidelity tree mesh model from multi-view images provided by the present application can be applied to any server or intelligent terminal with processing and computing capabilities.

[0081] Step 101: Obtain a plurality of three-dimensional Gaussian points of the target tree, and perform trunk point extraction on the plurality of three-dimensional Gaussian points to obtain a plurality of trunk Gaussian points.

[0082] Step 101 is described in detail below.

[0083] In some embodiments, in response to a request for generating a tree model of a target tree, data acquisition and preliminary segmentation are first performed. Specifically, when executed, a plurality of three-dimensional Gaussian points of the target tree are first reconstructed from multi-view images. The three-dimensional Gaussian point is a more abundant three-dimensional scene representation than the traditional three-dimensional point. It not only contains position information, but also describes the shape, size and opacity of the point through parameters such as covariance matrix, which can more finely express the geometric and texture features of the target tree. Subsequently, in order to separate the clear structure of the trunk from the complex morphology of the branches and leaves, the processor will perform trunk point extraction on all three-dimensional Gaussian points. This extraction process usually judges based on the geometric properties (such as linearity or non-scattering) of each Gaussian point, so as to select a plurality of trunk Gaussian points that best represent the trunk structure of the tree, thereby laying a high-quality data foundation for subsequent skeleton construction.

[0084] First, how to obtain a plurality of three-dimensional Gaussian points of the target tree is described below.

[0085] Referring to Figure 2 , a plurality of three-dimensional Gaussian points of the target tree are obtained, including steps 201-203.

[0086] Step 201: Obtain multi-view images of the target scene.

[0087] Step 202: Use three-dimensional point cloud technology to perform three-dimensional Gaussian extraction on the multi-view images to obtain a plurality of scene Gaussian points.

[0088] Step 203: Based on the low-dimensional features of each scene Gaussian point, select the scene Gaussian points consistent with the low-dimensional features and tree features from the plurality of scene Gaussian points as three-dimensional Gaussian points.

[0089] Steps 201-203 are described in detail below.

[0090] In some embodiments, a novel deep learning-based Segment Any 3D Gaussians (SAGA) segmentation framework is used to achieve multi-granularity 3D Gaussian segmentation. Figure 3 , which is a schematic diagram of a 3D tree Gaussian point segmentation process provided in an embodiment of the present application. First, a set of multi-view images of a target scene containing trees is input. Then, a sparse point cloud is estimated from the set of images using relevant 3D point cloud techniques, such as the Structure from Motion (SfM) method. Then, the adaptively optimized density of 3D Gaussians is used to compactly represent the fine structure of the 3D scene.

[0091] For the previously pre-trained 3D Gaussian model and training set, the Segment Anything Model (SAM) encoder is used to extract the 2D features of each image and the corresponding multi-granularity mask, and then the mask correspondence and feature Figure 1 The loss of feature correspondence is used to train the low-dimensional features of each 3D Gaussian to obtain cross-multi-view Figure 1 In the segmentation stage, a set of queries are generated based on the input prompt and box selection, and then these queries are matched with the learned multi-granularity features to segment the 3D Gaussian points of the corresponding 3D trees.

[0092] Based on the above process for extracting 3D Gaussian points from trees, in practical applications, the first step is to acquire multi-view images of the target scene. These images are a series of 2D images of the same scene containing the target tree, taken from different angles and positions. This acquisition method aims to comprehensively capture the tree's morphology, texture, and spatial relationships, providing comprehensive and comprehensive visual information for subsequent 3D reconstruction. For example, this step can be accomplished by using a drone to fly around the tree or a handheld device to walk around the tree to capture the image, ensuring data integrity and diversity.

[0093] Next, the acquired multi-view images are processed using a three-dimensional point cloud technology (such as a Structure from Motion (SfM) method) to perform three-dimensional Gaussian extraction, thereby obtaining a plurality of scene Gaussian points representing the entire scene including the target tree and the background, ground, etc. The three-dimensional Gaussian extraction is a reconstruction technology, and the scene Gaussian points generated thereby not only record the spatial position, but also describe the three-dimensional shape (such as ellipsoidal shape), color and opacity of the point through parameters such as covariance matrix, and can represent the scene more delicately and realistically than traditional point clouds.

[0094] Then, based on the low-dimensional features of each scene Gaussian point, the plurality of scene Gaussian points are screened. The low-dimensional features are a compact data representation learned by a deep learning model, which can highly summarize and distinguish the semantic information of objects. For example, it can capture essential attributes such as the texture of tree bark, the color and shape of leaves, etc. The processor compares the low-dimensional features of each scene Gaussian point with the pre-defined or learned standard tree features, and only selects those scene Gaussian points with consistent features. In this way, automatic and high-precision segmentation of the tree can be achieved, and the final output of the three-dimensional Gaussian points is high-quality three-dimensional data that only belongs to the target tree.

[0095] As shown in FIG. 1, Figure 3 The figure shows how to generate high-quality three-dimensional Gaussian points containing only the target tree from the input multi-view images. The process includes two parallel processing paths: the lower path generates a three-dimensional Gaussian rendering result of the entire scene including the background through Structure from Motion (SfM) and three-dimensional Gaussian splatting technology (3D Gaussian Splatting); the upper path uses an advanced segmentation model (SAM) to perform accurate two-dimensional segmentation on one of the images. The key is to "distill" the semantic information of two-dimensional segmentation into three-dimensional Gaussian representation through a "feature consistency" constraint, so that the final output of "segmented 3D Gaussian points" can be accurately stripped from the complex scene, thereby providing pure input data for subsequent processing.

[0096] By performing the above steps 201 to 203, the three-dimensional Gaussian points with stronger expression ability are used as the data basis, the fidelity of subsequent processing is improved, and through an automatic segmentation method based on low-dimensional features, the target tree can be intelligently and accurately stripped from the complex background environment, avoiding the interference of background objects on tree modeling in traditional methods, and obtaining high-purity input data without a large amount of manual preprocessing, thereby laying a solid and reliable foundation for subsequent accurate trunk skeleton extraction and high-fidelity model generation, and improving the automation degree of the entire generation process and the accuracy of the final model.

[0097] It is challenging to accurately reconstruct fine branches from point clouds due to limited scanning resolution, high self-occlusion, and complex tree geometry. From human visual perception, the main branches of a tree are the focus of modeling. Therefore, the main branches (excluding leaves and fine branches within the canopy) are first separated from the Gaussian point cloud of the tree. In a tree point cloud, the main branches usually exhibit non-scattered geometric characteristics, including linearity and planarity, while leaves and fine branches exhibit more scattered characteristics. In addition, main branch points usually have low curvature because they are mostly line segments or smooth curves, while fine branch and leaf points exhibit high local curvature. Here, the embodiment uses a method that combines non-scattered degree and curvature features to extract the main branches of a tree. How to extract main stem points from a plurality of three-dimensional Gaussian points to obtain a plurality of main stem Gaussian points will be described in detail below.

[0098] Referring to Figure 4 The main stem points are extracted from a plurality of three-dimensional Gaussian points to obtain a plurality of main stem Gaussian points, including the following steps 401 to 404.

[0099] Step 401: Determine a plurality of neighboring Gaussian points corresponding to each three-dimensional Gaussian point.

[0100] Step 402: Obtain a neighboring point difference value based on the difference between the neighboring Gaussian points and the centroid of the plurality of neighboring Gaussian points, and obtain a covariance matrix of the three-dimensional Gaussian point based on the average value of all neighboring point difference values.

[0101] Step 403: Perform eigenvalue decomposition on the covariance matrix to obtain a plurality of eigenvalues, and calculate the non-scattered degree of each three-dimensional Gaussian point based on the plurality of eigenvalues.

[0102] Step 404: Select main stem Gaussian points from the plurality of three-dimensional Gaussian points based on the non-scattered degree of the three-dimensional Gaussian points.

[0103] The steps 401 to 404 will be described in detail below.

[0104] In some embodiments, orthogonal vector calculation is first performed, and for each three-dimensional Gaussian point, a neighborhood search is performed, i.e., a plurality of neighboring Gaussian points are determined in the space around it. The neighboring Gaussian points refer to a set of points closest to the target three-dimensional Gaussian point in three-dimensional space, which can usually be obtained by setting a fixed search radius or finding the nearest k points (i.e., K-neighbor algorithm, and k is generally 45). The purpose of this is to construct a local neighborhood for each point, and by analyzing the distribution of the point set in this neighborhood, the macroscopic geometric characteristics of the position of the point can be inferred, such as whether it is located on a smooth surface or in a chaotic area.

[0105] After that, for each three-dimensional Gaussian point pi , the geometric center of its corresponding multiple neighboring Gauss points, i.e. the centroid As shown in the following equation (1).

[0106]

[0107] where p j is the jth neighboring Gauss point of the three-dimensional Gauss point p i . Then, the processor calculates the positional difference of each neighboring Gauss point relative to the centroid The multiple neighboring point differences describe the distribution of the point set in the neighborhood relative to the center. Finally, based on the statistical average of all these neighboring point differences, the processor constructs the covariance matrix C corresponding to the three-dimensional Gauss point as shown in the following equation (2).

[0108]

[0109] The covariance matrix describes the dispersion of points in the local neighborhood, which is crucial for calculating the normal vector.

[0110] Then, a more intuitive geometric measure is extracted from the covariance matrix. That is, the eigenvalue decomposition is performed on the covariance matrix (2) for each three-dimensional Gauss point p i , which obtains multiple eigenvalues, i.e. λ1, λ2 and λ3, and these eigenvalues are arranged in the order of λ1> λ2> λ3. These eigenvalues respectively quantify the dispersion or "stretch" degree of the local point set in three mutually perpendicular principal directions.

[0111] Based on these eigenvalues, the non-scattering degree P b of each three-dimensional Gauss point is further calculated as shown in the following equation (3).

[0112]

[0113] The non-scattering degree is a derived index, and its numerical value reflects the linearity or planarity of the distribution of the local point set; a high non-scattering degree value means that the point set is distributed in an elongated line or flat sheet, while a low value means that the point set is distributed in a cluster or scattered in disorder.

[0114] Finally, the final screening is completed according to the quantitative indicators calculated in the previous steps. That is, based on the non-scattering degree P b of each three-dimensional Gauss point, a discrimination threshold is set, and the three-dimensional Gauss points are screened according to the non-scattering degree P b to obtain the Gauss points indicating that the local region where the point is located has obvious linear or columnar structure, i.e. the Gauss points highly consistent with the geometric characteristics of the tree trunk, as the trunk Gauss points, as described below.

[0115] Referring to Figure 5 , the non-scattering degree based on the three-dimensional Gaussian point selects the main stem Gaussian point from a plurality of three-dimensional Gaussian points, including the following steps 501 to step 503.

[0116] Step 501: Select a three-dimensional Gaussian point with a non-scattering degree greater than a preset scattering degree from a plurality of three-dimensional Gaussian points as an initial main stem Gaussian point.

[0117] Step 502: Based on the Euclidean distance between each initial main stem Gaussian point, the plurality of initial main stem points are clustered to obtain a plurality of initial main stem point clusters.

[0118] Step 503: Select an initial main stem point cluster with a cluster point quantity greater than a preset cluster point quantity from a plurality of initial main stem point clusters as a target main stem point cluster, and select a plurality of initial main stem Gaussian points in the target main stem point cluster as main stem Gaussian points.

[0119] The steps 501 to 503 are described in detail below.

[0120] In some embodiments, first, from a plurality of three-dimensional Gaussian points, select a three-dimensional Gaussian point with a non-scattering degree P b greater than a preset scattering degree (i.e. 0.9) as an initial main stem Gaussian point. The non-scattering degree is used to measure the degree of linear distribution of the point set in the neighborhood around a point, and a high non-scattering degree value usually corresponds to a columnar structure with a continuous surface such as a tree trunk or a thick branch. By setting a preset scattering degree as a threshold, most of the points that meet the morphological characteristics of the main stem can be effectively selected as a candidate point set, i.e. the initial main stem Gaussian point.

[0121] Then, the obtained candidate point set is subjected to spatial structuring processing. At this time, based on the Euclidean distance between each initial main stem Gaussian point, the plurality of initial main stem points are clustered to obtain a plurality of initial main stem point clusters. The Euclidean distance is a standard for measuring the straight-line distance between points in three-dimensional space, and the clustering process automatically classifies points that are close to each other in space into a class. Thus, the discrete and disorganized initial main stem Gaussian points are condensed into one or more physically independent point clusters, i.e. initial main stem point clusters, according to their spatial proximity.

[0122] After that, from the multiple initial trunk point clusters, the initial trunk point cluster with a number of clustering points greater than a preset number of clustering points (i.e., Tn(50)) is selected as a target trunk point cluster, and the multiple initial trunk Gaussian points in the target trunk point cluster are taken as the trunk Gaussian points. Since the real tree trunk will form a large continuous point set, and the points generated by noise or irrelevant objects usually form small isolated clusters, by setting a lower limit of the preset number of clustering points, the meaningless small clusters can be effectively filtered out, only the large target trunk point cluster representing the trunk is retained, and finally the points therein are confirmed as high-purity trunk Gaussian points.

[0123] By performing the above steps 501 to 503, the preliminary screening based on the non-scattering degree can quickly lock the candidate points with the trunk geometric properties from the massive original data, realize accurate positioning based on the inherent characteristics, and then through the subsequent clustering processing and secondary screening based on the number of clustering points, the points set (i.e., small clusters) that are isolated noise or small non-trunk structures although the local geometric characteristics are similar to the trunk can be effectively removed as a whole. By using the progressive refinement strategy from "point attribute" to "cluster size", the purity and structural integrity of the finally obtained trunk Gaussian points can be greatly improved, which provides a vital, clean and reliable data source for the subsequent accurate generation of tree skeleton and model.

[0124] Reference Figure 6 is a schematic diagram of a trunk Gaussian point extraction provided by an embodiment of the present application. As shown in Figure 6 , the key data forms in the trunk point extraction process are shown. Fig. (a) is the three-dimensional Gaussian points representing the whole tree initially obtained, including the trunk, branches and leaves. Fig. (b) is the three-dimensional Gaussian points with real colors, for intuitive display. Fig. (c) is the final "trunk Gaussian points" obtained after performing the geometric analysis and screening based on the non-scattering degree, it can be seen that the shape scattered leaves have been effectively removed, only the point set with clear linear characteristics constituting the tree body structure is retained, which lays a solid foundation for the subsequent accurate skeleton generation. Figure 3 Most of the main trunk points are well separated. Although a few main trunk points may be mistakenly separated into other points in the tree crown occasionally, these errors will not affect the generation of the high-quality 3D model of the whole tree. This is because the subsequent tree crown synthesis combines morphological self-similarity and spatial competition to generate a tree grid model.

[0125] By performing the above steps 401 to 404, the morphological differences of different parts of the tree are quantified into a clear non-scattering index through rigorous local geometric analysis, the noise and density unevenness problems in the point cloud can be robustly handled, the points constituting the main structure of the tree with continuous surface can be accurately identified, and the leaf points with complex shape and scattered distribution can be effectively filtered out, so that a very pure and accurate main stem Gaussian point set can be obtained, which lays a solid and reliable data foundation for the accurate extraction of the tree skeleton and the high-fidelity reconstruction of the main stem model.

[0126] Step 102: clustering the plurality of main stem Gaussian points based on the shortest path length between each main stem Gaussian point and the root node to obtain a plurality of main stem clusters.

[0127] The step 102 is described in detail below.

[0128] In this embodiment, a skeleton generation and refinement method based on local clustering and weight distribution is proposed. The method first constructs a graph from the extracted main stem points, then calculates the shortest path using the shortest path fast algorithm, and finally performs local clustering according to the path length and adjacent graph points. The initial main stem skeleton is obtained by connecting each cluster center. Subsequently, a method combining vertex weight fusion and Douglas-Peucker algorithm is used to eliminate unreasonable and relatively minor points, thereby refining the skeleton.

[0129] Based on this, in some embodiments, in order to establish the topological connection relationship between the main stem Gaussian points, the plurality of main stem Gaussian points obtained are first abstracted into an undirected graph, and a point located at the bottom of the tree is specified as a root node, which is the Gaussian point with the lowest spatial coordinates (i.e., the smallest vertical coordinate) among the plurality of main stem Gaussian points. Then, the shortest path length between each main stem Gaussian point in the graph and the root node is calculated, where the shortest path length is the geodesic distance connected along the stem surface points, which can accurately reflect the relative height or position of each point on the stem. Finally, based on the shortest path length, the processor clusters the plurality of main stem Gaussian points, and classifies the points with similar path lengths into a class to form a plurality of main stem clusters. Each main stem cluster essentially represents a cross-sectional point set of the stem at a certain height, thereby organizing the unordered point cloud data into ordered point clusters distributed along the stem axis. The specific process is described as follows.

[0130] With reference to Figure 7 , the plurality of main stem Gaussian points are clustered based on the shortest path length between each main stem Gaussian point and the root node to obtain a plurality of main stem clusters, including the following steps 701 to 702.

[0131] Step 701: generating an undirected weighted graph based on the plurality of main stem Gaussian points, and calculating the shortest path length between each main stem Gaussian point and the root node based on the shortest path fast algorithm.

[0132] Step 702: Clustering the plurality of trunk Gaussian points based on the shortest path length between each trunk Gaussian point and the root node, to obtain a plurality of trunk clusters.

[0133] The steps 701 to 702 are described in detail as follows.

[0134] In some embodiments, a first undirected weighted graph G = (V, E) is generated based on the plurality of trunk Gaussian points, where V is a point set, i.e., each trunk Gaussian point is regarded as a node, and nodes that are adjacent to each other in space are connected by edges, and E represents the edges between the nodes, and the weight of the edge is set as the Euclidean distance between the two connected nodes. This representation saves more memory than an adjacency matrix, especially when dealing with large point clouds. The sparsity of the adjacency list speeds up the traversal by quickly skipping point pairs without edges. The size of the local neighborhood radius (r) is determined by the photo resolution and the quality of the point cloud, and the default value is 0.2 meters.

[0135] Subsequently, after a node located at the base of the tree is designated as the root node, the processor efficiently calculates the shortest path length between each trunk Gaussian point in the graph and the root node along the edges of the graph based on the shortest path fast algorithm. The "shortest path fast algorithm" here is an efficient graph algorithm, and the "shortest path length" calculated represents the geodesic distance of each point along the trunk surface to the root, rather than the simple spatial straight-line distance.

[0136] In this embodiment, unlike the original Dijkstra algorithm with a time complexity of O(V^2) when using an adjacency matrix, the shortest path fast algorithm (SPFA) is used to dynamically manage the relaxation points. SPFA optimizes the node relaxation order through the first-in-first-out feature of the queue to avoid unnecessary repeated operations. The worst-case time complexity of SPFA is O(VE). The specific steps of using the shortest path fast algorithm to calculate the shortest path length between each trunk Gaussian point and the root node are as follows:

[0137] 1) Initialization: Select a starting node s0 from the plurality of trunk Gaussian points and add it to the queue, and set its distance to 0, and set the distance of all other nodes to +∞. Use a priority queue (such as a minimum heap) to store nodes to ensure that the node with the smallest distance is popped out first, thereby achieving an efficient relaxation process.

[0138] 2) Loop iteration: node sj is popped from the queue and performs a relaxation operation on all adjacent nodes sk. The distance of node sk is updated according to the distance of node sj. If the relaxation condition is satisfied: the shortest path length of node sj and the weight between node sj and node sk is less than the shortest path length of node sk, i.e. distance[sj] + weight(sj, sk) < distance[sk], the value of distance[sk] is updated, and sk is added to the queue.

[0139] 3) Optimized queue: in order to efficiently obtain the minimum distance node, a priority queue is used as a data structure. Each operation allows the minimum node to be popped in O(log V) time.

[0140] 4) Termination condition: when the queue is empty, the shortest path of all nodes has been calculated, and all nodes with a distance of +∞ are considered unreachable.

[0141] Next, multiple trunk Gaussian points are clustered based on the shortest path length between each trunk Gaussian point and the root node. The core of this clustering method is to group points with similar shortest path length values into the same group. Since this length value effectively represents the relative position of each point along the axial direction of the trunk, this operation is actually equivalent to effectively dividing the unordered point cloud data according to its height on the trunk. Finally, this step obtains multiple trunk clusters, each of which contains a set of points within a certain height interval on the trunk, thereby organizing the originally chaotic point cloud into multiple point clusters distributed along the axial direction of the trunk.

[0142] By performing steps 701 to 702 above, the key indicator of the shortest path length along the graph connection is introduced. This indicator can accurately reflect the true manifold distance of each point on the trunk surface, thereby giving the discrete point cloud an inherent order along the axial direction of the trunk. The multiple trunk clusters generated finally, like an ordered "horizontal slicing" of the trunk, efficiently structure and sequence the originally chaotic point cloud data, providing a logically clear and correctly arranged input for the subsequent accurate calculation of cluster centroids and generation of a continuous and smooth tree skeleton, greatly improving the accuracy and robustness of skeleton extraction.

[0143] Step 103: Calculate the centroid of each trunk cluster and perform a refined connection process based on multiple centroids to obtain a tree skeleton.

[0144] Step 103 is described in detail below.

[0145] In some embodiments, for each obtained trunk cluster Bin k The centroid of each trunk cluster Bin kcomprising N points with coordinates p i = (x i , y i , z i ), where i = 1, 2,... to N k , the centroid C k of the trunk cluster Bin k is calculated as shown in equation (4).

[0146]

[0147] All the centroids are connected in order of their shortest path length to the cluster they belong to, resulting in an initial tree skeleton. However, this initial skeleton can have redundant nodes and non-smooth transitions, so further refinement is needed. For example, by removing nodes that have little impact on the skeleton's direction or smoothing the connection curves, a final tree skeleton is obtained that accurately and concisely describes the trunk's center direction and branching structure, as described below.

[0148] Referring to Figure 8 , the tree skeleton is obtained by refining the connection based on multiple centroids, including steps 801 to 804.

[0149] Step 801: Connect the multiple centroids to obtain an initial tree skeleton of the target tree.

[0150] Step 802: Calculate the vertex weight corresponding to each vertex in the initial tree skeleton based on the length of the sub-tree connected by each vertex.

[0151] Step 803: Select refinement vertices from the multiple vertices based on the distance between the vertex weight and the skeleton segment.

[0152] Step 804: Obtain the tree skeleton based on the multiple refinement vertices.

[0153] In step 801 of some embodiments, the main skeleton extracted above can contain unreasonable twigs when dealing with missing and noise. To solve this problem, this embodiment uses vertex weight fusion and the Douglas-Peucker method to eliminate unreasonable and relatively minor points, further optimizing the main skeleton. The core idea is to optimize the trunk skeleton by combining sub-tree length weighting (ensuring that small branches on the trunk have smaller weights) and the ability to simplify curves, as implemented by the Douglas-Peucker algorithm.

[0154] Based on this, we first connect the calculated multiple centroids according to the intrinsic order of their clusters to obtain the target tree's initial tree skeleton. This initial tree skeleton is a linear network composed of multiple vertices (i.e., centroids) and skeleton segments connecting these vertices. It preliminarily outlines the direction of the tree's trunk and major branches. However, because it is generated by connecting all centroids, this skeleton may contain too many redundant vertices, resulting in an unsmooth morphology and requires further optimization.

[0155] Next, each structural point of the skeleton is assigned an importance metric. This process calculates the vertex weight corresponding to each vertex based on the length of the subtree connected to each vertex in the initial tree skeleton. The subtree length here refers to the sum of the lengths of all branch skeleton segments below a specific vertex. Therefore, a vertex located at the bifurcation point of the trunk will have a very high vertex weight because it supports a large branch structure; while a vertex located on an ordinary, unbranched branch will have a relatively low weight. This weight value effectively quantifies the importance of each vertex in the entire tree topology, ensuring that small branches close to the trunk receive a significantly smaller weight than the trunk, thereby helping to remove unreasonably thin branches.

[0156] Afterwards, the vertices that need to be retained will be intelligently screened based on a dual standard. This process applies the Douglas-Peck algorithm (a line segment simplification method) and selects refined vertices from multiple vertices based on vertex weights and the distance between skeleton segments. It works by removing vertices that are close to the parent and child nodes that form the segments. Specifically, the processor considers two factors at the same time: one is the vertex weights calculated in the previous step. Key structural points with high weights will be retained first; the other is the distance between a vertex and its adjacent vertices. If a vertex is very close to the straight line segment formed by the two points before and after it, it means that the point contributes very little to describing the skeleton morphology and can be removed. Through this comprehensive judgment, the processor can accurately identify those refined vertices that can maintain the correctness of the skeleton topology and concisely express its geometric shape.

[0157] Finally, the selected thinned vertices are reconnected in their original connection order to create the final tree skeleton. Because this skeleton consists only of refined, filtered vertices, it removes redundant information and noise from the initial skeleton while retaining all key structural nodes. This results in a smoother, simpler form that more accurately reflects the tree's true topology.

[0158] Reference Figure 9 , is a schematic diagram of a main skeleton refinement provided in the embodiment of this application. Figure 9 As shown in , the effects before and after skeleton refinement are visually compared. Figure 9(a) and its local enlargement Figure 9 (c) shows the "initial tree skeleton" obtained by directly connecting the cluster centers. It can be seen that there are unnatural sharp inflection points and redundant small branches at regions 1, 2 and 3. Figure 9 (b) and its local enlargement Figure 9 (d) shows the final "tree skeleton" obtained after the "refined connection processing" based on the vertex weights. It can be seen that the original redundant vertices are removed, the connection of the skeleton becomes smoother and more natural, and the growth form of the branches is more accurately reflected.

[0159] By performing the above steps 801 to 804, the concept of vertex weight is introduced to quantify the structural importance of each vertex. By combining the structural importance (vertex weight) with the geometric redundancy (distance from the skeleton segment) for vertex screening, the key bifurcation points of the tree can be retained to the greatest extent, and the redundant nodes of the straight or gently curved part can be effectively removed, ensuring that the finally generated tree skeleton is not only topologically completely correct, but also more smooth and simplified in geometry, providing an ideal and high-quality central axis for subsequent construction of a high-fidelity model.

[0160] Referring to Figure 10 is a schematic diagram of generation and refinement of a main skeleton provided by an embodiment of the present application. As shown in Figure 10 , the whole process of generating and optimizing the tree skeleton from the trunk Gaussian points is illustrated. Figure 10 (a) is the input trunk Gaussian points; Figure 10 (b) is constructed into an undirected graph and a root node (Root) is specified; Figure 10 (c) shows the result obtained after clustering the points based on the shortest path length. Different colored point sets represent different "trunk clusters"; Figure 10 (d) is the "initial tree skeleton" obtained by connecting the cluster centers. Finally, Figure 10 (e) shows the smooth and simple final "tree skeleton" obtained after the refined connection processing.

[0161] Step 104: generating a tree trunk model based on the plurality of three-dimensional Gaussian points and the tree skeleton.

[0162] The step 104 is described in detail as follows.

[0163] In some embodiments, a tree trunk model with solid volume is constructed based on the generated tree skeleton. In this process, the processor takes the tree skeleton obtained in the previous step as the central axis, and refers to all the three-dimensional Gaussian points (especially the trunk Gaussian points) of the target tree obtained in advance as surface constraints. Specifically, for each segment on the skeleton, the processor finds the three-dimensional Gaussian points around it, and uses these points to fit the radius or cross-sectional shape of the trunk segment, for example by fitting a cylinder. By continuously performing this process along the entire tree skeleton, the linear skeleton is finally "inflated" into a three-dimensional mesh with accurate diameter and smooth surface, i.e. a tree trunk model, which accurately reflects the geometric shape of the trunk, as described below.

[0164] Referring to Figure 11 Based on the plurality of three-dimensional Gaussian points and the tree skeleton, the tree trunk model is generated, including the following steps 1101 to step 1105.

[0165] Step 1101: For each trunk cluster in the tree skeleton, a cylindrical stem is generated.

[0166] Step 1102: Based on the ratio between the straight-line distance of each three-dimensional Gaussian point and the cylindrical stem and the maximum straight-line distance, the distance weight of each three-dimensional Gaussian point is obtained.

[0167] Step 1103: Based on the accumulation of the product of the distance weight and the straight-line distance corresponding to all three-dimensional Gaussian points, a distance sum objective function is generated.

[0168] Step 1104: Based on the least squares method, the distance sum objective function is solved to obtain the optimal cylindrical radius, optimal end point and optimal direction vector corresponding to the trunk cluster, and the cylindrical stem is updated based on the optimal cylindrical radius, optimal end point and optimal direction vector.

[0169] Step 1105: Based on all the updated cylindrical stems, a tree trunk model is generated.

[0170] The steps 1101 to 1105 are described in detail below.

[0171] In order to realize efficient and high-fidelity three-dimensional generation of tree trunks, a more accurate trunk mesh model is generated based on the optimized refined skeleton. According to the research on tree geometry representation, a cylindrical model is selected in this embodiment because it is simple and robust, and is superior to more complex methods such as multi-scale curve fitting. Then, an improved Levenberg-Marquardt cylindrical fitting method is applied to determine the radius of each branch. This method approximates the trunk mesh by fitting a cylinder to the Gaussian point cloud, which is a typical nonlinear least squares problem ( Figure 6). The input includes the trunk point positions P, and the parameters to be solved are the axial direction vector a of the cylinder, the radius r, and the end point position P a .

[0172] Based on this, first, an initial geometric primitive is constructed for each segment of the trunk. That is, for each trunk cluster in the tree skeleton, a cylindrical trunk is generated. The cylindrical trunk here can be understood as a cylinder with an initial radius, position, and orientation, which serves as a preliminary approximation or template of the shape of this segment of the trunk, with its central axis aligned similarly to the tree skeleton generated in the previous step, providing a reference for subsequent accurate radius optimization.

[0173] In view of the existence of noise and outliers in the Gaussian point cloud of the trunk, the ordinary least squares fitting effect is not good. Therefore, in this embodiment, a nonlinear least squares method based on distance weight is used to fit the trunk radius, in which points close to the cylinder are given greater weight. Based on this, the straight line distance dist(p i ) between each three-dimensional Gaussian point and the central axis of the cylindrical trunk generated in the previous step is calculated, and the distance weight of each three-dimensional Gaussian point is obtained based on the ratio between the straight line distance and the maximum straight line distance dist max ) found in all related points as shown in the following formula (5).

[0174]

[0175] The distance weight can quantify the contribution of each three-dimensional Gaussian point to determining the final radius of this segment of the trunk, and its design aims to reduce the influence of abnormal points or noise points far from the central axis, and the weights of all points are normalized in the range of [0, 1].

[0176] Next, a mathematical model for measuring the pros and cons of the current cylindrical radius fitting will be constructed. This process will generate a distance sum objective function based on the cumulative value of the product of the distance weight and the straight line distance corresponding to all three-dimensional Gaussian points as shown in the following formula (6).

[0177]

[0178] The distance sum objective function is a single numerical value that integrates the position information and weight information of all related three-dimensional Gaussian points, and the value directly reflects the overall deviation degree between the current cylindrical trunk and the actual point cloud data. The final goal of the algorithm is to minimize the value of this function by adjusting the cylindrical radius.

[0179] Next, based on the least squares method, the distance sum objective function generated in the previous step is solved. Through this solving process, the processor can accurately calculate the axial direction vector a of the cylinder, the radius r, and the end point position Pa , the optimal radius, optimal endpoint and optimal direction vector of the corresponding cylinder. After obtaining this optimal value, the processor will immediately update the cylinder trunk with it, so that its radius changes from the initial value to the accurate value that can best match the surrounding point cloud data.

[0180] Referring to Figure 12 , a schematic diagram of a cylinder trunk fitting method provided by an embodiment of the present application is shown. As shown in Figure 12 , it is shown how to determine the optimal axial direction vector a, radius r and endpoint position P of a segment of the skeleton (represented by the center dashed line a). a The processor will consider the surrounding three-dimensional Gaussian points (gray points p i ), and based on the least square optimization, find a cylindrical surface that can best fit these points. The enlarged view more intuitively shows how the optimal radius r is determined by the distribution of the surrounding points on the cross section of the skeleton, and this process ensures that the final generated trunk model can accurately reflect the true thickness variation of the trunk.

[0181] Finally, based on all the optimized and updated cylinder trunks of the trunk clusters, they are smoothly connected and combined together in the original order of the tree skeleton. Since the radius of each cylinder trunk has been independently and data-drivenly optimized to accurately reflect the true thickness of the trunk at that position, after they are all combined, a complete tree trunk model with natural shape and accurate thickness variation can be generated.

[0182] By performing the above steps 1101 to 1105, for each segment of the trunk, the optimal cylinder radius, optimal endpoint and optimal direction vector are iteratively calculated by constructing and solving a weighted distance sum objective function, which can make full use of the distribution information of the surrounding three-dimensional Gaussian points and effectively suppress the interference of noise points by means of the distance weight mechanism, ensuring that the calculated radius has very high geometric accuracy, and the finally generated tree trunk model can accurately reproduce the natural thickness variation (i.e. taper) of the real trunk from the root to the top, thereby achieving a significant improvement in fidelity.

[0183] Referring to Figure 13 , a schematic diagram of the generation process of a tree trunk mesh model provided by an embodiment of the present application is shown. As shown in Figure 13 , it is shown how to generate the final tree trunk model from the tree skeleton. Figure 13 (a) is the input "tree skeleton"; then determine the radius of each branch based on the improved Levenberg-Marquardt cylinder fitting method; Figure 13 (b) illustrates the process of cylinder fitting for a local segment of the skeleton; Figure 13(c) shows all the optimized cylinder stems are connected together, and the final complete "tree trunk model" is generated, that is, the mesh model of the entire trunk is generated using the refined continuous skeleton constraints and topological connectivity relationship, and the surface is smooth and the transition is natural; Figure 13 (d) is a geometric error analysis chart, where blue indicates smaller errors, proving that the model generated by the method has high fitting degree with the original point cloud data.

[0184] Step 105: Based on the connection direction between each three-dimensional Gaussian point and the tree trunk model, the plurality of three-dimensional Gaussian points are screened for tree crowns to obtain a plurality of tree crown points, and a tree model of the target tree is generated based on the plurality of tree crown points and the tree trunk model.

[0185] The following describes step 105 in detail.

[0186] To solve the problem of poor data-driven fitting of fine branches inside the tree crown, a method combining morphological self-similarity and spatial competition is proposed to generate a tree crown consistent with the growth pattern of the tree. This is achieved through the self-similarity of the branching angle based on the assumptions of plant growth biology, combined with a spatial colonization algorithm. This method enables the generated tree crown to naturally blend with the trunk. By controlling and optimizing the branching angle, a tree crown that more accurately reflects the morphology of a real tree can be generated, rather than relying solely on parameterized L-systems. This method not only ensures the reasonableness of the tree crown morphology, but also effectively prevents the formation of abnormal structures.

[0187] Based on this, the task is to reconstruct a morphologically realistic tree crown based on the generated trunk model, thereby generating a complete tree model. Due to the sparsity and disorder of the original data in the tree crown part, the processor uses a screening strategy that combines knowledge and data. It analyzes the connection direction between each original three-dimensional Gaussian point and the generated tree trunk model. This "connection direction" refers to the vector from the nearest point on the trunk model to the three-dimensional Gaussian point. The processor then performs tree crown screening, that is, it determines whether the connection direction is reasonable based on prior knowledge of botany, such as branching angle rules, and only retains points that conform to natural growth patterns as tree crown points. Finally, based on these screened, more reasonably distributed tree crown points and the existing tree trunk model, the mesh structure of the tree crown is constructed through surface reconstruction or programmatic generation, and it is merged with the trunk model, thereby obtaining the final tree model of the target tree.

[0188] The following will further describe how to screen suitable tree crown points.

[0189] Reference Figure 14 Based on the connection direction between each three-dimensional Gaussian point and the tree trunk model, the plurality of three-dimensional Gaussian points are screened for tree crowns to obtain a plurality of tree crown points, including the following steps 1401 to 1404.

[0190] Step 1401: For each branch skeleton point in the tree trunk model, a branch angle of each branch skeleton point is calculated based on the branch skeleton point, a parent node of the branch skeleton point, and a branch child node of the branch skeleton point.

[0191] Step 1402: A branch angle interval is obtained based on a numerical relationship between the branch angle and a preset branch threshold.

[0192] Step 1403: A growth direction of each three-dimensional Gaussian point is calculated based on a connection direction of each three-dimensional Gaussian point and each branch skeleton point.

[0193] Step 1404: A three-dimensional Gaussian point with a corresponding angle of the growth direction in the branch angle interval is selected from the plurality of three-dimensional Gaussian points as a crown point.

[0194] The steps 1401 to 1404 are described in detail as follows.

[0195] First, a tree skeleton array is initialized. The tree skeleton array is initialized by setting each branch skeleton point in the skeleton set (i.e., the plurality of branch skeleton points) as a main skeleton point one by one, and defining a search radius R. The branch skeleton points p within the search radius are stored in the set S(p). The three-dimensional Gaussian points to be searched can only affect the nearest branch skeleton point p, while the branch skeleton point p can be affected by multiple three-dimensional Gaussian points to be searched within the search radius R. The specific mathematical relationship is shown in the following formula (7).

[0196]

[0197] where q is a point set composed of the plurality of three-dimensional Gaussian points to be searched within the search radius R of the branch skeleton point p.

[0198] At the same time, the processor performs structural analysis on the generated tree skeleton to extract the inherent growth rule. That is, for each branch skeleton point p (with a position coordinate c k ) in the tree trunk model, the branch skeleton point refers to the node at the branch position of the tree skeleton. The processor calculates a branch angle β of each branch skeleton point based on the position coordinate c k of the branch skeleton point, the position coordinate c k-1 of the parent node of the branch skeleton point at the upper level in the skeleton, and the position coordinate c k+1 of the branch child node at the next level, as shown in the following formula (8).

[0199]

[0200] The bifurcation angle β geometrically accurately quantifies the deviation angle of a new branch when it grows from the main trunk or the upper branch, thereby capturing the unique morphological characteristics of the tree.

[0201] Reference Figure 15 , is a schematic diagram of a bifurcation angle provided in an embodiment of the present application. Figure 15 As shown in the figure, a schematic diagram of the calculation principle of "bifurcation angle" used for crown screening. Figure 15 (a) The angle between the branch and the trunk (Axial Angle) is marked on the real tree morphology. Figure 15 (b) Abstract it into a mathematical definition on the skeleton, that is, at the branch skeleton point c k At, it is connected to the parent node c k-1 The line connecting the two branch nodes c k+1 The lines connecting the two branches form the bifurcation angles β (including β1 and β2). By calculating these angles, we can quantify and learn the unique growth pattern of the tree, providing a biological basis for subsequent crown point selection.

[0202] It is understandable that there are multiple bifurcation angles, such as Figure 15 In the case of β1 and β2, the largest one is taken as the bifurcation angle corresponding to the branch skeleton point, that is, β k =max(β1,β2).

[0203] Furthermore, the processor will convert the specific angle value obtained from the previous analysis into a universal screening rule. This process will obtain a bifurcation angle range based on the numerical relationship between the bifurcation angle and the preset bifurcation threshold. Specifically, the processor may count the distribution of all bifurcation angles, and combine a "preset bifurcation threshold" to determine a reasonable upper and lower limit, and finally form a bifurcation angle range I. This range represents the angle range that the branches of this particular tree should follow during its natural growth process and is considered to be reasonable, providing a clear, data-driven basis for subsequent screening.

[0204] The research experiments show that under normal distribution, the bifurcation angles in the tree skeletons of the tree sample statistically fall within the bifurcation angle intervals I1 = [β-σ, β+σ] and I2 = [β-2σ, β+2σ]. Approximately 68.27% and 95.45% respectively. For most tree species, most axis angles are within the interval defined by the bifurcation angle interval I1 = [β-σ, β+σ]. In this embodiment, the experiment further verified this observation, referring to Figure 16 , is a schematic diagram of the distribution of bifurcation angles in a tree skeleton provided by an embodiment of the present application. Figure 16As shown, the figure shows the refined trunk skeleton in the distribution of the axis angle, in which 19 of the 23 branch angles are located in I1=[β-σ, β+σ]. Based on the above morphological knowledge, the branch angle constraint interval generated by the crown layer is preferentially synthesized in the defined interval I1, while the branches located outside the interval I2=[β-2σ, β+2σ] are excluded. This ensures that the fine branches of the generated crown layer are similar to the branch characteristics of the existing trunk skeleton, avoiding abnormal shapes and structures in the synthesis process of fine branches.

[0205] Then, the processor will assign a potential growth attribute to each three-dimensional Gaussian point to be screened. This process will calculate the growth direction n of each three-dimensional Gaussian point based on the connection direction of each three-dimensional Gaussian point and each branch skeleton point. Specifically, for each original three-dimensional Gaussian point, the processor will find its nearest branch skeleton point on the tree trunk model, and the vector n from the nearest point to the three-dimensional Gaussian point will be defined as the growth direction of the point. This direction simulates the possible path of the point if it is part of the tree crown growing out of the trunk.

[0206] After that, the estimated skeleton growth direction is compared with the direction vector, i.e. the processor selects the three-dimensional Gaussian points whose growth direction angles are within the branch angle interval I as the tree crown points, and the points whose growth angle is not within the constraint interval I are deleted. Only those points whose growth direction angles are within the reasonable interval are considered to be valid points that conform to the growth pattern of the tree itself, and are finally confirmed as tree crown points, thus forming a high-fidelity tree crown data.

[0207] This algorithm combines morphological self-similarity and spatial competition and can only extract the corresponding three-dimensional skeleton of the tree. In order to further generate a geometric grid model, this paper uses the actual radius of the trunk fitted by the previous cylinder as a regularization term, and uses an "anisotropic growth model" to construct a constraint term for calculating the radius of the fine branches in the synthesized crown. The complete three-dimensional fine skeleton and branch radius of the tree are used to generate a high-quality tree grid model.

[0208] By performing the above steps 1401 to 1404, the "branch angle" rule specific to the tree is learned and quantified from the existing and reliable tree skeleton, and a reasonable "branch angle interval" is established accordingly. This rule learned from the tree itself is used as a judgment standard to judge each potential tree crown point, and only those points with "growth direction" conforming to the rule are retained, so that the structurally reasonable and morphologically realistic tree crown points can be effectively screened from the sparse and chaotic data, effectively solving the technical difficulties of traditional methods that cannot reconstruct fine branch structures due to data missing, leading to distortion of the crown model. The finally generated crown model therefore has higher biological credibility and visual realism.

[0209] Reference Figure 17 , is a schematic diagram of canopy generation that integrates morphological self-similarity and spatial competition, as provided in the embodiment of this application. Figure 17 As shown in , the iterative process of generating crown branches based on the generated trunk model and the selected crown points is depicted. Figure 17 (a) to Figure 17 (f) shows a dynamic growth process: starting from a skeleton end point ( Figure 17 (a)), within a certain search radius ( Figure 17 (b) Finding connectable crown points Figure 17 (c), then calculates the growth direction of these connections and removes those with unreasonable angles (17(d)), ultimately generating a new skeleton branch (17(e)), which serves as the starting point for the next iteration (17(f)). This process simulates the natural growth of tree branches and ensures that the generated crown is structurally sound.

[0210] Reference Figure 18 , is a flowchart of a tree model generation process provided by an embodiment of the present application. Figure 18 As shown in , it is the overall technical flow chart of the method for generating high-fidelity tree mesh models from multi-view images proposed in this application. The process is mainly divided into three modules: the "3D Gaussian expression" module on the left is responsible for generating high-quality 3D Gaussian point clouds after semantic segmentation from multi-view image input; the "trunk skeleton generation and optimization" module in the middle elaborates on the complete process from trunk point separation, trunk extraction (clustering and skeleton refinement) to the final generation of the trunk mesh model; the "three-dimensional crown generation" module on the right shows how to combine the screened crown points with the trunk model to generate a complete, textured final tree model. The entire process clearly shows the step-by-step refinement and construction process of the present invention from raw data to high-fidelity models.

[0211] In order to verify the reliability of the method provided in this application for generating a high-fidelity tree mesh model from multi-view images, a performance simulation is also carried out as described below.

[0212] Reference Figure 19 , is a schematic diagram of simulation results of a method for generating a high-fidelity tree mesh model from a multi-view image provided by the present application. Figure 19The complete process of generating five trees with different structures is shown, including three-dimensional Gaussian points segmented from multi-view images, trunk point clouds, clustered skeletons, trunk geometric mesh models, complete tree geometry, and geometric errors of the generated overall tree model. The smaller subgraph shows that the color gradually changes from blue to red, with blue representing the minimum geometric error between the generated tree mesh model and the original input points, and red representing the maximum error. From the perspective of visualization, the method proposed in this application can generate tree mesh models with high quality from multiple perspectives. Although the segmented Gaussian points have missing areas, noise and other defects, the tree mesh model generated by the method proposed in this application still maintains a reasonable topological structure and exhibits superior geometric accuracy.

[0213] Reference Figure 20 is the first contrast simulation result schematic diagram of the tree model generation provided by the embodiment of the application. As Figure 20 shown in FIG. 8, the relevant simulation data in the first simulation data schematic table shown in Figure 21 is used, and the visualization results and error distribution diagrams of the five tree trunks with different morphological structures generated by three methods in Figure 12 can be seen from the generated results that TreeQSM cannot reconstruct the continuous branch geometry and complete tree trunk branch. This is mainly because TreeQSM does not use continuous skeleton constraints in the reconstruction process and mainly relies on stage-by-stage cylinder fitting. Although AdTree can reconstruct wide tree trunk branches, it performs poorly in capturing the overall characteristics of the tree trunk and local curves. This is mainly because the input data consists of three-dimensional Gaussian points containing missing areas, noise and local sparseness, rather than high-density laser scanning point clouds. This leads to the fitting of some small branches and the generation of unreasonable radii in the tree trunk area. As shown in the visualization results and geometric error diagrams, compared with TreeQSM and AdTree, the method proposed in this application maintains a more reasonable branch structure. The generated model more accurately and more detailedly preserves the branch features such as bifurcation and curves. In addition, this embodiment uses two indicators, average error and root mean square error, to quantitatively analyze the advantages of the method proposed in this application.

[0214] Reference Figure 22 is the second contrast simulation result schematic diagram of the tree model generation provided by the embodiment of the application. As Figure 22The visualization results and error distribution plots of five complete trees generated using TreeQSM, AdTree and the method proposed in this application are shown in FIG. 6. As can be seen from the visualization results, TreeQSM fails to reconstruct complete and coherent branches. Although AdTree can roughly reconstruct the complete tree structure, it performs poorly in capturing reasonable branch structure and local curves, and in the presence of noise and insufficient data, it can generate distorted or incorrect structures. In contrast, the method proposed in this application better preserves the geometric branches and visualization of the tree shape. In addition, the advantages of the method proposed in this application are quantitatively evaluated using two indicators in the second simulation data schematic diagram shown in FIG. 7. Figure 23

[0215] In addition, in order to quantitatively analyze the quality of the model, the precision error of the tree trunk and the complete tree of the five samples is calculated in this embodiment. The average error and the root mean square error are shown in FIG. 8. Figure 21 In terms of fitting error, the method proposed in this application achieves the smallest geometric fitting error on multiple trees. For example, in fitting the tree trunks of Litchi Tree 1, Litchi Tree 2 and Papaya Tree, the method proposed in this application has the smallest average error and root mean square error. Although TreeQSM shows the smallest average distance error and root mean square error in fitting the tree trunks of Loquat Tree and Acacia Tree, the method proposed in this application is still very close to the true input data, with an average fitting error of 0.018 meters and an average root mean square error of 0.023 meters for the tree trunks of all five trees. The average fitting error of the entire tree of the five samples is 0.027 meters, and the average root mean square error is 0.045 meters. In addition, it needs to be emphasized that although TreeQSM performs better in terms of average error and root mean square error in some cases, it fails to generate coherent geometric branches, resulting in significant fragmentation and discretization in the model. Therefore, visualization and geometric error together highlight the reliability and accuracy of the method proposed in this application in generating tree trunks and entire trees.

[0216] Referring to FIG. 9, Figure 24 is a schematic diagram of the three-dimensional Gaussian point segmentation of various trees provided by the embodiment of the present application. As shown in FIG. 10, Figure 24 quantitatively and qualitatively demonstrates the effectiveness of the three-dimensional Gaussian model segmentation from multi-view photos. Figure 24 The three-dimensional Gaussian training and segmentation results of five different trees are shown. In addition, in order to make a consistent and meaningful comparison, commonly used metrics are selected: peak signal-to-noise ratio (PSNR), L1 loss and similarity (as shown in FIG. 11). Figure 23 PSNR represents the quality of the rendered three-dimensional Gaussian scene (the higher the value, the better the quality). A lower L1 loss indicates a higher quality of the trained three-dimensional Gaussian model. Similarity measures whether the segmented three-dimensional Gaussian model belongs to the same category. ​

[0217] The rendered images and 3D Gaussian splats are visualized in the 7K step and the 30K step iterations in the embodiments of the present application. As shown in Figure 24 there is a significant difference in visual quality between the two configurations. Although in many cases, the rendered image in the 7K step iteration looks acceptable, due to the adaptive density optimization, larger Gaussian spheres are used in the rendering, especially in certain parts of the canopy. Specifically, the Gaussian scene average PSNR in the 7K step iteration is 24.22 dB, and the average L1 loss is 0.036. In contrast, the average PSNR in the 30K step iteration increases to 26.85 dB, and the average L1 loss decreases to 0.026, increasing by 10.87% and 27.78%, respectively. Therefore, in order to obtain high-quality results in the 3D Gaussian segmentation process, the Gaussian scene obtained in the 30K step iteration is selected for segmentation. In this study, the SAGA framework is followed, and the 3D Gaussian segmentation is performed in combination with the provided interactive graphical interface (saga_gui.py). The similarity (ranging from 0 to 1) is used to quantitatively evaluate the segmentation performance, reaching an average similarity of 0.91, indicating that the segmented three-dimensional Gaussian distribution has high coherence. In addition, visual inspection of the segmented three-dimensional Gaussian points further confirms their high consistency with the overall appearance of the original trees.

[0218] Referring to Figure 25 is a schematic diagram of comparative simulation results of different search radii provided by the embodiments of the present application. The search radius R determines the number of candidate crown points in the growth direction of the skeleton point in each iteration, and significantly affects the final shape of the generated tree. If R is too small, the search area in each iteration will become limited, leading to a local density bias in the generation process. This makes it difficult to visually reconstruct the overall shape of the tree canopy. On the contrary, if R is too large, the number of points included in the search is too large, which usually leads to redundant twigs. As shown in Figure 25 , a comparison of three trees with different structures under different search radii is shown. When the radius is set to 1 centimeter, the visual quality is poor, and the generated crown structure cannot capture the overall tree shape. At a radius of 2.5 centimeters, the visual quality improves, but it may still be insufficient in the case of sparse or missing data. A radius of 5 centimeters can achieve satisfactory mesh model generation on the five experimental trees. It should be noted that in practical applications, the search radius R should be carefully selected according to the sparsity of the input point cloud and the balance between modeling accuracy and computational cost.

[0219] Referring to Figure 26 is a schematic diagram of comparative simulation results with and without self-similarity angle constraints provided by the embodiments of the present application. As shown in Figure 26As shown in the middle, we analyzed the effect of the self-similarity angle constraint on the canopy synthesis using two cases of lychee trees with complex branching structures. In this experiment, the search radius was fixed at 2.5 cm, and the only variable was whether the self-similarity angle constraint was present or not. The self-similarity angle constraint effectively regulated the growth direction of the fine branches within the canopy, reducing irregular or unnatural branch directions. In contrast, without this constraint, some branches would grow vertically upward or downward, resulting in excessively wide or narrow branching angles (as shown in the zoomed-in subplots). It is important to note that the integration of morphological self-similarity and spatial competition provides an ideal model for canopy generation, but it does not completely determine the final tree structure. In the real world, external factors, such as pruning in urban environments, can cause branches to deviate from their biologically ideal growth trajectories. Figure 16

[0220] ​The embodiment of the present application provides a method for generating a high-fidelity tree grid model from a multi-view image and related equipment, which comprises the following steps: firstly, obtaining a multi-view image of a target scene, the target scene comprising a target tree; using a three-dimensional point cloud technology to perform three-dimensional Gaussian extraction on the multi-view image to obtain a plurality of scene Gaussian points; selecting, based on the low-dimensional features of each scene Gaussian point, a scene Gaussian point consistent with the low-dimensional features and the tree features from the plurality of scene Gaussian points as a three-dimensional Gaussian point, and determining a plurality of adjacent Gaussian points corresponding to each three-dimensional Gaussian point; obtaining an adjacent point difference based on the adjacent Gaussian points and the difference between the adjacent Gaussian points and the centroid of the plurality of adjacent Gaussian points; obtaining a covariance matrix of the three-dimensional Gaussian point based on the average value of all adjacent point differences; performing eigenvalue decomposition on the covariance matrix to obtain a plurality of eigenvalues; calculating the non-scattering degree of each three-dimensional Gaussian point based on the plurality of eigenvalues; selecting a three-dimensional Gaussian point with a non-scattering degree greater than a preset scattering degree from the plurality of three-dimensional Gaussian points as an initial trunk Gaussian point; performing clustering processing on the plurality of initial trunk points based on the Euclidean distance between each initial trunk Gaussian point to obtain a plurality of initial trunk point clusters; selecting an initial trunk point cluster with a cluster point quantity greater than a preset cluster point quantity from the plurality of initial trunk point clusters as a target trunk point cluster, and taking the plurality of initial trunk Gaussian points in the target trunk point cluster as trunk Gaussian points; then, generating an undirected weighted graph based on the plurality of trunk Gaussian points, and calculating the shortest path length between each trunk Gaussian point and a root node based on a shortest path fast algorithm; clustering the plurality of trunk Gaussian points based on the shortest path length between each trunk Gaussian point and the root node to obtain a plurality of trunk clusters, each trunk cluster comprising a plurality of trunk Gaussian points; next, calculating the centroid of each trunk cluster, and connecting the plurality of centroids to obtain an initial tree skeleton of the target tree, the initial tree skeleton comprising a plurality of vertices, each two adjacent vertices forming a skeleton line segment; calculating the vertex weight corresponding to each vertex based on the length of the sub-tree connected by each vertex in the initial tree skeleton; selecting a refined vertex from the plurality of vertices based on the distance between the vertex weight and the skeleton line segment; and obtaining a tree skeleton based on the plurality of refined vertices; subsequently, for each trunk cluster in the tree skeleton, generating a cylindrical stem; obtaining a distance weight of each three-dimensional Gaussian point based on the ratio between the straight-line distance of each three-dimensional Gaussian point and the cylindrical stem and the maximum straight-line distance; generating a distance sum objective function based on the accumulated value of the product of the distance weight and the straight-line distance corresponding to all three-dimensional Gaussian points; solving the distance sum objective function based on the least square method to obtain the optimal cylindrical radius, the optimal end point and the optimal direction vector corresponding to the trunk cluster; and updating the cylindrical stem based on the optimal cylindrical radius, the optimal end point and the optimal direction vector, and generating a tree trunk model based on all the updated cylindrical stems.Finally, for each branch skeleton point in the tree trunk model, a branch angle of each branch skeleton point is calculated based on the branch skeleton point, a parent node of the branch skeleton point, and a branch child node of the branch skeleton point, a branch angle interval is obtained based on a numerical relationship between the branch angle and a preset branch threshold, a growth direction of each three-dimensional Gaussian point is calculated based on a connection direction between each three-dimensional Gaussian point and each branch skeleton point, three-dimensional Gaussian points corresponding to angles in the growth direction and in the branch angle interval are selected as crown points from the plurality of three-dimensional Gaussian points, and a tree model of a target tree is generated based on the plurality of crown points and the tree trunk model.

[0221] The embodiment of the application can overcome noise and breakpoints in original data by extracting backbone points from three-dimensional Gaussian points and using a clustering algorithm based on shortest path length and skeleton thinning processing, generate a tree trunk skeleton with correct topological structure and accurate geometric shape, effectively solve the technical problem of trunk model distortion caused by data quality problems in the prior art, and for the tree crown part with serious data missing, a filtering strategy based on connection direction is used, which implicitly contains the morphological law of plant growth, can intelligently filter out the tree crown points that meet the natural growth pattern from the sparse point cloud, effectively make up for the defect that pure data-driven methods cannot reconstruct fine branch structure, so that the finally generated complete tree model not only has clear and accurate trunk structure, but also has full and detailed crown shape, thereby significantly improving the overall realism; in addition, three-dimensional Gaussian points with stronger expression ability are used as data basis, which improves the fidelity of subsequent processing, and an automatic segmentation method based on low-dimensional features is used to intelligently and accurately separate the target tree from the complex background environment, avoiding the interference of background objects on tree modeling in traditional methods, and obtaining high-purity input data without a large amount of manual preprocessing, laying a solid and reliable foundation for subsequent accurate trunk skeleton extraction and high-fidelity model generation, thereby improving the automation degree of the entire generation process and the accuracy of the final model; and through rigorous local geometric analysis, the morphological differences of different parts of the tree are quantified into a clear non-scattering index, which can robustly handle noise and density unevenness in point cloud, accurately identify points with continuous surface that constitute the main structure of the tree, and effectively filter out leaf points with complex shape and scattered distribution, so that a very pure and accurate trunk Gaussian point set can be obtained, laying a solid and reliable data foundation for subsequent accurate extraction of tree skeleton and high-fidelity reconstruction of trunk model; and the shortest path length along the graph connection is introduced as a key indicator.The index can accurately reflect the real manifold distance of each point on the trunk surface, thereby giving the discrete point cloud an inherent order along the trunk axis, so that the plurality of trunk clusters generated finally, like the ordered "horizontal sectioning" of the trunk, efficiently structures and sequences the originally disordered point cloud data, provides a logically clear and correctly arranged input for subsequent accurate calculation of cluster centroids and generation of continuous and smooth tree skeletons, greatly improving the accuracy and robustness of skeleton extraction; in addition, the concept of vertex weight is introduced to quantify the structural importance of each vertex, and vertex selection is performed by combining structural importance (vertex weight) with geometric redundancy (distance from the skeleton segment), which can effectively remove redundant nodes in straight or gently curved parts while retaining key branching points of the tree to the greatest extent, ensuring that the final generated tree skeleton is not only topologically completely correct, but also more smooth and compact in geometry, providing an ideal and high-quality central axis for subsequent high-fidelity model construction; furthermore, for each segment of the trunk, the optimal cylindrical radius, optimal end point and optimal direction vector are iteratively calculated by constructing and solving a weighted distance and objective function, which can fully utilize the distribution information of surrounding three-dimensional Gaussian points and effectively suppress the interference of noise points by means of distance weight mechanism, ensuring that the calculated radius has high geometric accuracy, and the final generated tree trunk model can accurately reproduce the natural thickness change (i.e. taper) of the real trunk from the root to the top, thereby achieving significant improvement in fidelity; finally, the unique "branching angle" rule of the tree is learned and quantified from the existing and reliable tree skeleton, and a reasonable "branching angle interval" is established, and this rule learned from the tree itself is used as a judgment standard to judge each potential crown point, only retaining those points with "growth direction" conforming to the rule, so that the structurally reasonable and morphologically realistic crown points can be effectively selected from the sparse and disordered data, effectively solving the technical problems of traditional methods that cannot reconstruct fine branch structure due to data loss and cause distortion of the crown model, and the finally generated crown model has higher biological credibility and visual realism.

[0222] The embodiments of the present application also provide an electronic device, comprising:

[0223] at least one memory;

[0224] at least one processor;

[0225] at least one program;

[0226] The program is stored in the memory, and the processor executes at least one program to implement the method for generating a high-fidelity tree grid model from a multi-view image. The electronic device can be any smart terminal, including a mobile phone, a tablet computer, a personal digital assistant (PDA), a vehicle-mounted computer, etc.

[0227] Please refer to Figure 27 , Figure 27 The hardware structure of the electronic device of another embodiment is illustrated, and the electronic device includes:

[0228] The processor 2701 can be implemented in the form of a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, etc., for executing related programs to implement the technical solutions provided by the embodiments of the present application;

[0229] The memory 2702 can be implemented in the form of a ROM (Read Only Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory), etc. The memory 2702 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are stored in the memory 2702 and called and executed by the processor 2701 to implement the method for generating a high-fidelity tree grid model from a multi-view image according to the embodiments of the present application;

[0230] The input / output interface 2703 is used to realize information input and output;

[0231] The communication interface 2704 is used to realize the communication interaction between the device and other devices, which can realize communication through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.);

[0232] The bus 2705 transmits information between various components (such as the processor 2701, the memory 2702, the input / output interface 2703, and the communication interface 2704) of the device.

[0233] Among them, the processor 2701, the memory 2702, the input / output interface 2703, and the communication interface 2704 realize the communication connection between each other inside the device through the bus 2705.

[0234] The embodiment of the present application further provides a storage medium, which is a computer readable storage medium, and stores a computer program. The computer program is executed by a processor to implement the method for generating a high-fidelity tree grid model from a multi-view image.

[0235] The memory is a non-transitory computer readable storage medium, and can be used to store a non-transitory software program and a non-transitory computer executable program. In addition, the memory can include a high-speed random access memory, and can further include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and the remote memory can be connected to the processor through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0236] The embodiments described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0237] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps than those shown in the figures, or combine certain steps or different steps.

[0238] The device embodiments described above are merely illustrative, and the units described as separate components can or can not be physically separated, that is, can be located in one place, or can be distributed on multiple network units. According to actual needs, part or all of the modules can be selected to achieve the purpose of the embodiments of the present application.

[0239] Those skilled in the art can understand that all or some steps in the above disclosed method, the functions of the modules / units in the system and the device can be implemented as software, firmware, hardware and their appropriate combinations.

[0240] The terms "first", "second", "third", "fourth", and the like in the description of this application and in the claims hereof, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so termed herein is solely for the convenience of the reader and does not limit the scope of the application. It is also to be understood that the description and examples in this application are intended to cover all possible combinations where any of the several elements can represent one or more elements.

[0241] It should be understood that, in this application, "at least one" means one or more, "multiple" means two or more. "And / or" is used to describe the relationship between associated objects, which means that there can be three relationships, for example, "A and / or B" can mean: only A, only B, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c, can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0242] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the above-mentioned units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. The coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0243] The units described above as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on a plurality of network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment of the present application.

[0244] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0245] When the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in part, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions used to cause a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various other media that can store programs.

[0246] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and are not intended to limit the scope of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A method for generating a high-fidelity tree mesh model from multi-view images, characterized in that: The method comprises: Acquire multiple three-dimensional Gaussian points of the target tree, and extract trunk points from the multiple three-dimensional Gaussian points to obtain multiple trunk Gaussian points; Clustering the plurality of trunk Gaussian points based on the shortest path length between each of the trunk Gaussian points and a root node to obtain a plurality of trunk clusters, each of the trunk clusters including a plurality of the trunk Gaussian points; Calculating the centroid of each trunk cluster, and performing a thinning connection process based on multiple centroids to obtain a tree skeleton; Generate a tree trunk model based on the plurality of three-dimensional Gaussian points and the tree skeleton; Based on the connection direction between each of the three-dimensional Gaussian points and the tree trunk model, the multiple three-dimensional Gaussian points are subjected to crown screening to obtain multiple crown points, and the tree model of the target tree is generated based on the multiple crown points and the tree trunk model.

2. The method for generating a high-fidelity tree mesh model from multi-view images according to claim 1, characterized in that: The extracting of the main points from the plurality of three-dimensional Gaussian points to obtain the plurality of main Gaussian points comprises: Determine a plurality of adjacent Gaussian points corresponding to each of the three-dimensional Gaussian points; Obtaining adjacent point difference values ​​based on a difference between the adjacent Gaussian point and the centroids of the plurality of adjacent Gaussian points, and obtaining a covariance matrix of the three-dimensional Gaussian points based on an average of all the adjacent point difference values; Performing eigenvalue decomposition on the covariance matrix to obtain a plurality of eigenvalues, and calculating the non-scattering degree of each of the three-dimensional Gaussian points based on the plurality of eigenvalues; The main Gaussian point is selected from a plurality of the three-dimensional Gaussian points based on the non-scattering degree of the three-dimensional Gaussian point.

3. The method for generating a high-fidelity tree mesh model from multi-view images according to claim 2, characterized in that: The selecting the trunk Gaussian point from the plurality of three-dimensional Gaussian points based on the non-scattering degree of the three-dimensional Gaussian point includes: Selecting, from the plurality of three-dimensional Gaussian points, the three-dimensional Gaussian points whose non-scattering degree is greater than the preset scattering degree as the initial backbone Gaussian points; Based on the Euclidean distance between each of the initial backbone Gaussian points, clustering the multiple initial backbone points to obtain multiple initial backbone point clusters; An initial trunk point cluster having a number of cluster points greater than a preset number of cluster points is selected from the multiple initial trunk point clusters as a target trunk point cluster, and multiple initial trunk Gaussian points in the target trunk point cluster are used as the trunk Gaussian points.

4. The method for generating a high-fidelity tree mesh model from multi-view images according to claim 1, wherein: The clustering of the plurality of backbone Gaussian points based on the shortest path length between each backbone Gaussian point and the root node to obtain a plurality of backbone clusters includes: Generate an undirected weighted graph based on the plurality of backbone Gaussian points, and calculate the shortest path length between each backbone Gaussian point and a root node based on a shortest path fast algorithm; Based on the shortest path length between each of the backbone Gaussian points and the root node, the plurality of backbone Gaussian points are clustered to obtain a plurality of backbone clusters.

5. The method for generating a high-fidelity tree mesh model from multi-view images according to claim 1, characterized in that: The step of performing a thinning and connecting process based on the plurality of centroids to obtain a tree skeleton includes: Connecting the plurality of mass centers to obtain an initial tree skeleton of the target tree, wherein the initial tree skeleton includes a plurality of vertices, and every two adjacent vertices form a skeleton line segment; Based on the length of the subtree connected to each vertex in the initial tree skeleton, the vertex weight corresponding to each vertex is calculated; Selecting a thinning vertex from the plurality of vertices based on the vertex weight and the distance between the skeleton line segment; The tree skeleton is obtained based on the plurality of refined vertices.

6. The method for generating a high-fidelity tree mesh model from multi-view images according to claim 5, characterized in that: The step of generating a tree trunk model based on the plurality of three-dimensional Gaussian points and the tree skeleton comprises: For each of the trunk clusters in the tree skeleton, generating a cylindrical backbone; Obtaining a distance weight of each three-dimensional Gaussian point based on a ratio of a straight-line distance between each three-dimensional Gaussian point and the cylindrical backbone to a maximum straight-line distance; Generate a distance and objective function based on the accumulated value of the product of the distance weight and the straight-line distance corresponding to all the three-dimensional Gaussian points; Solving the distance and the objective function based on the least squares method to obtain the optimal cylinder radius, optimal endpoint, and optimal direction vector corresponding to the backbone cluster, and updating the cylindrical backbone based on the optimal cylinder radius, the optimal endpoint, and the optimal direction vector; The tree trunk model is generated based on all the updated cylindrical backbones.

7. The method for generating a high-fidelity tree mesh model from multi-view images according to claim 1, characterized in that: The method of performing crown screening on the plurality of three-dimensional Gaussian points based on the connection direction between each of the three-dimensional Gaussian points and the tree trunk model to obtain a plurality of crown points includes: For each branch skeleton point in the tree trunk model, based on the branch skeleton point, the parent node of the branch skeleton point, and the branch child nodes of the branch skeleton point, a bifurcation angle of each branch skeleton point is calculated; Obtaining a bifurcation angle interval based on a numerical relationship between the bifurcation angle and a preset bifurcation threshold; Based on the connection direction between each of the three-dimensional Gaussian points and each of the branch skeleton points, a growth direction of each of the three-dimensional Gaussian points is calculated; A three-dimensional Gaussian point whose angle corresponding to the growth direction is within the bifurcation angle interval is selected from the plurality of three-dimensional Gaussian points as the crown point.

8. The method for generating a high-fidelity tree mesh model from multi-view images according to claim 1, wherein: The step of obtaining multiple three-dimensional Gaussian points of the target tree includes: Acquire multi-view images of a target scene, wherein the target scene includes the target tree; Using three-dimensional point cloud technology, performing three-dimensional Gaussian extraction on the multi-view image to obtain multiple scene Gaussian points; Based on the low-dimensional features of each of the scene Gaussian points, the scene Gaussian points that are consistent with the low-dimensional features and the tree features are selected from the multiple scene Gaussian points as the three-dimensional Gaussian points.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the method for generating a high-fidelity tree mesh model from multi-view images according to any one of claims 1 to 8 when executing the computer program.

10. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for generating a high-fidelity tree mesh model from multi-view images according to any one of claims 1 to 8 is implemented.

Images