A method for instance segmentation of torreya grandis seedlings point cloud by fusing deep learning and adaptive clustering
By employing multi-view data acquisition, neural network semantic segmentation, and adaptive clustering optimization, the accuracy and efficiency issues of point cloud instance segmentation for Torreya grandis seedlings were resolved, achieving high-precision point cloud instance segmentation for Torreya grandis seedlings and promoting the automation and low-cost application of coniferous plant phenotypic analysis.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHEJIANG FORESTRY UNIVERSITY
- Filing Date
- 2026-03-30
- Publication Date
- 2026-06-26
AI Technical Summary
Existing technologies struggle to efficiently and accurately segment point cloud instances of Torreya grandis seedlings, especially when leaves are slender, spatially overlapping, and instance boundaries are blurred. Traditional methods are prone to undersegmentation or oversegmentation and rely heavily on labeled data and clear geometric boundaries, thus limiting the high-precision segmentation of Torreya grandis seedling point clouds.
High-quality point clouds are generated by multi-view data acquisition and 3D reconstruction. Semantic segmentation is performed by combining the dedicated neural network TorreyagrandisSegNet. The DBSCAN algorithm parameters are optimized by adaptive clustering and the skeleton topology is post-processed to achieve high-precision instance segmentation of Torreya grandis seedling point clouds.
It significantly improves the accuracy and efficiency of point cloud instance segmentation for Torreya grandis seedlings, with a semantic segmentation mIoU of 90.61% and an instance segmentation mAP_50 of 84.88%. It reduces equipment costs and improves the robustness and applicability of the segmentation process, making it suitable for high-throughput phenotypic analysis of other coniferous plants.
Smart Images
Figure CN122289683A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of plant phenotypic analysis technology, specifically to a method for segmenting point cloud instances of Torreya grandis seedlings by integrating deep learning and adaptive clustering. Background Technology
[0002] As an evergreen coniferous tree species of significant economic value, Torreya grandis exhibits marked phenotypic segregation and complex intraspecific phenotypic variation in its seedling offspring. Therefore, efficient and accurate phenotypic measurement at the seedling stage is crucial for selecting superior strains and genetic improvement. Traditional methods relying on manual measurement are not only inefficient and labor-intensive but also often destructive, failing to meet the demands for high-throughput phenotypic acquisition. In particular, the long, dense leaves of Torreya grandis, with severe shading between organs, lead to high errors and poor repeatability in manual measurements, becoming a key bottleneck restricting the improvement of breeding efficiency. With the development of crop breeding towards precision and intelligence, the development of non-destructive and automated phenotypic analysis technologies has become an urgent need in the industry.
[0003] In recent years, significant progress has been made in plant phenotypic analysis techniques based on 3D point clouds. Multi-view image reconstruction methods (such as Neural Radiation Field (NeRF) and 3D Gaussian Splashing (3DGS)) have gradually become mainstream data acquisition methods due to their low cost and high point cloud quality. In point cloud segmentation, deep learning models such as PointNet++ and DGCNN avoid voxelization computational redundancy by directly processing unordered point sets and have achieved good organ-level segmentation in broadleaf crops such as tomatoes and corn. Instance segmentation methods further distinguish individual organs within the same category, laying the foundation for multi-scale phenotypic analysis. However, these techniques are mostly aimed at plants with large or regularly structured leaves. For coniferous species such as Torreya grandis, whose leaves are small in scale, densely distributed, and have blurred instance boundaries, general models are prone to undersegmentation or oversegmentation. In addition, existing instance segmentation methods usually rely on a large amount of labeled data and clear geometric boundaries, making it difficult to directly adapt to the small sample size and high overlap of Torreya grandis seedling point cloud scenarios.
[0004] Segmentation of point cloud instances of Torreya grandis seedlings faces three core challenges: First, the leaves are slender and spatially overlapping, easily generating noise and uncertainty during 3D reconstruction, increasing the segmentation difficulty; second, the lack of clear geometric boundaries between instances makes traditional clustering algorithms (such as DBSCAN) parameter-sensitive, and fixed parameters are difficult to adapt to density variations in different plants; third, segmentation of overlapping leaves relies on topological structure analysis, while existing methods are insufficient in modeling the skeletal features of conifers. Although end-to-end deep learning methods perform well on general datasets, their high requirements for labeled data volume and instance clarity limit their application in complex scenarios such as Torreya grandis. Therefore, there is an urgent need for a segmentation framework that integrates deep learning feature learning and adaptive optimization mechanisms. Through targeted network design, parameter adaptive clustering, and topological post-processing, high-precision instance segmentation of Torreya grandis seedling point clouds can be achieved, providing a new approach for phenotypic analysis of conifers.
[0005] To address this, we propose a method for segmenting point cloud instances of Torreya grandis seedlings that integrates deep learning and adaptive clustering. Summary of the Invention
[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for segmenting point cloud instances of Torreya grandis seedlings by integrating deep learning and adaptive clustering, comprising the following steps:
[0007] S1: Multi-view data acquisition and 3D reconstruction: Multi-view image sequences of Torreya grandis seedlings were acquired using consumer-grade mobile devices in a surround shooting manner. Camera parameters were optimized by motion recovery structure algorithm and bundle adjustment, and dense point clouds were generated by 3D Gaussian sputtering technology. Simultaneously, scale recovery and point cloud denoising preprocessing were performed based on a calibration board.
[0008] S2: Semantic segmentation network inference: The preprocessed point cloud is input into a dedicated neural network TorreyagrandisSegNet, which integrates a dual attention kernel point convolution module and an adaptive augmented inverse residual MLP (AE_InvResMLP) module to achieve semantic segmentation of the point cloud of the trunk, hidden buds and lateral branches through multi-scale feature learning;
[0009] S3: Adaptive clustering initial segmentation: For the lateral leaf point cloud after semantic segmentation, the Newton-Raphson optimization strategy is used to automatically search for the optimal parameter combination (neighborhood radius Eps and minimum number of points MinPts) of the DBSCAN algorithm, and the instance-level initial segmentation of non-overlapping leaves is completed with the silhouette coefficient as the fitness index.
[0010] S4: Skeleton topology post-processing: For overlapping leaf areas, extract the point cloud skeleton topology structure, determine the segmentation boundary through leaf vein recognition and direction angle threshold, and realize refined instance separation of adhered leaves.
[0011] S5: Phenotypic parameter quantification: Based on instance segmentation results, calculate phenotypic traits such as plant volume, height, crown width, leaf area, and leaf tilt angle, and output structured data for breeding analysis.
[0012] Preferably, the multi-view data acquisition in step S1 specifically includes:
[0013] Record videos using a smartphone at 1920×1080 resolution and 60fps, taking shots from three heights: 5cm above the top of the plant, the middle of the plant, and 5cm below the surface of the pot. Extract 120 images from each seedling.
[0014] During 3D reconstruction, a calibration plate with a side length of 10cm was placed on the edge of the flowerpot as a geometric reference. Sparse point clouds were first generated by SfM and BA optimization using COLMAP software, and then dense point clouds were generated using 3DGS-30k. Point cloud preprocessing included noise removal based on statistical filtering, interactive flowerpot removal, and point cloud annotation to construct a dataset. Organ categories were divided into three types: main stem, latent bud, and lateral branches and leaves.
[0015] Preferably, the network structure of Torreya grandisSegNet in step S2 includes: DA_KPM module: sequentially executing channel attention mechanism and spatial attention mechanism, wherein the channel attention weights are calculated as follows:
[0016]
[0017] Spatial attention weights are calculated as follows:
[0018]
[0019] Finally, features are aggregated through KP convolution and enhanced by residual connections.
[0020] The AE_InvResMLP module generates residual signals through progressive feature enhancement. And combined with adaptive gating weights Fusion with main path features:
[0021]
[0022] Among them, the gating weight It is dynamically generated by global average pooling and two layers of convolution.
[0023] Preferably, the specific process for optimizing DBSCAN parameters using NRBO in step S3 includes:
[0024] Initialize the search ranges of Eps and MinPts, with their upper and lower bounds constrained by lb and ub. The initialization formula is as follows:
[0025]
[0026] The iterative update formula is:
[0027]
[0028] Where adaptive parameters
[0029] IT represents the current iteration number, and dim=2 indicates the parameter dimension;
[0030] Introducing the TAO operator for random perturbation:
[0031]
[0032] in and It is a random number. This is a perturbation term to avoid local optima;
[0033] With contour coefficient As a combination of fitness function evaluation parameters, The average distance within the class. This represents the minimum average distance between classes.
[0034] Preferably, the skeleton topology post-processing in step S4 includes:
[0035] Skeleton extraction: The point cloud is sparsified into a vertex set by Laplacian shrinkage and farthest point sampling (FPS), and a minimum spanning tree (MST) is constructed to form the skeleton graph;
[0036] Leaf vein identification: The Dijkstra algorithm is used to calculate the longest path from the root vertex to the top vertex. When the smoothness of the path length exceeds a set threshold, it is determined to be a leaf vein skeleton.
[0037] Overlapping segmentation: For overlapping bifurcation points with an angle ≥ 3, calculate the angle between the directions of adjacent skeleton segments. If θi < θth or π−θi < θth, then they are determined to be the same blade instance;
[0038] Point cloud assignment: The original point cloud is assigned to the segmented leaf skeleton by minimizing the Euclidean distance. The calculation formula is as follows:
[0039]
[0040] in Let be the skeleton point set of the i-th leaf instance.
[0041] Preferably, in the skeleton extraction, the number of neighbors K for the farthest point sampling is set to 15, the long side detection threshold is set to 0.1 meters, and the long side threshold factor is... The maximum value is set to 2.0 times the average nearest neighbor distance.
[0042] Preferably, the phenotypic parameter quantification method in step S5 includes:
[0043] Plant volume: calculated using the voxel method, voxel side length ,volume ,in The effective number of voxels;
[0044] Plant height: Based on the Z-axis extreme difference of the point cloud oriented bounding box. ;
[0045] Crown width: Project the point cloud onto the XY plane and calculate the maximum Euclidean distance of the vertex set of the convex hull.
[0046]
[0047] Leaf area: Generate a triangular mesh through greedy projection triangulation, sum the areas of all meshes, and calculate the area of a single triangle using Heron's formula;
[0048] Leaf tilt angle: Perform PCA analysis on the point clouds of the trunk and leaves to calculate the angle between the first principal vectors.
[0049] .
[0050] Compared with existing technologies, this invention provides a point cloud instance segmentation method for Torreya grandis seedlings that integrates deep learning and adaptive clustering, which has the following beneficial effects:
[0051] 1. This method for segmenting point cloud instances of Torreya grandis seedlings, integrating deep learning and adaptive clustering, achieves significant improvements in accuracy and efficiency through multi-level technological innovation. First, the point cloud generated by the 3D Gaussian sputtering reconstruction technology based on consumer-grade devices is rich in detail and low in noise, providing a high-quality data foundation for subsequent segmentation and overcoming the limitations of high cost and complex operation of traditional LiDAR equipment. Second, the TorreyagrandisSegNet network, specifically designed for coniferous plants, enhances the feature discrimination ability for slender leaves through the dual attention mechanism of the DA_KPM module, and effectively alleviates the feature degradation problem of deep networks by combining the progressive residual enhancement of the AE_InvResMLP module. Experiments show that the semantic segmentation structure mIoU of this network is 90.61%, which is about 30% higher than general models such as PointNet++, laying a high-precision foundation for instance segmentation.
[0052] 2. This method for segmenting point cloud instances of Torreya grandis seedlings, integrating deep learning and adaptive clustering, significantly improves the robustness of instance segmentation through the introduction of adaptive clustering and skeleton post-processing mechanisms. The NRBO-optimized DBSCAN algorithm automatically searches for the optimal parameter combination (Eps and MinPts) using the contour coefficients s(i), avoiding the over-segmentation or under-segmentation problems caused by parameter sensitivity in traditional clustering, thus improving clustering stability by 30%. Furthermore, skeleton topology analysis determines overlapping boundaries through leaf vein recognition and orientation angle thresholding, solving the failure problem of general end-to-end models in scenarios with blurred instance boundaries. The final instance segmentation mAP_50 reaches 84.88%, which is 16.68% and 5.66% higher than SoftGroup++ and PointTransformerV3 methods, respectively, and maintains clear instance separation even in highly overlapping regions.
[0053] 3. This point cloud instance segmentation method for Torreya grandis seedlings, integrating deep learning and adaptive clustering, boasts advantages of low cost and high applicability through its end-to-end design from data acquisition to phenotypic extraction. Image acquisition solutions using consumer-grade smartphones reduce equipment costs by over 90%, while the automated segmentation process shortens the phenotypic analysis time for a single seedling from several hours of traditional manual measurement to minutes. Parameters extracted based on the segmentation results, such as plant volume (voxel method) and leaf area (triangulation), show a high degree of agreement with measured values (R0). 2 The method (with a mean value >0.75 and RMSE <16%) provides reliable technical support for screening superior varieties of Torreya grandis, optimizing seedling management, and molecular-assisted breeding. This method can be extended to other coniferous plants, promoting the widespread application of high-throughput phenotypic analysis in precision breeding.
[0054] Figure 1 A flowchart of Torreya grandis dataset collection and preprocessing for a Torreya grandis seedling point cloud instance segmentation method that integrates deep learning and adaptive clustering;
[0055] Figure 2 TorreyagrandisSegNet network diagram for a point cloud instance segmentation method for Torreya grandis seedlings that integrates deep learning and adaptive clustering.
[0056] Figure 3 The block DA_KPM module structure diagram is a method for segmenting point cloud instances of Torreya grandis seedlings that integrates deep learning and adaptive clustering.
[0057] Figure 4 The AE_InvResMLP module structure diagram is a method for segmenting point cloud instances of Torreya grandis seedlings that integrates deep learning and adaptive clustering.
[0058] Figure 5This is a schematic diagram of overlapping leaf segmentation, which is a point cloud instance segmentation method for Torreya grandis seedlings that integrates deep learning and adaptive clustering.
[0059] Figure 6 A schematic diagram of a method for calculating phenotypic traits of Torreya grandis seedlings that integrates deep learning and adaptive clustering;
[0060] Figure 7 A comparison of semantic segmentation results for a point cloud instance segmentation method for Torreya grandis seedlings that integrates deep learning and adaptive clustering;
[0061] Figure 8 Comparison of instance segmentation results for a point cloud instance segmentation method for Torreya grandis seedlings that integrates deep learning and adaptive clustering;
[0062] Figure 9 The image shows the extracted phenotypic parameters of a point cloud instance segmentation method for Torreya grandis seedlings that integrates deep learning and adaptive clustering. Detailed Implementation
[0063] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0064] Example
[0065] An embodiment of a point cloud instance segmentation method for Torreya grandis seedlings that integrates deep learning and adaptive clustering.
[0066] A method for segmenting point cloud instances of Torreya grandis seedlings by integrating deep learning and adaptive clustering includes the following steps:
[0067] S1: Multi-view data acquisition and 3D reconstruction: Multi-view image sequences of Torreya grandis seedlings were acquired using consumer-grade mobile devices in a surround shooting manner. Camera parameters were optimized by motion recovery structure algorithm and bundle adjustment, and dense point clouds were generated by 3D Gaussian sputtering technology. Simultaneously, scale recovery and point cloud denoising preprocessing were performed based on a calibration board.
[0068] S2: Semantic segmentation network inference: The preprocessed point cloud is input into a dedicated neural network TorreyagrandisSegNet, which integrates a dual attention kernel point convolution module and an adaptive augmented inverse residual MLP (AE_InvResMLP) module to achieve pixel-level semantic segmentation of the trunk, hidden buds and lateral branches through multi-scale feature learning.
[0069] S3: Adaptive clustering initial segmentation: For the lateral leaf point cloud after semantic segmentation, the Newton-Raphson optimization strategy is used to automatically search for the optimal parameter combination (neighborhood radius Eps and minimum number of points MinPts) of the DBSCAN algorithm, and the instance-level initial segmentation of non-overlapping leaves is completed with the silhouette coefficient as the fitness index.
[0070] S4: Skeleton topology post-processing: For overlapping leaf areas, extract the point cloud skeleton topology structure, determine the segmentation boundary through leaf vein recognition and direction angle threshold, and realize refined instance separation of adhered leaves.
[0071] S5: Phenotypic parameter quantification: Based on instance segmentation results, calculate phenotypic traits such as plant volume, height, crown width, leaf area, and leaf tilt angle, and output structured data for breeding analysis.
[0072] Specifically, the multi-view data acquisition in step S1 includes:
[0073] Record videos using a smartphone at 1920×1080 resolution and 60fps, taking shots from three heights: 5cm above the top of the plant, the middle of the plant, and 5cm below the surface of the pot. Extract 120 images from each seedling.
[0074] During 3D reconstruction, a calibration plate with a side length of 10cm was placed on the edge of the flowerpot as a geometric reference. Sparse point clouds were first generated by SfM and BA optimization using COLMAP software, and then dense point clouds were generated using 3DGS. Point cloud preprocessing included noise removal based on statistical filtering, interactive flowerpot removal, and point cloud annotation to construct a dataset. Organ categories were divided into three types: main stem, latent bud, and lateral branches and leaves.
[0075] Specifically, the Torreya grandisSegNet network structure in step S2 includes: the DA_KPM module: which sequentially executes the channel attention mechanism and the spatial attention mechanism, wherein the channel attention weights are calculated as follows:
[0076]
[0077] Spatial attention weights are calculated as follows:
[0078]
[0079] Finally, features are aggregated through KP convolution and enhanced by residual connections.
[0080] The AE_InvResMLP module generates residual signals through progressive feature enhancement. And combined with adaptive gating weights Fusion with main path features:
[0081]
[0082] Among them, the gating weight It is dynamically generated by global average pooling and two layers of convolution.
[0083] Specifically, the process of optimizing DBSCAN parameters using NRBO in step S3 includes:
[0084] Initialize the search ranges of Eps and MinPts, with their upper and lower bounds determined by lb and ub constraints. The initialization formula is as follows:
[0085]
[0086] The iterative update formula is:
[0087]
[0088] Where adaptive parameters
[0089] IT represents the current iteration number, and dim=2 indicates the parameter dimension;
[0090] Introducing the TAO operator for random perturbation:
[0091]
[0092] in and It is a random number. This is a perturbation term to avoid local optima;
[0093] With contour coefficient As a combination of fitness function evaluation parameters, The average distance within the class. This represents the minimum average distance between classes.
[0094] Specifically, the skeleton topology post-processing in step S4 includes:
[0095] Skeleton extraction: The point cloud is sparsified into a vertex set by Laplacian shrinkage and farthest point sampling (FPS), and a minimum spanning tree (MST) is constructed to form the skeleton graph;
[0096] Leaf vein identification: The Dijkstra algorithm is used to calculate the longest path from the root vertex to the top vertex. When the smoothness of the path length exceeds a set threshold, it is determined to be a leaf vein skeleton.
[0097] Overlapping segmentation: For overlapping bifurcation points with an angle ≥ 3, calculate the angle between the directions of adjacent skeleton segments. If θi < θth or π−θi < θth, then they are determined to be the same blade instance;
[0098] Point cloud assignment: The original point cloud is assigned to the segmented leaf skeleton by minimizing the Euclidean distance. The calculation formula is as follows:
[0099]
[0100] in Let be the skeleton point set of the i-th leaf instance.
[0101] Specifically, in skeleton extraction, the number of neighbors K for the farthest point sampling is set to 15, the long side detection threshold is set to 0.1 meters, and the long side threshold factor is... The maximum value is set to 2.0 times the average nearest neighbor distance.
[0102] Specifically, the phenotypic parameter quantification method in step S5 includes:
[0103] Plant volume: calculated using the voxel method, voxel side length ,volume ,in The effective number of voxels;
[0104] Plant height: Based on the Z-axis extreme difference of the point cloud oriented bounding box. ;
[0105] Crown width: Project the point cloud onto the XY plane and calculate the maximum Euclidean distance of the vertex set of the convex hull.
[0106] ;
[0107] Leaf area: Generate a triangular mesh through greedy projection triangulation, sum the areas of all meshes, and calculate the area of a single triangle using Heron's formula;
[0108] Leaf tilt angle: Perform PCA analysis on the point clouds of the trunk and leaves to calculate the angle between the first principal vectors.
[0109] .
[0110] Through the above technical solutions, this invention achieves a significant improvement in the accuracy and efficiency of point cloud instance segmentation for Torreya grandis seedlings through multi-level technological innovation. First, the point cloud generated by the 3D Gaussian sputtering reconstruction technology based on consumer-grade devices is rich in detail and low in noise, providing a high-quality data foundation for subsequent segmentation and overcoming the limitations of high cost and complex operation of traditional LiDAR equipment. Second, the Torreya grandisSegNet network, specifically designed for coniferous plants, enhances the feature discrimination ability for slender leaves through the dual attention mechanism of the DA_KPM module, and effectively alleviates the feature degradation problem of deep networks by combining the progressive residual enhancement of the AE_InvResMLP module. Experiments show that the semantic segmentation result mIoU of this network reaches 90.61%, which is about 30% higher than general models such as PointNet++, laying a high-precision foundation for instance segmentation. The introduced adaptive clustering and skeleton post-processing mechanisms significantly improve the robustness of instance segmentation. The NRBO-optimized DBSCAN algorithm automatically searches for the optimal parameter combination (Eps and MinPts) using the silhouette coefficient s(i), avoiding the oversegmentation or undersegmentation problems caused by parameter sensitivity in traditional clustering, thus improving clustering stability by 20%. Furthermore, skeleton topology analysis determines overlapping boundaries through leaf vein recognition and orientation angle thresholding, solving the failure problem of general end-to-end models in scenarios with blurred instance boundaries. The final instance segmentation mAP_50 reaches 84.88%, which is 16.68% and 5.66% higher than the current state-of-the-art end-to-end deep learning instance methods SoftGroup++ and PointTransformerV3, respectively. It maintains clear instance separation even in highly overlapping regions. Through a complete workflow design from data acquisition to phenotypic extraction, it combines low cost and high applicability. Image acquisition solutions for consumer smartphones reduce equipment costs by over 90%, while the automated segmentation process shortens the time for single seedling phenotypic analysis from several hours of traditional manual measurement to minutes. Parameters such as plant volume (voxel method) and leaf area (triangulation) extracted based on the segmentation results are highly consistent with the measured values (R...). 2 The results (value >0.75, RMSE <16) provide reliable technical support for screening superior varieties of Torreya grandis, optimizing seedling management, and molecular-assisted breeding. This method can be extended to other coniferous plants, promoting the widespread application of high-throughput phenotypic analysis in precision breeding.
[0111] 1. Data Acquisition and Preprocessing
[0112] Implementation steps:
[0113] (1) Sample preparation: Select two-year-old Torreya grandis seedlings with uniform growth to ensure that the plants are healthy and free from pests and diseases.
[0114] (2) Image acquisition: A consumer-grade smartphone, iPhone 14 Plus, was used with a resolution of 1920×1080 pixels and a frame rate of 60fps. The operator filmed videos from three different heights: about 5cm above the top of the seedling, about 5cm below the middle of the plant, and about 5cm below the surface of the pot. Three video sequences were collected for each seedling. After frame extraction using OpenCV, 40 frames were extracted from each path, resulting in a total of 120 multi-view images per seedling.
[0115] (3) Scale calibration: Place a calibration plate with a side length of 10cm on the edge of the flowerpot as a geometric reference for three-dimensional reconstruction to ensure that the subsequent point cloud has a real scale.
[0116] (4) Data preprocessing: After the video frames are extracted, the images are subjected to brightness equalization and deblurring to avoid reconstruction errors.
[0117] 2. 3D Reconstruction and Point Cloud Generation
[0118] Implementation steps:
[0119] (1) Sparse reconstruction: The SfM algorithm in COLMAP software is used to extract SIFT feature points from 120 input images and perform feature matching. Sparse point clouds are generated by triangulation, and the initial pose of the camera is estimated. Then, bundle adjustment (BA) is used to jointly optimize the camera parameters (intrinsic and extrinsic parameters) and the 3D point structure to obtain accurate reconstruction results.
[0120] (2) Dense Reconstruction: Based on the optimized camera parameters, dense reconstruction was performed using 3DGS (3D Gaussian Sputtering). 3DGS represents the scene as a set of 3D Gaussian distributions (including position, covariance, color, and opacity). Through differentiable Gaussian projection and rendering optimization, smooth and detailed dense point clouds were generated within 30,000 training iterations (3DGS-30k). The training environment was an NVIDIA RTX 4090 GPU, and the reconstruction time for a single seedling was approximately 20 minutes.
[0121] (3) Point cloud post-processing: CloudCompare software was used to perform noise filtering (statistical outlier removal algorithm) and background removal (manual interactive deletion of non-target parts such as flower pots) on the generated point cloud, and finally pure Torreya grandis seedling point cloud data was obtained.
[0122] 3. Semantic Segmentation Network Design and Training
[0123] Implementation details:
[0124] (1) Network architecture: Torreya grandisSegNet, which is an improvement on PointNeXt, and its core includes the DA_KPM module and the AE_InvResMLP module. The input point cloud has a size of 40,960 points and a feature dimension D=256.
[0125] (2) DA_KPM module:
[0126] Channel attention: input features Perform average pooling and max pooling to obtain and Weights are generated by sharing MLP and Sigmoid function. The calculation formula is:
[0127]
[0128] Spatial attention: Max pooling and average pooling are performed on the enhanced features f′xi, and the concatenation is followed by MLP to generate spatial weights MK, with the formula as follows:
[0129]
[0130] KPConv aggregation: Uses KPConv to aggregate neighborhood features, and finally outputs the features through residual connections. .
[0131] (3) AE_InvResMLP module:
[0132] Progressive reinforcement: through KNN (number of neighbors) =16) Aggregate neighborhood features, generate residual signal Fpe through linear mapping and FFN, the formula is:
[0133]
[0134] Adaptive gating: for main path features Global average pooling is performed, and gated weights s are generated through two layers of Conv1×1 and Sigmoid. The final output is:
[0135]
[0136] (4) Training configuration:
[0137] Hardware environment: Intel Core i9-14900KF CPU, NVIDIA RTX 4090 GPU.
[0138] Hyperparameters: Batch size = 8, Learning rate = 0.001 (cosine decay), Optimizer = AdamW, Loss function = Labeled smooth cross-entropy loss.
[0139] Data augmentation: random scaling (range [0.8, 1.2]), rotation (gravity dimension = 2), translation (±0.02m), and jitter (standard deviation = 0.01).
[0140] Dataset: 51 samples (38 strains in the training set and 13 strains in the test set), trained for 300 epochs.
[0141] 4. Instance Segmentation: Adaptive Clustering and Backbone Post-processing
[0142] Implementation steps:
[0143] NRBO-optimized DBSCAN clustering:
[0144] Parameter search: For the semantically segmented lateral point cloud, initialize the search ranges of Eps and MinPts (lb=[0.010,0.020], ub=[2,15]). Update the parameters iteratively using Newton-Raphson optimization.
[0145]
[0146] Where adaptive parameters The maximum number of iterations is Max_IT=100.
[0147] To avoid local optima, the TAO operator is introduced for perturbation, as shown in the formula:
[0148]
[0149] Optimal parameter selection: based on profile coefficient For the fitness function, screening makes The optimal combination of parameters.
[0150] (2) Post-processing of skeleton topology (for overlapping blades):
[0151] Skeleton extraction: Input point cloud After Laplace shrinkage, the graph is sparsified to a vertex set U using farthest point sampling (FPS), and a minimum spanning tree (MST) is constructed to form a skeleton graph G=(U,E). Parameter settings: long side threshold = 0.1m.
[0152] Leaf vein identification: The Dijkstra algorithm is used to calculate the longest path from the root vertex to the top vertex. When the path length smoothness exceeds the threshold (the path smoothness is set to 0.7), it is identified as a leaf vein.
[0153] Overlapping segmentation: For bifurcation points with a degree ≥ 3, calculate the angle between the directions of adjacent skeleton segments. A directional consistency threshold of θth = 30° is set. If θi < θth or π − θi < θth, it is determined to be the same blade.
[0154] Point cloud assignment: The original point cloud is assigned to leaf instances by minimizing the Euclidean distance. The calculation formula is as follows:
[0155]
[0156] in Let be the skeleton point set of the i-th leaf instance.
[0157] 5. Phenotypic parameter extraction and validation
[0158] Implementation steps:
[0159] (1) Volume calculation: Using the voxel method, the point cloud OBB space is divided into voxels with a side length of d = 0.001m. The number of effective voxels N is counted, and the volume is calculated. .
[0160] (2) Plant height and crown width: Plant height H = Zmax − Zmin, where Zmax and Zmin are the extreme values of the Z-axis of the point cloud OBB; Crown width is calculated by projecting the point cloud onto the XY plane and calculating the maximum Euclidean distance between the convex hull vertices:
[0161]
[0162] (3) Leaf area: Generate a triangular mesh using greedy projection triangulation, calculate the area of each triangle using Heron's formula, and then sum them up:
[0163]
[0164] (4) Leaf tilt angle: PCA analysis was performed on the point clouds of the trunk and the leaves respectively to extract the first principal vector. and Calculate the included angle:
[0165] (5) Accuracy verification: Calculate the coefficient of determination based on manual measurement values. And the root mean square error (RMSE), this method >0.90, RMSE <5%.
[0166] 6. Results Analysis and Performance Evaluation
[0167] Implementation results:
[0168] (1) Segmentation performance: The semantic segmentation mIoU reached 90.61%, and the instance segmentation mAP_50 was 84.11, which is significantly better than PointNet++ (mIoU=66.98%) and DGCNN (mIoU=79.63%).
[0169] (2) Efficiency comparison: The entire process of processing a single seedling takes about 22 minutes (including 20 minutes for reconstruction, 1 minute for segmentation, and 1 minute for phenotypic extraction), which is more than 10 times more efficient than manual measurement.
[0170] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A method for segmenting point cloud instances of Torreya grandis seedlings by integrating deep learning and adaptive clustering, characterized in that: Includes the following steps: S1: Multi-view data acquisition and 3D reconstruction: Multi-view image sequences of Torreya grandis seedlings were acquired using consumer-grade mobile devices in a surround shooting manner. Camera parameters were optimized by motion recovery structure algorithm and bundle adjustment, and dense point clouds were generated by 3D Gaussian sputtering technology. Simultaneously, scale recovery and point cloud denoising preprocessing were performed based on a calibration board. S2: Semantic segmentation network inference: The preprocessed point cloud is input into a dedicated neural network TorreyagrandisSegNet, which integrates a dual attention kernel point convolution module and an adaptive augmented inverse residual MLP (AE_InvResMLP) module to achieve semantic segmentation of the point cloud of the trunk, hidden buds and lateral branches through multi-scale feature learning; S3: Adaptive clustering initial segmentation: For the lateral leaf point cloud after semantic segmentation, the Newton-Raphson optimization strategy is used to automatically search for the optimal parameter combination (neighborhood radius Eps and minimum number of points MinPts) of the DBSCAN algorithm, and the instance-level initial segmentation of non-overlapping leaves is completed with the silhouette coefficient as the fitness index. S4: Skeleton topology post-processing: For overlapping leaf areas, extract the point cloud skeleton topology structure, determine the segmentation boundary through leaf vein recognition and direction angle threshold, and realize refined instance separation of adhered leaves. S5: Phenotypic parameter quantification: Based on instance segmentation results, calculate phenotypic traits such as plant volume, height, crown width, leaf area, and leaf tilt angle, and output the results for breeding analysis.
2. The method according to claim 1, wherein the method is characterized in that: The multi-view data acquisition in step S1 specifically includes: Record videos using a smartphone at 1920×1080 resolution and 60fps, taking shots from three heights: 5cm above the top of the plant, the middle of the plant, and 5cm below the surface of the pot. Extract 120 images from each seedling. During 3D reconstruction, a calibration plate with a side length of 10cm was placed on the edge of the flowerpot as a geometric reference. Sparse point clouds were first generated by SfM and BA optimization using COLMAP software, and then dense point clouds were generated using 3DGS. Point cloud preprocessing included noise removal based on statistical filtering, interactive flowerpot removal, and point cloud annotation to construct a dataset. Organ categories were divided into three types: main stem, latent bud, and lateral branches and leaves.
3. The method for segmenting point cloud instances of Torreya grandis seedlings by integrating deep learning and adaptive clustering according to claim 1, characterized in that: The Torreya grandisSegNet network structure in step S2 includes: DA_KPM module: sequentially executing channel attention mechanism and spatial attention mechanism, wherein the channel attention weights are calculated as follows: Spatial attention weights are calculated as follows: Finally, features are aggregated through KP convolution and enhanced by residual connections. The AE_InvResMLP module generates residual signals through progressive feature enhancement. And combined with adaptive gating weights Fusion with main path features: Among them, the gating weight It is dynamically generated by global average pooling and two layers of convolution.
4. The method for segmenting point cloud instances of Torreya grandis seedlings by integrating deep learning and adaptive clustering as described in claim 1, characterized in that: The specific process for optimizing DBSCAN parameters using NRBO in step S3 includes: Initialize the search ranges of Eps and MinPts, with their upper and lower bounds constrained by lb and ub. The initialization formula is as follows: The iterative update formula is: Where adaptive parameters IT represents the current iteration number, and dim=2 indicates the parameter dimension; Introducing the TAO operator for random perturbation: in and It is a random number. This is a perturbation term to avoid local optima; With contour coefficient As a combination of fitness function evaluation parameters, The average distance within the class. This represents the minimum average distance between classes.
5. The method for segmenting point cloud instances of Torreya grandis seedlings by integrating deep learning and adaptive clustering according to claim 1, characterized in that: The skeleton topology post-processing in step S4 includes: Skeleton extraction: The point cloud is sparsified into a vertex set by Laplacian shrinkage and farthest point sampling (FPS), and a minimum spanning tree (MST) is constructed to form the skeleton graph; Leaf vein identification: The Dijkstra algorithm is used to calculate the longest path from the root vertex to the top vertex. When the smoothness of the path length exceeds a set threshold, it is determined to be a leaf vein skeleton. Overlap segmentation: For overlapping bifurcation points with an angle ≥ 3, calculate the angle between the directions of adjacent blade sub-skeletal segments. If θi < θth or π−θi < θth, then they are determined to be the same blade instance; Point cloud assignment: The original point cloud is assigned to the segmented leaf skeleton by minimizing the Euclidean distance. The calculation formula is as follows: wherein is the set of skeleton points for the i-th leaf instance.
6. The method of claim 1, wherein the method further comprises: determining a plurality of clusters of the plurality of points based on the plurality of point cloud features; and determining a plurality of point cloud instances based on the plurality of clusters. The phenotypic parameter quantification method in step S5 includes: Plant volume: calculated using the voxel method, voxel side length ,volume ,in The effective number of voxels; Plant height: Based on the Z-axis extreme difference of the point cloud oriented bounding box. ; Crown width: Project the point cloud onto the XY plane and calculate the maximum Euclidean distance of the vertex set of the convex hull. ； Leaf area: Generate a triangular mesh through greedy projection triangulation, sum the areas of all meshes, and calculate the area of a single triangle using Heron's formula; Leaf tilt angle: Perform PCA analysis on the point clouds of the trunk and leaves to calculate the angle between the first principal vectors. 。