Wheat organ three-dimensional point cloud segmentation method based on dynamic voxel and category perception

Abstract

The invention discloses a wheat organ three-dimensional point cloud segmentation method based on dynamic voxel and category perception. Three-dimensional reconstruction is carried out on a wheat sample based on a three-dimensional Gaussian splash method, and dense point cloud data are obtained; performing data amplification on the dense point cloud data by adopting a dynamic voxel rasterization random sampling method combining dynamic voxel division and a random farthest point sampling strategy, and performing standardized point cloud quantity processing to obtain a standard point cloud data set; and based on the standard point cloud data set, carrying out key organ segmentation by adopting a category perception segmentation network. According to the data enhancement method, key geometric features are maintained while data diversity is expanded, a feature extraction module of the category perception segmentation network solves the problem of ignoring important points during sampling by fusing DCA and a weighted farthest point sampling strategy, and an improved loss function combining a point cloud space structure and information is adopted to improve the accuracy of data enhancement. And the segmentation precision of the wheat complex point cloud data is effectively improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image recognition, relates to wheat organ recognition, and in particular to a three-dimensional point cloud segmentation method for wheat organs based on dynamic voxels and category perception. Background Art

[0002] The segmentation and precise identification of key organs such as wheat flag leaves and ears can provide key technical support for wheat phenotypic analysis and the breeding of high-yield and high-quality wheat varieties.

[0003] Traditional methods for measuring plant phenotypic parameters rely on manual measurement and approximate calculations, which are subject to large errors, low efficiency, and destructiveness, making them difficult to meet the needs of large-scale, high-throughput phenotyping. In recent years, the integration of computer vision and deep learning has been widely used in phenotyping analysis using two-dimensional images. However, two-dimensional images lack depth information and have difficulty handling complex occlusions. In contrast, three-dimensional point clouds contain not only richer texture information but also the most critical depth information, effectively breaking through the bottleneck of two-dimensional images. Kolodiazhnyi et al. converted irregularly structured point clouds into regular voxel grids and achieved furniture segmentation based on a three-dimensional convolutional network. Xiang et al. revealed the hierarchical structure of sorghum stems, leaves, and tillers by simplifying point clouds into topological skeletons. Qi et al. pioneered the PointNet deep learning framework, which achieved an end-to-end framework for unordered point cloud segmentation through a permutation-invariant processing mechanism. Li et al. introduced spatial and channel attention mechanisms in PSegNet and PlantNet networks, providing new insights for point cloud segmentation. Although the current research on crop point cloud segmentation has made some progress, the complex point cloud data of wheat, especially wheat point cloud, still faces difficulties in segmentation: (1) Traditional point cloud downsampling methods are prone to loss of key organ features in complex plant scenes, affecting the subsequent segmentation accuracy; (2) The existing segmentation network still has deficiencies in the joint modeling of spatial features and information, and it is difficult to deal with the mutual occlusion problem between dense leaves and wheat ears in the wheat canopy. Summary of the Invention

[0004] In order to overcome the shortcomings of the above-mentioned prior art, the purpose of the present invention is to provide a three-dimensional point cloud segmentation method for wheat organs based on dynamic voxels and category perception, so as to solve the problems of low accuracy and efficiency in key organ segmentation caused by the small amount of wheat point cloud data, dense organs, and severe occlusion.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] A three-dimensional point cloud segmentation method for wheat organs based on dynamic voxels and category perception includes the following steps:

[0007] Step 1: Reconstruct the wheat sample in 3D based on the 3D Gaussian splash method and obtain dense point cloud data;

[0008] Step 2: A dynamic voxel rasterization random sampling method combining dynamic voxel partitioning and random farthest point sampling strategy is used to augment the dense point cloud data, and a standard point cloud dataset is obtained by standardizing the number of point clouds.

[0009] Step 3: Based on the standard point cloud dataset, a category-aware segmentation network is used to segment key organs.

[0010] In one embodiment, in step 1, the number of wheat samples does not exceed 100, and the image dataset thereof is obtained for 3D reconstruction by the following method:

[0011] Using a surround-view shooting method, video data was acquired from three observation angles: the upper part of the wheat canopy, the middle of the ear, and the lower part of the base. Sequential images were extracted at a set frame rate to construct a multi-view image dataset covering 360° of wheat space.

[0012] After obtaining the dense point cloud data, it is labeled and divided into wheat ears, flag leaves and other categories, and the wheat sample point cloud data below the threshold is eliminated based on the peak signal-to-noise ratio.

[0013] In one embodiment, the step 2, dynamic voxel partitioning, is implemented by dynamic voxel downsampling, as follows:

[0014] Step 211 , constructing a voxel grid of the point cloud based on an octree hierarchical adaptive mechanism, where each voxel grid retains the coordinates and information of all points within it;

[0015] In step 212, the point cloud coordinates in each voxel grid are averaged to generate a centroid point, and the label is determined by the majority voting principle to achieve perceptual centroid sampling.

[0016] In one embodiment, step 211 constructs a voxel grid of the point cloud based on an octree hierarchical adaptive mechanism, and the implementation method is as follows:

[0017] Set up the octree hierarchy and dynamically calculate the voxel size of each point cloud, thereby dividing the original point cloud into a uniform cubic voxel grid. The pixel size is calculated as follows:

[0018]

[0019] Where Cell_size is the side length of the voxel, L is the side length of the maximum bounding box of the point cloud, and l is the octree level.

[0020] In one embodiment, the random farthest point sampling strategy in step 2 is as follows:

[0021] (1) Randomly select the initial point as the sampling point, then screen the K candidate points farthest from the current sampling point set, and randomly select the next sampling point from them to alleviate the initial point dependence, where K is the data amplification factor;

[0022] (2) By calculating the neighborhood density weight of the candidate points, feature points in sparse areas are selected to avoid redundant sampling in dense areas;

[0023] (3) Perform F independent random farthest point samplings on each dynamic voxelization downsampling result to ensure that the initial point selected each time is random and non-repetitive, thereby ensuring data diversity, where F is the data amplification factor.

[0024] In one embodiment, during the encoding phase, the feature extraction module of the category-aware segmentation network performs hierarchical local feature extraction of the point cloud by: selecting a central point according to its importance score during downsampling; using a multi-scale radius query strategy during grouping to enhance adaptability to the non-uniform density distribution of the point cloud; and obtaining local features through MLP and maximum pooling operations during feature extraction.

[0025] In the decoding stage, the point cloud resolution is gradually restored through the feature propagation module combined with the skip connection structure;

[0026] During the feature propagation process, the KNN weighted interpolation strategy is adopted to pass the features of the current layer upward to the previous layer, splice the interpolated features with the original features of the previous layer, and complete feature fusion through MLP; finally, the classification probability of each point is output to achieve point-level segmentation.

[0027] In one embodiment, the category-aware segmentation network adopts a category-aware local feature extraction module for feature extraction. The local feature extraction module adopts a two-stage category attention mechanism and a weighted farthest point sampling strategy. The two-stage category attention mechanism integrates multi-head self-attention and category-specific attention. Multi-head self-attention captures diverse, complex local and global information in point cloud data. Category-specific attention uses the separate spatial attention of each category to generate different category attention scores for each point in the point cloud to select the center point with higher value. The weighted farthest point sampling strategy integrates the category attention score and the geometric distance score to measure the importance of the point, thereby achieving sampling that is coordinated with spatial optimization.

[0028] In one embodiment, the two-stage category attention mechanism firstly focuses on the input point cloud features f in The feature f is obtained by modeling the global point relationship through the multi-head self-attention layer mth , the feature f mth Mapping to category space via MLP Get feature f Class , where N is the number of points in the input point cloud; then, a double pooling fusion strategy is used to calculate the category-specific attention, where the average pooling extracts the overall information of the category and the maximum pooling focuses on the most representative points of the category, respectively obtaining the feature vectors f avg and f max , add the results of average pooling and maximum pooling to get the global category attention A Class , by splicing A Class and f Class Get the category feature f of each point fusion,i , and then use MLP to get the final category attention A of each point point ; Finally, for A point Normalize to get the attention score A of each point norm .

[0029] In one embodiment, the weighted farthest point sampling strategy is implemented as follows:

[0030] Randomly select the initial center point, initialize the global minimum distance matrix, build a set of sampling points, calculate the Euclidean distance from the current center point to all points, update the global minimum distance and normalize it, then calculate the importance score, select the point with the highest score as the new sampling point, and update the sampling point set; the importance score of each point in the point cloud is defined as:

[0031] I i =αA norm,i +(1-α)D norm,i

[0032] Where D norm,i is the shortest Euclidean distance from the i-th point to the currently selected point set after normalization, α is the balance coefficient of the category attention score, and (1-α) is the balance coefficient of the geometric distance score.

[0033] In one embodiment, the category-aware segmentation network uses a composite loss function L that combines the spatial structure and information of the point cloud. CIR , which is determined by the cross entropy loss L CE , complete intersection loss L CIoU and feature regularization loss L reg Composition, expressed as: L CIR =βL CE +γL CIoU +δL reg , where β, γ and δ are the weights of each loss respectively.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] (1) A DVFPS data enhancement method is proposed to expand data diversity while maintaining key geometric features.

[0036] (2) The CAFE module is designed in the encoding stage, which improves the problem of ignoring important points during sampling by integrating DCA and weighted farthest point sampling strategy.

[0037] (3) An improved loss function that combines the spatial structure and information of point clouds is used to effectively improve the segmentation accuracy of complex point cloud data of wheat.

[0038] The experimental results show that the method of the present invention achieved 62.28%, 76.41%, 72.63% and 74.47% in mIoU, Precision, Recall and F1-score, respectively, which are all superior to methods such as PointNet++ and PlantNet. At the same time, ablation experiments also verified the effectiveness of DVFPS, CAFE and improved loss functions in improving model performance. The R2 of key organ phenotypes such as wheat ear volume and leaf area calculated based on the segmentation results reached above 0.75, further verifying the effectiveness of this method in phenotypic analysis. The present invention effectively improves the segmentation accuracy of key organs in wheat point clouds, providing a reliable technical means for wheat phenotypic detection and precision breeding in smart agriculture. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a schematic diagram of the process of wheat three-dimensional reconstruction of the present invention.

[0040] Figure 2 It is a schematic diagram of the process of constructing voxels from the point cloud octree of the present invention.

[0041] Figure 3 This is a schematic diagram of the wheat point cloud data downsampling process of the present invention.

[0042] Figure 4 It is a schematic diagram of the CA-SegNet network architecture of the present invention.

[0043] Figure 5 Schematic diagram of the dual-stage category attention structure of the present invention.

[0044] Figure 6 2 is a schematic diagram comparing the downsampling results of the present invention.

[0045] Figure 7 The following is a visualization diagram of the wheat point cloud data segmentation results of the present invention. (a) is a normal wheat sample with clear structure and no obvious occlusion, and (b) is a complex wheat sample with overlapping leaves and ears, which occlude each other.

[0046] Figure 8 Schematic diagram of adjustment of the balance coefficient α in the CAFE module of the present invention.

[0047] Figure 9 It is a regression analysis diagram of the wheat ear volume control value and the calculated value of the present invention. DETAILED DESCRIPTION

[0048] The embodiments of the present invention are described in detail below with reference to the accompanying drawings and examples.

[0049] Accurate identification of wheat organs, such as flag leaves and ears, is crucial for phenotypic analysis and the breeding of high-yield varieties. Existing techniques face difficulties in segmenting key organs from complex wheat point cloud data due to sampling difficulties and limited data. To address this, the present invention constructs a data augmentation strategy and an improved Class-Aware Segmentation Network (CA-SegNet), achieving 3D point cloud segmentation of wheat organs based on dynamic voxels and class awareness.

[0050] The main process of the present invention can be described as:

[0051] Firstly, a small number of wheat plants (no more than 100) were 3D reconstructed based on 3D Gaussian Splatting (3DGS) to obtain dense point cloud data.

[0052] Secondly, in order to enhance sample diversity and maintain key geometric structures, a dynamic voxel farthest point sampling (DVFPS) method was proposed. By combining dynamic voxel partitioning with random farthest point sampling strategy, a 10-fold expansion of the dataset and standardized point cloud number (i.e., 8192 points) was achieved.

[0053] Finally, taking the standard point cloud dataset as input, an improved point cloud segmentation network CA-SegNet was proposed. Through the Class-Aware Feature Extraction (CAFE) module and an improved loss function that combines classification accuracy, spatial consistency, and feature constraints, the key organ segmentation performance was effectively improved.

[0054] In a specific embodiment of the present invention, two wheat varieties, "Xinnong 877" and "Xiaoyan 22", were selected as experimental materials, and 40 pots of cultivation samples were set for each, totaling 80 pots. All samples were cultivated in the research greenhouse of the National Key Laboratory of Crop Resistance and Efficient Production of Northwest Agriculture and Forestry University (34°7′39″N, 107°59′50″E, 648m above sea level). In view of the complex phenotypic characteristics of wheat and the problem of mutual occlusion between plants, the present invention established a multi-perspective image acquisition system: a surround shooting method was adopted to obtain video data from three observation perspectives: the upper part of the plant canopy, the middle part of the ear, and the lower part of the base. The sequence images were extracted by sampling at a frame rate of 2 frames per second to construct a multi-perspective image dataset covering 360° of the plant's stereoscopic space.

[0055] Based on this dataset, the 3DGS algorithm is used to achieve three-dimensional reconstruction of wheat plants. Figure 1 As shown in the figure, this method effectively solves the problem of missing point clouds caused by plant organ self-occlusion through differentiable rendering technology. During the reconstruction process, the key parameters are set as follows: the number of iterations is 30,000, the order of the spherical harmonic function is 3, the opacity learning rate is 0.025, and the rest of the parameters remain at their default values.

[0056] The present invention accurately annotates wheat point cloud data based on the CloudCompare tool. Before annotation, since the original wheat point cloud data is generated by multi-view video reconstruction, it will be accompanied by complex background interference other than the wheat body. Therefore, manual segmentation processing is required: extract the wheat body area and remove outliers. During the annotation stage, the wheat point cloud is divided into three categories: wheat ears (category label: 0), flag leaves (category label: 1), and other categories (category label: 2).

[0057] In order to ensure the quality of the data set, the present invention introduces the Peak Signal-to-Noise Ratio (PSNR) to evaluate the quality of the 3D reconstructed point cloud. It is generally believed that when the PSNR is higher than 30dB, the reconstruction quality is better. When the PSNR is lower than 20dB, the wheat target structure in the point cloud is blurred, and it is difficult to accurately distinguish the wheat morphology from the scene. Therefore, the present invention screens the point cloud reconstruction results of all samples, eliminates 5 low-quality samples with PSNR lower than 20dB, and finally retains 75 high-quality samples (with an average PSNR of 35.23±1.71dB) for subsequent experimental analysis.

[0058] Before establishing a deep learning network, the diversity of the data set and the network input requirements need to be considered. On the one hand, the sample size of the 75 wheat point cloud data sets established in the early stage of the experiment is small, which makes the model prone to overfitting during the training process. On the other hand, the point cloud segmentation network usually needs to limit the scale of input data to reduce the number of network parameters and improve computational efficiency. In current crop phenotyping studies, three-dimensional point clouds are usually downsampled to 4096 points as network input. Although fewer points will improve training efficiency, the wheat point cloud data set used in the present invention is complex in structure and information-intensive. If the number of points is too small, key phenotypic features will be lost. In order to take into account both feature integrity and model training requirements, the present invention downsamples the point cloud to 8192 points and combines data enhancement strategies to improve data diversity and model generalization capabilities.

[0059] Currently, a variety of point cloud data enhancement methods have been proposed, such as voxel-based sampling (VBS) and farthest point sampling (FPS). Among them, VBS has a high processing speed and can retain as many point cloud features as possible, but the algorithm has two disadvantages: (1) the voxel size needs to be adjusted according to the density of the point cloud, which results in an uncertain number of points after sampling of different point cloud data and cannot be directly used for model training; (2) the sampling points selected by VBS are the centroid points of the voxels, which may lead to constant local density and generate a regular point cloud with uniform density. The data does not have diversity and is not conducive to model training; FPS can maintain the local density of the sampled point cloud and obtain a fixed number of sampling points, but this method still has disadvantages: (1) key features may be ignored in sparse areas, and redundant points may be over-sampled in dense areas. (2) the sampling results are sensitive to the initial points, making it difficult to directly use FPS to generate diverse training samples.

[0060] After studying the advantages and disadvantages of VBS and FPS, this paper proposes a new point cloud data enhancement method - dynamic voxel rasterization random sampling. It combines the advantages of VBS and FPS, achieving data scale controllability and diversity expansion while preserving key geometric features. The specific design is as follows:

[0061] 1. Dynamic voxel downsampling

[0062] The present invention introduces an octree level adaptive mechanism, such as Figure 2 As shown in the figure, the mechanism dynamically calculates the voxel size of each point cloud through the maximum bounding box size of the point cloud and the level of the octree:

[0063]

[0064] Where Cell_size is the side length of the voxel, L is the side length of the maximum bounding box of the point cloud, and l is the octree level.

[0065] This formula ensures that the voxel resolution and the local density of the point cloud are self-adaptive through spatial recursive partitioning, significantly reducing the volatility of the number of points after sampling. The specific implementation process is as follows: (1) Voxel grid construction: Set the octree level to 8, calculate the Cell_size, and divide the original point cloud into a uniform cubic voxel grid. Each voxel retains the coordinates and information of all points within it; (2) Perceptual center of gravity sampling: Take the average of the point cloud coordinates within each voxel to generate the center of gravity point. The label is determined by the majority voting principle to avoid the ambiguity caused by the traditional center of gravity method. At this stage, the point cloud size is compressed to approximately 50,000 points, while retaining the key structures of the flag leaf and ear of the plant, providing a high-density feature foundation for subsequent refined sampling.

[0066] 2. Random farthest point sampling

[0067] The present invention proposes the following improvement strategies: (1) Random selection of multiple candidate points: In each iteration, the K candidate points (K=10) farthest from the current sampling point set are screened, and the next sampling point is randomly selected from them to alleviate the initial point dependence; (2) Local density balance constraint: By calculating the neighborhood density weight of the candidate point, feature points in sparse areas are preferentially selected to avoid redundant sampling in dense areas; (3) Diversity amplification mechanism: F independent random farthest point samplings are performed on each dynamic voxelization downsampling result to ensure that the initial point selected each time is random and non-repetitive, thereby ensuring data diversity. Finally, F groups of 8192 point cloud data are generated to simulate the spatial distribution uncertainty of point clouds in real scenes. In this embodiment, F=10, which means a 10-fold amplification is achieved.

[0068] Finally, the original 75 wheat pots point cloud data were expanded into 750 groups of standardized point clouds (8192 points per group) using the DVFPS method. Figure 3 As shown, it effectively alleviates data scarcity and overfitting risks.

[0069] In 3D point cloud processing, data augmentation operations need to preserve key geometry and information during the downsampling process. This paper constructs a multi-dimensional geometry and evaluation system to quantitatively analyze the sampling quality.

[0070] 1. Geometric structure preservation

[0071] Chamfer Distance (CD) is an important indicator for measuring the geometric similarity between two point clouds, and is particularly widely used in downsampling tasks. This formula quantifies the global geometric deviation through a bidirectional nearest neighbor search. The specific expression is:

[0072]

[0073] Where P ori is the original point cloud set, P down is the down-sampled point cloud, T ori is the number of original point cloud points, T down is the number of downsampled point cloud points, is the inner product of the point cloud.

[0074] The Hausdorff distance (HD) is used to measure the local maximum geometric deviation between two point clouds, focusing on the areas with the greatest mismatch between the point clouds (such as the edge of a wheat flag leaf). Its core idea is to calculate the maximum distance between the "furthest neighboring points" between two point clouds, reflecting the fidelity of the local structure. The specific expression is:

[0075] HD(P ori ,P down )=max(h(P ori ,P down ),h(P down ,P ori )) (3)

[0076]

[0077] Where ‖‖2 is the L2 norm.

[0078] CD reflects the overall geometric consistency of a point cloud by averaging the bidirectional nearest neighbor distances. Its normalization properties make the evaluation results of point clouds of different sizes comparable. HD captures the farthest neighbor distance between point clouds, is highly sensitive to local outliers, and can reveal differences in fine-grained structure resulting from different numbers of sampling points. Combining these two methods can assess the ability of point cloud downsampling to preserve the curvature and topological connectivity of wheat organs.

[0079] 2. Distribution consistency verification

[0080] L1 distribution distance, also known as Manhattan distance, is used to measure the absolute difference between two probability distributions and is suitable for evaluating the consistency of point cloud label distribution before and after downsampling. The specific expression is:

[0081]

[0082] Where M is the label category set, P(m) is the original point cloud label probability distribution, and Q(m) is the downsampled point cloud label probability distribution.

[0083] KL divergence is an important indicator for measuring the difference between two probability distributions. It can be used to quantify the information loss of the downsampling results for the data of this invention. The specific expression is:

[0084]

[0085] The L1 distribution distance directly measures the absolute difference in label distributions, while the KL divergence reflects the asymmetry between distributions from the perspective of information entropy. This metric system has important diagnostic value for sampling sensitivity to fine-grained boundaries, such as the overlap between wheat ears and flag leaves.

[0086] The key organ segmentation method of the present invention is mainly based on CA-SegNet. CA-SegNet is based on the traditional encoding-decoding structure of PointNet++ and optimizes the feature extraction process according to the characteristics of wheat point cloud data. Figure 4 shown.

[0087] Specifically, during the encoding phase, an improved category-aware feature extraction module based on a multi-level sampling, grouping, and aggregation (SA) module is proposed to achieve hierarchical local feature extraction from point clouds. During downsampling, center points are selected based on their importance scores. During grouping, a multi-scale radius query strategy is used to enhance adaptability to the non-uniform density distribution of wheat point clouds. During feature extraction, local features are obtained through MLP and max pooling operations. During decoding, the feature propagation (FP) module is combined with a skip connection structure to gradually restore the point cloud resolution. During feature propagation, a weighted interpolation strategy based on KNN is used to propagate features from the current layer to the previous layer, thereby more precisely recovering local geometric details. Simultaneously, the interpolated features are concatenated with the original features from the previous layer, and feature fusion is achieved through MLP. Finally, the classification probability of each point is output, achieving point-level segmentation.

[0088] The CAFE module of the present invention integrates the two-stage category attention mechanism and the weighted farthest point sampling strategy, which enhances the focus on key areas while ensuring spatial uniformity, thereby improving the effect of wheat point cloud segmentation.

[0089] Specifically, in the traditional feature extraction module, since the FPS-based center point selection method only relies on the distribution of the point cloud space, it is easy to overlook some important feature points. The CAFE module of the present invention expands on the original work and achieves adaptive improvements from two-dimensional images to three-dimensional point clouds.

[0090] The two-stage category attention mechanism combines multi-head self-attention and category-specific attention, such as Figure 5Based on the characteristics of different categories in the point cloud data, a category attention score is generated for each point. Multi-head attention can capture the diverse and complex local and global information in the point cloud data, thereby helping the model to more comprehensively understand the structural information in the point cloud. Category-specific attention can fully utilize the separate spatial attention of each category to generate different category attention scores for each point in the point cloud, which is used to select the center point with higher value. The process can be described as follows:

[0091] First, the input point cloud feature f in The feature f is obtained by modeling the global point relationship through the multi-head self-attention layer mth To adapt the input of category-specific attention, the feature f mth Mapping to category space via MLP Get feature f Class Afterwards, a dual-pooling fusion strategy is used to calculate category-specific attention. Among them, average pooling extracts the overall information of the category, while maximum pooling focuses on the most representative points of the category, and the feature vectors f are obtained respectively. avg and f max The global category attention A is obtained by adding the results of average pooling and maximum pooling. Class , calculated as follows:

[0092]

[0093] Where N is the number of points in the input point cloud, and T represents transpose.

[0094]

[0095] A Class =f avg +f max (9)

[0096] Since max pooling obtains the maximum value of all spatial positions for each category, it can be regarded as a category-specific attention mechanism. Class and f Class Get the category feature f of each point fusion,i , and then the final category attention A of each point is obtained through MLP point Finally, for A point Normalize to get the attention score A of each point norm .

[0097] f fusion,i =concat(A Class ,f Class,i )(10)

[0098] A point =MLP(ffusion )(11)

[0099]

[0100] Where f Class,i For each point’s feature, A point,i For each point’s category attention, A norm,i Give each point a category attention score.

[0101] The weighted farthest point sampling strategy is an attention-guided farthest point sampling strategy based on the traditional center point selection method. This strategy combines the category attention score and the geometric distance score to measure the importance of the point, achieving a sampling strategy that is co-optimized with the space. The algorithm flow is as follows:

[0102]

[0103]

[0104] The importance score of each point in the wheat point cloud is defined as:

[0105] I i =αA norm,i +(1-α)D norm,i (13)

[0106] Where D norm,i is the shortest Euclidean distance from the normalized point i to the currently selected point set, α is the balance coefficient of the category attention score, and (1-α) is the balance coefficient of the geometric distance score.

[0107] Through the dual-stage category attention module mechanism and the weighted farthest point sampling strategy, the module effectively integrates features and spatial distribution information, thereby optimizing the local feature extraction process.

[0108] In order to further improve the effect of wheat point cloud segmentation, this paper proposes a composite loss function L CIR , comprehensively considering classification accuracy, spatial consistency and feature constraints. This loss function consists of three parts: Cross-Entropy loss, Complete Intersection over Union (CIoU) loss and Feature Regularization loss. The final total loss function is expressed as:

[0109] L CIR =βL CE +γL CIoU +δL reg (14)

[0110] Here, β, γ, and δ are weights for each loss. To ensure that each loss contributes appropriately to the overall optimization process during training, this paper sets β = 1.0, γ = 1.0, and δ = 0.01. This setting comprehensively considers the magnitude differences of each loss term during training, preventing a single loss term from dominating or weakening the model's learning direction due to excessively large or small values. This loss function design not only improves segmentation accuracy but also enhances the stability of point cloud feature extraction, and its effectiveness has been demonstrated.

[0111] Among them, the cross entropy loss is used to measure the difference between the model prediction category and the true category. Its calculation formula is as follows:

[0112]

[0113] Where Class is the number of categories, y i,c is the true label of the i-th point in category c, The probability of the i-th point in category c predicted by the model.

[0114] This loss function encourages the network to learn a more accurate category distribution by minimizing the difference between the model prediction and the true category.

[0115] Among them, the traditional intersection over union (IoU) is mainly used for detection tasks, but in point cloud segmentation tasks, the simple overlap measurement of IoU may not be enough to reflect the segmentation quality. To this end, this paper introduces CIoU, which not only calculates IoU but also considers the center offset and shape similarity of the category distribution. The CIoU loss is defined as follows:

[0116] L CIoU =1-[IoU-D center -D shape ] (16)

[0117] Intersection-over-Union (IoU): Calculates the mean IoU of each category to enhance sensitivity to small categories.

[0118]

[0119] Where TP is the point that is correctly classified, FP is the point that is misclassified, and FN is the point that is misclassified as a non-target category.

[0120] Center shift penalty: measures the spatial center shift between the predicted category and the true category distribution:

[0121]

[0122] In the formula is the weighted spatial center of the predicted category c, is the weighted spatial center of the true category c.

[0123] Shape similarity term: constrains the covariance similarity between the prediction and the true category distribution.

[0124]

[0125] In the formula is the point coordinate covariance matrix of the predicted category c, is the point coordinate covariance matrix of the true category c, |||| F is the Frobenius norm.

[0126] By introducing the center offset and shape similarity of category distribution, CIoU loss improves the stability and accuracy of point cloud segmentation, especially when dealing with small objects and dense areas, and can effectively avoid common segmentation inconsistency problems.

[0127] Among them, the feature regularization loss is used to constrain the feature space of the model, prevent overfitting and enhance the robustness of feature extraction. Apply L2 regularization.

[0128]

[0129] Where F i is the eigenvector of the i-th point.

[0130] By introducing feature regularization, the model can avoid overfitting during training and improve the generalization ability of the model.

[0131] The present invention builds a model based on the PyTorch framework and trains it on the NVIDIA RTX 4090D GPU (24GB video memory) and Intel (R) Xeon (R) Platinum 8481C (16vCPU) platforms to ensure the efficiency and stability of the calculation. The present invention uses a self-built wheat point cloud dataset as the experimental dataset. In order to ensure the objectivity of the evaluation and the independence of the training samples, 10 of the 75 original samples are randomly selected as the source of the test set, and the corresponding 100 enhanced samples constitute the test set, and the remaining 650 enhanced samples are used for model training. The model training is carried out for a total of 150 rounds (Epochs), and the batch size (Batch Size) is finally set to 16 after multiple experiments, and the Adam optimizer is used to update the parameters to improve the convergence speed and stability of the training.

[0132] In order to comprehensively evaluate the segmentation performance of the model, this paper uses mean intersection over union (mIoU), precision, recall, and F1-score as the main evaluation indicators. The calculation formula is as follows:

[0133]

[0134] In order to verify the effectiveness of the DVFPS method proposed in this paper, the data enhancement results of VBS, FPS, 3D Edge-Preserving Sampling (3DEPS) and DVFPS are compared and analyzed based on the established evaluation system. Figure 6 As shown in the figure. In the comparative experiments, VBS first downsampled using a fixed voxel size (0.01), then used a random algorithm to sample 8192 points. FPS and DVFPS directly sampled at a fixed 8192 points. 3DEPS followed the recommended ratio of 30% edge points to 70% interior points. All experiments were conducted under the same environment.

[0135] Comparing the four sampling methods yielded the results shown in Table 1. The experimental results demonstrate that DVFPS significantly outperforms the other algorithms in terms of geometric fidelity. The CD index for DVFPS, FPS, and 3DEPS all reached 0.0001, a 66.67% decrease compared to VBS (0.0003), demonstrating their superior global geometric consistency. The HD index further reveals differences in local features: DVFPS achieved an HD value of 0.0225, a 28.34% decrease compared to FPS (0.0314), an 83.61% decrease compared to VBS (0.1373), and a 48.51% decrease compared to 3DEPS (0.0437).

[0136] The consistency of label distribution directly affects the model's recognition accuracy of organ boundaries. The experimental results show that DVFPS performs best in both L1 distance and KL divergence. Its L1 distance is 0.1580, which is 8.09%, 14.87% and 2.77% lower than VBS (0.1719), FPS (0.1856) and 3DEPS (0.1625), respectively; the KL divergence is 0.0261, which is 29.27%, 18.44% and 7.45% lower than VBS (0.0369), FPS (0.0320) and 3DEPS (0.0282), respectively. Based on the above results, it can be concluded that the DVFPS proposed in the present invention has significant advantages in both geometric preservation and consistency, and is suitable for subsequent model training.

[0137] Table 1 Comparison of the effects of downsampling points

[0138] Sampling method CD HD L1 KL VBS 0.0003 0.1373 0.1719 0.0369 FPS 0.0001 0.0314 0.1856 0.0320 3DEPS 0.0001 0.0437 0.1625 0.0282 DVFPS 0.0001 0.0225 0.1580 0.0261

[0139] To validate the model's segmentation performance, we evaluated wheat ears, flag leaves, and other organs on a self-constructed dataset. The experimental results, shown in Table 2, demonstrate excellent results across multiple evaluation metrics. The IoU for the wheat ear region reached 66.16%, demonstrating that the proposed method is effective in identifying wheat ear structures. Furthermore, the IoU for other organs was also high, demonstrating the model's strong overall segmentation capabilities.

[0140] Table 2 Model evaluation results (%)

[0141] category IoU Precision Recall F1-score wheat ears 66.16 82.99 76.54 79.63 flag leaf 34.55 56.70 46.91 51.34 Other organs 86.13 89.86 95.41 92.55 average 62.28 76.41 72.63 74.47

[0142] Although the present invention achieved good results in overall segmentation, the flag leaf and other leaves differ only in position, and there is a certain degree of mutual occlusion and intersection between the flag leaf, wheat ear, and other parts in the wheat point cloud data. This results in a relatively low segmentation effect in the flag leaf area, with an IoU of only 34.55%. The IoU of other parts (non-flag leaf and wheat ear areas) is higher because the number of point clouds is the highest. The IoU of the wheat ear is between the two.

[0143] To verify the DVFPS, CAFE and L used in the network CIR In order to understand the contribution of the design to the model, different modules were removed and ablation tests were performed. The results are shown in Table 3. It can be found that removing DVFPS, CAFE and L CIR All of these factors will deteriorate the network segmentation performance, with average decreases of 2.54%, 1.76%, 1.03%, and 1.39% in mIoU, Precision, Recall, and F1-score, respectively. This shows that the DVFPS, CAFE, and optimized loss design proposed in this paper play an important role in improving point cloud segmentation performance, effectively enhancing wheat point cloud segmentation accuracy and model generalization capabilities.

[0144] Table 3 Ablation test evaluation results (%)

[0145]

[0146] To validate the effectiveness of our proposed method, we compared it with several existing point cloud segmentation methods, including PointNet, PointNet++, PlantNet, and PSegNet. These networks used the same number of input point clouds (8192) as ours and employed their respective recommended parameters. The experimental results are shown in Table 4.

[0147] Table 4 Comparison of model effects (%)

[0148]

[0149] From the test results, we can see that for the wheat point cloud dataset, the proposed method achieves optimal or near-optimal results in terms of mIoU, Precision, Recal and F1-score, showing its good comprehensive performance. Figure 7 (a) and (b) show the segmentation results of different methods in the normal case and the case of overlapping leaf spikes, respectively. The advantages of CA-SegNet in accuracy and detail recovery can be clearly seen. This shows that CA-SegNet can better capture key features when dealing with complex organ occlusion and dense structures.

[0150] The balance coefficient plays a crucial role in weighted farthest point sampling, which determines the relative influence of category attention and geometric distance in point selection. In order to optimize the selection of center points in local feature extraction, the attention-weighted farthest point sampling strategy comprehensively considers the category attention weight A norm and geometric distance weight D norm , the importance of each point is calculated by weighted formula (4). In order to determine the optimal balance coefficient, the present invention uses 0.05 as a step size and conducts experiments under different α values, such as Figure 8 The experimental results show that when α = 0.50, the comprehensive effect of the four indicators of the model reaches the best, mIoU is the highest, and the segmentation performance is the best.

[0151] Data amplification is crucial for improving model training results and generalization capabilities. To explore the data amplification factor that is more suitable for model training, this paper generates training sets with various amplification factors (5×, 10×, 15×, and 20×) based on the DVFPS method and compares their segmentation performance and training effectiveness. The experimental results show that a 10× data amplification factor achieves the best balance between model performance and computational cost:

[0152] Table 5 Comparison of data amplification effects (%)

[0153] Amplification factor Total amount of data mIoU Precision Recall F1-score 5 375 60.33 75.54 71.97 73.71 10 750 62.28 76.41 72.63 74.47 15 1125 62.34 76.52 72.75 74.59 20 1500 62.06 76.26 72.44 74.30

[0154] When the amplification factor increased from 5× to 10×, mIoU significantly increased by 3.23%, and other metrics (Precision, Recall, and F1-score) also increased. This indicates that 10× amplification increased the diversity of the original data, and the samples generated by DVFPS fully covered the spatial distribution characteristics of the wheat point cloud, enhancing the model's ability to identify key structures such as flag leaves and ears. However, when the amplification factor was increased to 20×, mIoU decreased by 0.35%, indicating that excessive amplification may introduce redundant noise, which is particularly evident in occluded areas of the wheat point cloud (such as the intersection of the flag leaf and the ear).

[0155] From the perspective of training efficiency, the relationship between amplification and training time increases nonlinearly. On an NVIDIA RTX4090D GPU, 10x data (750 samples) takes about 1 hour to complete 150 rounds of training, while 15x data (1125 samples) takes about 3 hours. Therefore, this paper chooses 10x amplification as the training scheme, and ultimately amplifies the original wheat point cloud of 75 basins by 10x to 750 basins.

[0156] To verify the practical application value of the proposed method in wheat phenotypic parameter extraction, the present invention realizes the quantitative extraction of wheat ear volume based on wheat ear point cloud data obtained by CA-SegNet network segmentation and combined with the convex hull volume calculation method. The specific process is as follows: (1) According to the manually measured wheat plant height, the point cloud data is scaled to make it consistent with the actual height; (2) The density-based spatial clustering of applications with noise (DBSCAN) algorithm is used to cluster the wheat ear point cloud. After multiple tests, the neighborhood radius ε is set to 0.05 times the scaling ratio and the minimum number of neighborhood sample points (min-Points, minPts) is set to 5 to obtain good clustering effect; (3) The convex hull volume of each cluster substructure is calculated and accumulated to obtain the total volume of wheat ears in a single pot.

[0157] To evaluate the accuracy of the model in measuring wheat ear volume, the present invention analyzed the wheat ear volume control value based on the CloudCompare tool based on the voxel method and compared it with the calculated wheat ear volume. The determination coefficient R was obtained by regression analysis. 2 The root mean square error (RMSE) is 117.73 cm 3 ,like Figure 9 As shown, the regression determination coefficient R in the leaf inclination angle task is calculated with reference to 2 When it is 0.71, it is considered to have good accuracy. The error between the control value and the calculated value under the complex structural conditions of wheat is within an acceptable range, which verifies the effectiveness of CA-SegNet in wheat phenotyping.

[0158] In summary, the DVFPS method proposed in this paper enhances the diversity of data and effectively reduces the risk of model overfitting; the improved CAFE module improves the performance of the model in complex scenarios by optimizing the center point selection in feature extraction; the designed loss function L CIR The segmentation accuracy and robustness of the model are optimized. Ablation tests show that removing DVFPS, CAFE and LCIR All of these lead to a decline in model performance, with mIoU, Precision, Recall, and F1-score decreasing by an average of 2.54%, 1.76%, 1.03%, and 1.39%, respectively, verifying the effectiveness of each part.

[0159] The results of this study demonstrate that CA-SegNet achieves significant performance improvements in the segmentation of key organs in a self-built wheat point cloud dataset. The optimal mIoU (mIoU) score is 62.28%, Precision is 76.41%, Recall is 72.63%, and F1-score is 74.47%, outperforming networks such as PointNet, PointNet++, PlantNet, and PSegNet. Further research could introduce instance segmentation capabilities to distinguish individual organs of the same type and support the automatic calculation of phenotypic parameters such as flag leaf area.

Claims

1. A three-dimensional point cloud segmentation method for wheat organs based on dynamic voxels and category perception, characterized in that: The steps include: Step 1: Reconstruct the wheat sample in 3D based on the 3D Gaussian splash method and obtain dense point cloud data; Step 2: A dynamic voxel rasterization random sampling method combining dynamic voxel partitioning and random farthest point sampling strategy is used to augment the dense point cloud data, and a standard point cloud dataset is obtained by standardizing the number of point clouds. Step 3: Based on the standard point cloud dataset, a category-aware segmentation network is used to segment key organs.

2. The wheat organ 3D point cloud segmentation method based on dynamic voxel and category perception according to claim 1 is characterized in that: In step 1, the number of wheat samples does not exceed 100, and the image dataset is obtained for 3D reconstruction in the following manner: Using a surround-view shooting method, video data was acquired from three observation angles: the upper part of the wheat canopy, the middle of the ear, and the lower part of the base. Sequential images were extracted at a set frame rate to construct a multi-view image dataset covering 360° of wheat space. After obtaining the dense point cloud data, it is labeled and divided into wheat ears, flag leaves and other categories, and the wheat sample point cloud data below the threshold is eliminated based on the peak signal-to-noise ratio.

3. The wheat organ three-dimensional point cloud segmentation method based on dynamic voxel and category perception according to claim 1 or 2, characterized in that: In step 2, dynamic voxel division is achieved by dynamic voxel downsampling, as follows: Step 211 , constructing a voxel grid of the point cloud based on an octree hierarchical adaptive mechanism, where each voxel grid retains the coordinates and information of all points within it; In step 212, the point cloud coordinates in each voxel grid are averaged to generate a centroid point, and the label is determined by the majority voting principle to achieve perceptual centroid sampling.

4. The wheat organ 3D point cloud segmentation method based on dynamic voxel and category perception according to claim 3, characterized in that: The step 211 constructs a voxel grid of the point cloud based on the octree hierarchical adaptive mechanism, and the implementation method is as follows: Set up the octree hierarchy and dynamically calculate the voxel size of each point cloud, thereby dividing the original point cloud into a uniform cubic voxel grid. The pixel size is calculated as follows: Where Cell_size is the side length of the voxel, L is the side length of the maximum bounding box of the point cloud, and l is the octree level.

5. The wheat organ three-dimensional point cloud segmentation method based on dynamic voxel and category perception according to claim 1 or 2, characterized in that: In step 2, the random farthest point sampling strategy is as follows: (1) Randomly select the initial point as the sampling point, then screen the K candidate points farthest from the current sampling point set, and randomly select the next sampling point from them to alleviate the initial point dependence, where K is the data amplification factor; (2) By calculating the neighborhood density weight of the candidate points, feature points in sparse areas are selected to avoid redundant sampling in dense areas; (3) Perform F independent random farthest point samplings on each dynamic voxelization downsampling result to ensure that the initial point selected each time is random and non-repetitive, thereby ensuring data diversity, where F is the data amplification factor.

6. The wheat organ 3D point cloud segmentation method based on dynamic voxel and category perception according to claim 1, characterized in that: In the encoding phase, the feature extraction module of the category-aware segmentation network performs hierarchical local feature extraction of the point cloud by: selecting the center point according to the point importance score during downsampling; using a multi-scale radius query strategy during grouping to enhance adaptability to the non-uniform density distribution of the point cloud; and obtaining local features through MLP and maximum pooling operations during feature extraction. In the decoding stage, the point cloud resolution is gradually restored through the feature propagation module combined with the skip connection structure; During the feature propagation process, the KNN weighted interpolation strategy is adopted to pass the features of the current layer upward to the previous layer, splice the interpolated features with the original features of the previous layer, and complete feature fusion through MLP; finally, the classification probability of each point is output to achieve point-level segmentation.

7. The wheat organ three-dimensional point cloud segmentation method based on dynamic voxel and category perception according to claim 1 or 6, characterized in that: The category-aware segmentation network uses a category-aware local feature extraction module for feature extraction. The local feature extraction module adopts a two-stage category attention mechanism and a weighted farthest point sampling strategy. The two-stage category attention mechanism combines multi-head self-attention and category-specific attention. Multi-head self-attention captures the diverse and complex local and global information in point cloud data. Category-specific attention uses the separate spatial attention of each category to generate a different category attention score for each point in the point cloud, which is used to select the center point with higher value. The weighted farthest point sampling strategy measures the importance of points by fusing category attention scores and geometric distance scores, thereby achieving sampling that is co-optimized with space.

8. The wheat organ 3D point cloud segmentation method based on dynamic voxel and category perception according to claim 7, characterized in that: The two-stage category attention mechanism firstly focuses on the input point cloud features f in The feature f is obtained by modeling the global point relationship through the multi-head self-attention layer mth , the feature f mth Mapping to category space via MLP Get feature f Class , where N is the number of points in the input point cloud; then, a double pooling fusion strategy is used to calculate the category-specific attention, where the average pooling extracts the overall information of the category and the maximum pooling focuses on the most representative points of the category, respectively obtaining the feature vectors f avg and f max , add the results of average pooling and maximum pooling to get the global category attention A Class , by splicing A Class and f Class Get the category feature f of each point fusion,i , and then use MLP to get the final category attention A of each point point ; Finally, for A point Normalize to get the attention score A of each point norm .

9. The wheat organ 3D point cloud segmentation method based on dynamic voxel and category perception according to claim 7, characterized in that: The weighted farthest point sampling strategy is implemented as follows: Randomly select the initial center point and initialize the global minimum distance matrix, build a set of sampling points, calculate the Euclidean distance from the current center point to all points, update the global minimum distance and normalize it, then calculate the importance score, select the point with the highest score as the new sampling point, and update the sampling point set; The importance score of each point in the point cloud is defined as: I i =αA norm,i +(1-α)D norm,i Where D norm,i is the shortest Euclidean distance from the i-th point to the currently selected point set after normalization, α is the balance coefficient of the category attention score, and (1-α) is the balance coefficient of the geometric distance score.

10. The wheat organ 3D point cloud segmentation method based on dynamic voxel and category perception according to claim 1, characterized in that: The category-aware segmentation network adopts a composite loss function L that combines the spatial structure and information of the point cloud. CIR , which is determined by the cross entropy loss L CE , complete intersection loss L CIoU and feature regularization loss L reg Composition, expressed as: L CIR =βL CE +γL CIoU +δL reg , where β, γ and δ are the weights of each loss respectively.

Images