Method for separating crop three-dimensional point cloud branches and leaves based on semantic prototype-driven graph attention network

Abstract

The application discloses a crop three-dimensional point cloud branch and leaf separation method based on a semantic prototype driving graph attention network, relates to the fields of plant high-throughput phenotype analysis and three-dimensional computer vision technology, adopts random down-sampling and attention pooling mechanism to extract local geometric features, reduces the size of a point cloud by randomly generating a pooling index, aggregates neighborhood features by using geometric topology coding, constructs a dynamic topology structure based on a K-neighbor algorithm, multi-levelly aggregates node features through a graph attention network, establishes double constraints of geometric difference perception and feature association, realizes deep integration of local and global context information, gradually up-sample low-resolution features to high-resolution features by using K-neighbor interpolation based on attention weighting, and fuses the high-resolution features corresponding to the coding stage by using a skip connection and weighting, improves the distribution structure of a class feature space through an inter-class separability discriminant optimization function, and guarantees the stability of feature distribution.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of plant high-throughput phenotype analysis and three-dimensional computer vision, and particularly relates to a crop three-dimensional point cloud branch and leaf separation method based on a semantic prototype-driven graph attention network. BACKGROUND

[0002] Plant high-throughput phenotype analysis is a key link in plant breeding and crop improvement. By accurately measuring and analyzing leaf traits (such as leaf length, leaf width, leaf area, etc.), the complex interactions between genes and the environment are revealed, providing a scientific basis for improving crop stress resistance, optimizing yield, and efficient use of resources. However, in the complex plant (such as sorghum and tomato) plant structure, the diversity of leaf shape and the complexity of topological structure (such as the overlap and occlusion of leaves and stems) make high-precision leaf segmentation and trait extraction a great challenge and a key technical problem in plant high-throughput phenotype analysis.

[0003] In recent years, researchers have proposed many deep learning-based methods to solve the problem of accurately separating crop branches and leaves from three-dimensional point cloud data. These deep learning-based crop point cloud branch and leaf separation methods are mainly divided into three categories: multi-view convolutional neural network-based methods, 3D voxel convolutional neural network-based methods, and point deep learning network-based methods. Early multi-view convolutional neural networks (such as MVCNN) use two-dimensional convolution techniques to effectively model three-dimensional shapes, but due to the loss of spatial information during the projection process, there are obvious limitations; 3D voxel convolutional neural network-based methods (such as VoxelNet) can capture local geometric features by discretizing point cloud data into regular grids, but face the problem of precision loss and high computational overhead. In contrast, point deep learning network-based methods directly operate on point cloud data, avoiding the loss of information during the projection and discretization process, and have gradually become a research hotspot in the field of plant phenotype analysis. PointNet first proposed an end-to-end learning strategy for point cloud, using global max-pooling operations to capture overall features, solving the problems of point cloud disorder and rotation invariance, but lacking local feature modeling capability, making it difficult to express complex structures. On this basis, PointNet++ introduced a multi-scale feature extraction strategy, by grouping and sampling point clouds and constructing local neighborhoods, layer-by-layer extracting and aggregating local features, significantly enhancing the modeling capability of local geometric features. However, the strong dependence of PointNet++ on neighborhood features leads to unstable performance when dealing with sparse and irregular point clouds, and the feature aggregation method relies on a simple symmetric function, making it difficult to capture complex geometric relationships between points, limiting the effective expression of fine-grained features.

[0004] Although the proposal of these deep learning-based methods promotes the development of crop point cloud branch-leaf separation technology, in the segmentation task of complex plant structure, the existing methods extract neighborhood features through a simple local aggregation strategy, which is difficult to fully capture the complex geometric relationship between points in the point cloud, resulting in limited modeling ability for complex topological structures (such as the overlap and occlusion of leaves and stems) in plant point cloud. At the same time, the interaction between local features and global context information is weak, and the deep integration of global semantics and local geometry cannot be achieved. In addition, the existing methods have limited ability to distinguish different categories in the feature space, and the visual features of different categories overlap a lot, which further limits the crop point cloud branch-leaf segmentation accuracy and model robustness. Therefore, how to fully exploit the complementarity of multi-level features, realize the deep fusion of local and global information of three-dimensional point cloud, and on this basis, further optimize the construction process of decision space, has important value for accurately modeling the complex geometric relationship in plant point cloud and significantly improving the crop point cloud branch-leaf segmentation performance. SUMMARY

[0005] To solve the above problems, the present application proposes a crop three-dimensional point cloud branch-leaf separation method based on semantic prototype driven graph attention network, aiming to solve the problems of insufficient visual feature expression and low category distinguishability in the segmentation task of complex plant structure (such as leaves and stems). Through the design of efficient feature extraction, feature enhancement, feature reconstruction and feature space optimization mechanism, the accuracy and robustness of crop point cloud branch-leaf separation are significantly improved. First, random down-sampling and attention pooling mechanism is adopted to efficiently extract local geometric features in multi-layer encoder structure, the pool index is randomly generated to reduce the size of the point cloud, and the neighborhood features are weighted aggregated by using geometric topology coding to ensure the high-quality extraction of local geometric details. Second, based on K- nearest neighbor algorithm, dynamic topological structure is constructed, node features are multi-level aggregated by graph attention network, geometric difference perception and feature association dual constraints are established, and deep integration of local and global context information is realized. Then, the low-resolution features are gradually up-sampled to reconstruct high-resolution features by using K- nearest neighbor interpolation based on attention weighting, and the corresponding high-resolution features in the encoding stage are weighted fused through the jump connection to enhance the synergistic effect of local geometric details and global semantics, effectively making up for the possible loss of local geometric structure details in the deep encoding process. Finally, the class separability discriminant optimization function is used to improve the distribution structure of the class feature space, the aggregation of the same class points is enhanced from three aspects of intra-class aggregation term, inter-class separation term and centroid constraint term, the distance between the centroids of different classes is enlarged, and the stability of feature distribution is guaranteed. Through the cooperative optimization of class separability discriminant and cross entropy, the class distinguishability is enhanced, and the simultaneous improvement of branch-leaf separation accuracy and feature space optimization is realized.

[0006] The crop three-dimensional point cloud branch and leaf separation method of the semantic prototype driving graph attention network comprises a multi-level feature coding module, a feature enhancement module, a feature decoding module and an optimization module;

[0007] The multi-level feature coding module adopts a random down-sampling and attention pooling mechanism to extract local geometric features, reduces the point cloud scale by randomly generating a pooling index, and aggregates neighborhood features by using geometric topology coding;

[0008] The feature enhancement module constructs a dynamic topology structure based on a K-neighbor algorithm, aggregates node features in multiple levels by using a graph attention network, establishes dual constraints of geometric difference perception and feature correlation, and realizes deep integration of local and global context information;

[0009] The feature decoding module gradually up-samples and reconstructs high-resolution features by using K-neighbor interpolation based on attention weighting, and fuses the high-resolution features corresponding to the encoding stage by using a skip connection and weighted fusion;

[0010] The optimization module improves the distribution structure of the class feature space by using an inter-class separability discrimination optimization function, and enhances the aggregation of points of the same class from three aspects of intra-class aggregation, inter-class separation and centroid constraint.

[0011] In the preferred embodiment, the multi-level feature coding module adopts a random sampling method of pre-generated sub-sampling and neighborhood index, and retains the core three-dimensional structure information of the crop plant:

[0012] Let the visual feature of the input point cloud be where N is the number of sampling points, D is the visual feature dimension of each sampling point, and an index set S is first randomly generated The random sampling process ensures that each sampling point has an equal probability of being selected. For the i-th sampling point p i ∈S, find a set of k neighborhood points in the Euclidean space by using a K-neighbor algorithm In the feature extraction stage, the neighborhood index is used to complete feature aggregation by using an attention pooling method, and the visual feature f′ i of the down-sampled point cloud is calculated N′ represents the number of sampling points after random down-sampling, and the real number domain is denoted by

[0013] In the preferred embodiment, an attention pooling mechanism is introduced in the down-sampling process:

[0014] The K-neighbor algorithm is used to construct the local neighborhood of a query point p i extract its neighborhood structure difference and perform feature coding to obtain the point cloud feature representation f iThe encoding mode is as follows:

[0015]

[0016] wherein, represents the three-dimensional geometric structure difference between points, Concat(·) represents the connection operation, represents the geometric relationship between the query point and the neighborhood;

[0017] The shared multi-layer perception MLP and Softmax operation are adopted to perform nonlinear transformation and weight allocation on the visual features, and the weight a is obtained i :

[0018] a i = Softmax(MLP(f i ))

[0019] By weighted aggregation of K neighborhood features, the down-sampled point cloud visual feature f′ i and the new down-sampled point feature matrix F′ t are obtained.

[0020]

[0021] In each encoder layer, the down-sampled point feature matrix F′ t of the current layer is fused with the feature matrix F′ t-1 of the previous layer through a splicing operation:

[0022] F′ = F′ t + Conv(F′ t-1 )

[0023] wherein, Conv(F′ t-1 ) is the result of the previous layer feature matrix after convolution.

[0024] In the preferred embodiment, the feature enhancement module regards the down-sampled point cloud data output by the encoding layer as a node set V = {p1, p2, …, p N′} of a graph, each node in the N′ node set represents a sampling point, and the connection relationship between the nodes is represented by an edge set E. The K-neighborhood algorithm is used to construct a neighborhood connection for each sampling point in the Euclidean space:

[0025] The distance between all pairs of points in the point cloud is calculated to obtain a distance matrix The K-neighborhood algorithm is used to search for K nearest neighbors for each sampling point p i , and construct a neighborhood relationship E:

[0026]

[0027] The features of the edges are encoded, and the features e ij The geometric difference and feature difference between the center point and the neighborhood points are jointly represented as:

[0028] e ij = concat(p j -p i ,f j -f i )

[0029] where p i , p j are the three-dimensional coordinates of the i, j sampling points, and f i , f j are the visual features of the corresponding points in the down-sampled point feature matrix F′.

[0030] In the preferred embodiment, after completing the neighborhood topology modeling, the node visual features are iteratively aggregated and updated based on the graph attention mechanism. The node features are updated layer by layer through neighborhood aggregation and attention weighting calculation. The update formula is:

[0031]

[0032] where F represents the node features of the l+1 layer, represents the feature vector of the neighbor node pj in the l layer, W (l) is the weight matrix of the current layer l, and σ is a nonlinear activation function.

[0033] In the preferred embodiment, based on the attention mechanism, feature transformation is performed to integrate local and global features at different levels to form a global semantic representation:

[0034]

[0035] where F G represents the final feature matrix after processing, F′ (0) , F′ (1) , and F′ (2) represent the point cloud feature matrices extracted at layers 0, 1, and 2, and Linear(·) represents the dimensionality reduction operation.

[0036] In the preferred embodiment, the feature decoding module adopts a K-neighbor interpolation strategy based on geometric attention weighting. The local geometric relationship of the point cloud is used to calculate the weight of each high-resolution point cloud inheriting features from the low-resolution point cloud, and Softmax normalization is used to ensure the smoothness and stability of the interpolation:

[0037] Let the down-sampled point cloud coordinate set be {p′ j}, j = 1, 2, …, N', each sampling point p' j with feature vector f' j ∈ F G ; the original resolution or the point cloud coordinate set of the last higher resolution layer is {p i}, for each high-resolution point p i , the K nearest neighbor points p' j with the nearest geometric distance are selected in the low-resolution point set by using the up-sampling index or the geometric neighborhood relationship stored in the encoding stage

[0038] The interpolation calculation result is obtained by weighted sum of the neighborhood features by the following formula

[0039]

[0040] Wherein:

[0041]

[0042] Wherein, f' j is the feature vector of the point p' j in the low-resolution point cloud, f' k and p' k represent the feature vector and point coordinates of the kth neighbor point in the low-resolution space respectively; α ij represents the attention weight of point p i inherited from point p' j , and MLP represents a multi-layer perceptron.

[0043] In the preferred embodiment, the feature decoding module adopts a skip connection strategy:

[0044] By interpolation up-sampling, the output features of the last decoding layer are mapped to the current layer, and the features obtained by up-sampling are fused with the corresponding high-resolution features in the encoding stage. Let the decoding output features of the current layer be denoted as The output features of the last decoding layer are denoted as The encoding features corresponding to the current layer are denoted as

[0045]

[0046] The fusion weight α l is normalized:

[0047]

[0048] Wherein, UpInterp represents the up-sampling operation based on K-neighbor interpolation, and σ represents a nonlinear activation function.

[0049] In preferred embodiments, a class centroid c k is computed for each class, and a constraint is imposed that all sample feature vectors e j in the class are attracted towards the class centroid, with a penalty, intra-aggregation loss intra if the distance of a sample from the class centroid exceeds a threshold τ is defined as follows:

[0050]

[0051] where C denotes the total number of classes, c denotes the feature centroid of the c-th class, e j denotes the feature of the j-th point in the c-th class, N c denotes the number of samples in the c-th class, and ReLU denotes a rectified linear unit that returns the input value as is if it is non-negative, and 0 otherwise;

[0052] inter-class separation term

[0053]

[0054] where and denote the centroids of class c and class n, respectively, denotes the Euclidean distance between the centroids of the two classes;

[0055] centroid constraint term

[0056]

[0057] The total objective of the discriminative loss is defined as:

[0058]

[0059] where δ, β, γ are weight coefficients.

[0060] In preferred embodiments, a cross-entropy loss is used as the base classification loss:

[0061]

[0062] where C is the number of classes, n is the class index, t n denotes the predicted value, denotes the true label, and the predicted probability distribution of the model is gradually approaching the true probability distribution by minimizing through gradient backpropagation;

[0063] total loss function ​is defined as:

[0064]

[0065] wherein, λ is a weight coefficient.

[0066] Compared with the prior art, the present application has the following beneficial technical effects:

[0067] A crop three-dimensional point cloud branch and leaf separation method based on a semantic prototype driven graph attention network is provided. With the information transmission characteristics of the graph attention mechanism, local visual features and global context information are efficiently integrated to improve the description ability and discrimination ability of visual feature expression. At the same time, an inter-class separability discrimination and cross-entropy collaborative optimization strategy is designed to further enhance the distinguishability between different classes in the high-dimensional feature space, thereby significantly improving the crop three-dimensional point cloud branch and leaf separation precision and robustness.

[0068] The present application has the following beneficial effects: The present application combines the feature extraction capability of a deep learning network, utilizes graph construction and attention mechanism, and realizes efficient interaction of point cloud local features and global context information. By introducing a feature enhancement module based on a graph attention network, the feature expression capability in the leaf and stem segmentation task in complex plant point clouds is optimized, and the distinguishability between classes is significantly improved. At the same time, an optimization function of inter-class separability discrimination and cross-entropy collaboration is designed to optimize the distribution structure of the class feature space, enhance the aggregation of points within the class and the separation of points between classes, and effectively improve the segmentation precision and robustness under complex topological structures. The present application can be widely applied in the fields of plant high-throughput phenotype analysis, crop improvement and precision agriculture, and provides reliable technical support for the in-depth development of plant scientific research and related technologies. BRIEF DESCRIPTION OF DRAWINGS

[0069] Figure 1 A visual comparison chart of the plant branch and leaf separation results of the present application. DETAILED DESCRIPTION

[0070] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0071] The crop three-dimensional point cloud branch and leaf separation method based on the semantic prototype driving graph attention network mainly includes four modules: a multi-level feature encoding module based on random sampling and attention pooling, a feature enhancement module based on a graph attention network, a feature decoding module based on neighborhood interpolation and multi-scale fusion, and an optimization module based on inter-class separability discrimination and cross-entropy cooperation.

[0072] (1) Multi-level feature encoding module based on random sampling and attention pooling

[0073] This module aims to encode key multi-level visual features from the input point cloud data, and reduce the size of the point cloud while retaining important geometric structure information through an efficient downsampling method.

[0074] The multi-level feature encoding module uses a random sampling method that generates a subsampling and neighborhood index in advance, reducing the computational cost and preserving as much of the core crop plant three-dimensional structure information as possible.

[0075] Specifically, let the visual feature of the input point cloud be where N is the number of points and D is the dimension of the visual feature of each point. To achieve downsampling, first generate a set of indices randomly where |S|=N′=512, N′ represents the number of points after random downsampling, and N′<N. This random sampling process ensures that each point has an equal probability of being selected, thereby avoiding bias introduced by the sampling strategy. Subsequently, for each sampled point p i ∈S, find its neighborhood point set In the subsequent feature extraction stage, these neighborhood indices are used to aggregate features through methods such as max-pooling and attention-pooling, thereby calculating the downsampled point cloud visual feature f′ i . Finally, the downsampled point cloud feature matrix

[0076] To further capture the fine-grained features of the point cloud and enhance the model's ability to model local information, the invention introduces an attention pooling mechanism in the downsampling process. First, use the K-neighborhood algorithm to construct the local neighborhood of the query point p i , extract its neighborhood structure differences and perform feature encoding to obtain the point cloud feature representation f i .

[0077] The encoding method is as follows:

[0078]

[0079] where, represents the three-dimensional geometric structure difference between points, and Concat(·) represents the concatenation operation, This represents the geometric relationship between a point and its neighborhood. Represents the real number field.

[0080] Subsequently, a shared multi-layer perceptron (MLP) and softmax operation are used to perform nonlinear transformation and weight assignment on the visual features to obtain the weights:

[0081] a i =Softmax(MLP(f i ))

[0082] By weighted aggregation of K neighborhood features, the downsampled point cloud visual features f′ are obtained. i And the new downsampling feature matrix F′ t :

[0083]

[0084] As downsampling proceeds, the neighborhood information of the sampling points changes in the multi-layer encoder structure, and the receptive field gradually expands. Although deeper encoding contains richer semantic information, some local geometric details may be lost. Therefore, the fusion of multi-layer features is ensured by passing geometric relationship encoding information across different layers. Specifically, in each encoder layer, the downsampled features of the current layer are fused with the features of the previous layer through a concatenation operation:

[0085] F′=F′ t +Conv(F′ t-1 )

[0086] Among them, F′ t Let F′ represent the new downsampled point feature matrix after concatenation of the current layer, and let F′ still represent the current downsampled point cloud feature matrix. Conv(F′) t-1 The result is the convolution of the feature matrix from the previous layer. By fusing visual features from different receptive fields layer by layer, this module significantly improves the expressive power of local structural features and enhances the richness and discriminativeness of visual feature representation.

[0087] (2) Feature enhancement module based on graph attention network

[0088] In the task of separating branches and leaves from 3D point clouds of crops, accurately capturing the local geometric relationships and global contextual information of the point cloud is crucial for improving the classification and segmentation accuracy of the model. However, traditional methods usually rely only on the local neighborhood information of points, ignoring the complex geometric relationships and global semantic connections between point clouds, resulting in limitations in overall morphological representation and handling of complex category boundaries. To address these issues, this invention proposes a feature enhancement module based on graph attention networks to enhance the features of downsampled point clouds. The geometry relationship of the point cloud is explicitly depicted by adaptive neighborhood modeling, and the local visual features and global context information are efficiently integrated by means of the information transmission characteristics of the graph convolution operation and the attention mechanism, so as to improve the description ability and discrimination ability of the visual feature expression.

[0089] The module regards the down-sampling point cloud data output by the encoding layer as a node set V = {p1, p2, …, pN} of a graph, each node in the N' node set represents a sampling point, and the connection relationship between the nodes is represented by an edge set E, and a K-neighborhood algorithm is used to construct a neighborhood connection for each sampling point in the Euclidean space. N′}, and the connection relationship between the nodes is represented by an edge set E, and a K-neighborhood algorithm is used to construct a neighborhood connection for each sampling point in the Euclidean space. Based on the distance matrix, the K-neighborhood algorithm is used to search for K nearest neighbors for each point p i , and the adjacency relationship is constructed:

[0090]

[0091] To enhance the geometric expression ability, the features of the edges are encoded, and the features of each edge are jointly represented by the geometric difference and feature difference between the center point and the neighborhood points:

[0092] e ij =concat(p j -p i ,f j -f i )

[0093] Wherein, p i , p j are three-dimensional coordinates of points, and f i , f j are visual features of corresponding points in the down-sampled point cloud feature F'.

[0094] After completing the neighborhood topology modeling, the application uses a graph attention mechanism to iteratively aggregate and update the node visual features, and efficiently interacts across layers. Specifically, the model updates the node features layer by layer through neighborhood aggregation and attention weighting calculation, to ensure the preservation of local feature information and enhance the transmission of global information. The update formula is:

[0095]

[0096] Wherein, represents the node feature of the l+1 layer, represents the feature vector of the neighbor node p j in the l layer, and W (l)Let be the weight matrix of the current layer, and σ be the non-linear activation function. The K-nearest neighbor algorithm is used to construct a local neighborhood structure, ensuring the capture of local geometric features. An attention mechanism is then used to dynamically adjust the weights of neighboring points, promoting the effective transfer of information across layers.

[0097] To further optimize information representation, the model performs feature transformation based on the attention mechanism, making the feature space more segregated and enhancing the differences in local structures. Furthermore, to improve the correlation between cross-layer information, the attention mechanism not only considers the information interaction of the current layer but also integrates local and global features from different layers through a cross-scale information fusion strategy. Finally, after multi-layer fusion, the model forms a global semantic representation:

[0098]

[0099] Among them, F G Let F′ represent the final characteristic matrix after processing. (l) This represents the point cloud features at layer l, where Linear(·) represents the dimensionality reduction operation, and F′ (0) ,F′ (1) ,F′ (2) ) represents the point cloud feature matrix extracted at layers 0, 1, and 2.

[0100] This feature enhancement module based on graph attention networks, by combining graph convolution and attention mechanisms, not only enhances the fine-grained geometric representation of point clouds, but also takes into account global contextual information, effectively improving the richness and discriminativeness of feature representation.

[0101] (3) Feature decoding module based on neighborhood interpolation and multi-scale fusion

[0102] After multi-level feature extraction and enhancement, the point cloud features with graph attention enhancement are obtained. As input to the decoding stage, this invention proposes a feature decoding module based on neighborhood interpolation and multi-scale fusion to progressively restore these low-resolution features F′ to a spatial scale consistent with the original resolution while preserving previously extracted multi-level semantic information and geometric details. This module employs a K-nearest neighbor interpolation strategy based on geometric attention weighting, utilizing the local geometric relationships of the point clouds to calculate the weights of features inherited from the low-resolution point clouds by each high-resolution point cloud, and using Softmax normalization to ensure the smoothness and stability of the interpolation.

[0103] Specifically, let the set of point cloud coordinates after downsampling be {p′} j}(j=1,2,…,N′), each point p′ j Having eigenvector f′ j ∈F G The set of point cloud coordinates at the original resolution or a higher resolution layer is {p}.j The goal is to recover the feature representation of the input point cloud F with the same resolution as the input point cloud F (i = 1, 2, ..., N). For each high-resolution point p i Using the upsampling index or geometric neighborhood relationship stored in the encoding stage, select the K nearest neighbor points p′ in the low-resolution point set. j , construct its neighborhood

[0104] Then, the neighborhood features are weighted and summed using the following formula to complete the interpolation calculation:

[0105]

[0106] in:

[0107]

[0108] Where, f′ j For the midpoint p′ of the low-resolution point cloud j eigenvectors, f′ k and p′ k Let α represent the feature vector of the k-th neighbor in the low-resolution space and its coordinates, respectively, used to construct the complete attention distribution in the normalized weight calculation; ij Point p i From point p′ j Inherited attention weights, MLP stands for Multilayer Perceptron, are used to compute attention weights and perform Softmax normalization.

[0109] After interpolation and upsampling, this module employs a skip connection strategy to fully integrate multi-scale geometric and semantic information. First, through interpolation and upsampling, the output features of the previous decoding layer are mapped to the current layer to recover the feature representation of the high-resolution point cloud. Then, the upsampled features are fused with the corresponding high-resolution features from the encoding stage to compensate for potential loss of local details during deep encoding, thereby enhancing the completeness and precision of the features. Let the decoding output features of the current layer be represented as... The output features of the previous decoding layer are represented as follows The encoding features corresponding to the current layer are

[0110]

[0111] For fusion weight α l Normalize:

[0112]

[0113] where UpInterp denotes the up-sampling operation based on K-Nearest Neighbor interpolation for recovering high-resolution features; and σ denotes a nonlinear activation function for mapping the fusion weight a to the interval [0, 1].

[0114] The method not only reconstructs the geometric information close to the original resolution in space, but also effectively fuses the high-level semantics and local geometric features, thereby improving the feature integrity and expression ability in the decoding stage. With the gradual progress of multi-layer decoding, the final feature expression matched with the input point cloud resolution is obtained, thereby improving the ability to distinguish fine-grained structures such as branches and leaves in crop point clouds. Further combined with a subsequent optimization module, high-precision and stable segmentation of crop branches and leaves can be achieved in complex environments.

[0115] (4) Optimization module of inter-class discriminability discrimination and cross-entropy cooperation

[0116] In the task of crop three-dimensional point cloud branch and leaf separation, the visual feature discriminability of different categories of point clouds directly determines the classification performance of the model. Traditional optimization strategies (such as cross-entropy loss) mainly optimize the classification probability distribution through label supervision, ignoring the geometric distribution characteristics in the feature space. Such traditional optimization strategies perform well in simple scenarios, but in the case of complex class decision boundaries or highly similar feature distributions, they are prone to cause problems such as blurred class boundaries and insufficient feature discrimination, thereby limiting the generalization performance and robustness of the model. In order to solve this problem, the present application designs an optimization module of inter-class discriminability discrimination and cross-entropy cooperation, which further improves the distinguishability and discriminability of visual feature representation by explicitly optimizing the geometric distribution of the feature space.

[0117] The core idea of the class discriminability criterion is to optimize the intra-class compactness and inter-class separability of the feature space, and to realize a clearer class feature space distribution structure. Specifically:

[0118] 1. Intra-class compactness: By minimizing the feature distance of points of the same class, the features of points of the same class are more concentrated in space, reducing feature aliasing.

[0119] 2. Inter-class separability: By maximizing the feature distance between the centroids of different classes, sufficient separation is provided between different classes, thereby avoiding blurred boundaries between classes.

[0120] The inter-class discriminability discrimination optimization function is composed of three parts from the perspective of feature distribution optimization: intra-class aggregation term, inter-class separation term and centroid constraint term, and the specific formula is as follows:

[0121] 1. Intra-class aggregation term:

[0122] Since the intra-class samples may have a large dispersion in the high-dimensional feature space, if not constrained, it may lead to a loose distribution of samples within the same class in the feature space, making it more difficult to distinguish between classes. To enhance intra-class consistency, an intra-class aggregation loss is introduced to make the feature vectors of samples in the same class as close as possible to their class centroid. Specifically, the class centroid c k of each class is calculated j and a constraint is imposed to make all sample feature vectors e intra of the class move closer to the class centroid. If the distance between the sample and the class centroid exceeds the threshold τ c , a penalty will be generated, thus prompting the network to learn a tighter intra-class feature distribution. In this way, samples of the same class are no longer randomly distributed in the entire feature space, but are constrained within a smaller range, improving intra-class consistency and laying a foundation for subsequent inter-class separation. Its definition is as follows:

[0123]

[0124] where C represents the total number of classes, m j represents the feature centroid of the cth class, e c represents the feature of the jth point in the cth class, N intra represents the number of samples in the cth class, τ inter is the distance threshold for intra-class feature aggregation, and ReLU represents the positive value operation, which takes the value 0 when the input value is negative, otherwise returns the original input value. This loss minimizes the squared distance between each point and its class centroid, reduces intra-class variance, and thus improves model accuracy.

[0125] 2. Inter-class separation term:

[0126] Aggregating only intra-class samples is not enough to ensure good classification results. If the class centroids of different classes are still close, the model will still have difficulty effectively distinguishing them, which may lead to confusion at the decision boundary. Therefore, based on intra-class aggregation, the geometric structure of the feature space is further optimized, and an inter-class separation loss is introduced to ensure that features of different classes maintain sufficient distance in the feature space, thereby enhancing inter-class separability. Specifically, this loss calculates the Euclidean distance between class centroids and imposes a minimum separation constraint to ensure that the distance between the centroids of all classes is at least greater than the threshold τ inter , otherwise a penalty will be imposed. This optimization strategy is particularly important in high-class density or close-class distribution datasets, as it can effectively reduce inter-class overlap and improve the stability and accuracy of the classification task.

[0127]

[0128] where, and respectively represent the centroids of class c and class n, denotes the Euclidean distance between the two class centroids, τ inter is the threshold of inter-class separation distance. This loss suppresses the inter-class overlap by increasing the distance between centroids of different classes, thus improving the class discrimination ability of the feature space to ensure that the model can learn clearer classification boundaries.

[0129] 3. Centroid constraint term:

[0130] However, in the process of optimizing inter-class separation and intra-class aggregation, if the class centroids are not properly regularized, the centroids may be shifted too much, affecting the stability and training effect of the model. To this end, we introduce the centroid constraint term loss to limit the shift range of the class centroid, prevent it from shifting too much in the training process, and cause the instability of the class distribution. If not constrained, some class centroids may be pulled too far, affecting the balance of the overall optimization process. Therefore, the centroid regularization not only controls the scale of the centroid, so that the feature space of the model maintains a reasonable structure, but also improves the stability of the training, avoiding training failure due to gradient explosion or unstable convergence.

[0131]

[0132] where, denotes the centroid of the c-th class. This loss limits the shift of the class centroid by applying a small pull to the class centroid, maintaining the boundedness and stability of the feature distribution.

[0133] Combining the above three parts, the total goal of the discriminative loss is defined as:

[0134]

[0135] where δ, β, γ are weight coefficients for balancing the contributions of the three loss parts. In the entire optimization process, intra-class aggregation makes the features of the same class more compact and reduces the dispersion; inter-class separation ensures the discrimination of different classes and enhances the feature separability; and centroid regularization avoids excessive shift of the centroid, maintaining the stability of the optimization. It is due to the synergistic effect of the three that the feature space is both compact and has clear class boundaries, laying the foundation for the efficient performance of the model in the classification task.

[0136] In addition, to supervise the classification task, the invention adopts Cross-Entropy Loss as the basic classification loss, which is defined as:

[0137]

[0138] where C is the number of classes, t n denotes the predicted value, represents the true label. The prediction probability distribution of the model gradually approaches the true probability distribution by minimizing

[0139] To achieve the synergy of the classification task and the optimization of the feature space, the class separability discrimination and the cross-entropy jointly act, and the two promote and constrain each other, so the total loss function is defined as:

[0140]

[0141] wherein, λ is a weight coefficient, used to balance the relative importance of class separability discrimination and cross-entropy.

[0142] The present application adopts cross-entropy loss as the main optimization target, and introduces class separability discrimination as a supervision signal, to synergistically optimize the expression ability of the high-dimensional feature space. Cross-entropy ensures that the model converges to the correct class distribution, providing strong supervision signals, while class separability discrimination optimizes the feature structure, reduces feature aliasing by enhancing intra-class aggregation and expanding inter-class separation, and stabilizes the feature distribution through centroid constraint, improving classification accuracy. Experimental verification shows that in the case of fuzzy class boundary, uneven data distribution or highly similar features, the optimization strategy can effectively enhance the generalization ability of the model and improve the precision and stability of crop three-dimensional point cloud branch and leaf separation. This design not only applies to crop three-dimensional point cloud branch and leaf separation tasks, but also has good universality and can be extended to point cloud classification, target detection and other tasks, providing theoretical support and practical guidance for high-dimensional feature space optimization.

[0143] It should be apparent to those skilled in the art that the application is not limited to the details of the above-described exemplary embodiments, but can be implemented in other concrete forms without departing from the spirit or essential characteristics of the application. Therefore, the embodiments should be considered exemplary and non-limiting, and the scope of the application is defined by the appended claims rather than the above description, and all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the application. Any reference signs in the claims should not be considered as limiting the claims involved.​

Claims

1. A crop three-dimensional point cloud branch and leaf separation method based on a semantic prototype-driven graph attention network, characterized in that: The multi-level feature coding module, the feature enhancement module, the feature decoding module and the optimization module are constructed. The multi-level feature coding module adopts random down-sampling and attention pooling mechanism to extract local geometric features, reduces the size of the point cloud by randomly generating a pooling index, and aggregates neighborhood features by using geometric topology coding. The feature enhancement module constructs a dynamic topology structure based on K-neighbor algorithm, aggregates node features in multiple levels by graph attention network, establishes dual constraints of geometric difference perception and feature association, and realizes deep integration of local and global context information. The feature decoding module uses K-neighbor interpolation based on attention weighting to gradually up-sample low-resolution features to reconstruct high-resolution features, and weights and fuses the corresponding high-resolution features in the encoding stage through a skip connection. The optimization module improves the distribution structure of the class feature space by inter-class separability discriminant optimization function, and enhances the aggregation of the same class points from three aspects of intra-class aggregation, inter-class separation and centroid constraint. Calculate the class centroid c for each category. k And impose constraints such that the feature vector e of all samples belonging to that category j The sample moves closer to the class centroid if the distance between the sample and the class centroid exceeds a threshold τ. intra This will result in a penalty, specifically an intra-class aggregation loss. The definition is as follows: where C denotes the total number of classes, denotes the feature centroid of the c-th class, e j denotes the feature of the j-th point in the c-th class, N c denotes the number of samples in the c-th class, ReLU denotes a rectified linear unit operation that returns the input value as is if it is positive, otherwise returns 0. inter-class separation term wherein, and denote the centroids of class c and class n, respectively, denotes the Euclidean distance between the centroids of the two classes. centroid constraint term Total goal of discriminative loss is defined as: Wherein, δ, β, γ are weight coefficients.

2. The crop three-dimensional point cloud branch and leaf separation method of the semantic prototype-driven graph attention network according to claim 1, characterized in that, The multi-level feature coding module adopts a random sampling method of pre-generated sub-sampling and neighborhood index to retain the core three-dimensional structure information of crop plants: Set the visual feature of input point cloud where N is the number of sampling points, D is the dimension of visual feature of each sampling point, first randomly generate the index set The random sampling process ensures that each sampling point is selected with equal probability, for the i-th sampling point p i ∈S, find the k-neighborhood point set in Euclidean space by K-neighborhood algorithm In the feature extraction stage, the neighborhood index is used to complete feature aggregation by attention pooling, and the down-sampled point cloud visual feature f' i is calculated, and the down-sampled sampling point feature matrix is generated N' represents the number of sampling points after random down-sampling, represents the real number field.

3. The crop three-dimensional point cloud branch and leaf separation method of the semantic prototype-driven graph attention network according to claim 2, characterized in that, In the down-sampling process, an attention pooling mechanism is introduced: Using K-Nearest Neighbors algorithm on query point p i Constructing local neighborhood, extracting its neighborhood structure difference and feature encoding, obtaining point cloud feature representation f i : wherein, denotes the three-dimensional geometric structure difference between points, Concat(·) denotes the concatenation operation, denotes the geometric relationship between the query point and the neighborhood; The visual features are non-linearly transformed and weight allocated by using shared multi-layer perceptron (MLP) and Softmax operation to obtain weights a i : a i = Softmax(MLP(f i )) By weighting and aggregating K neighborhood features, a down-sampled point cloud visual feature f' is obtained i and a new down-sampled point feature matrix F' t : In each encoder layer, the down-sampled point feature matrix F t with the feature matrix F t-1 is fused by a concatenation operation: F' = F t + Conv(F t-1 ) wherein Conv(F t-1 ) is the result of the previous layer feature matrix after convolution.

4. The crop three-dimensional point cloud branch and leaf separation method of the semantic prototype-driven graph attention network according to claim 3, characterized in that, The feature enhancement module regards the down-sampled point cloud data output by the encoding layer as a node set V = {p1, p2, …, p N'} of a graph, each node in the N' node sets represents a sampling point, and the connection relationship between the nodes is represented by an edge set E. A K-neighbor algorithm is used to construct a neighborhood connection for each sampling point in the Euclidean space. Compute the distance between all pairs of points in the point cloud, resulting in a distance matrix Search for the K-nearest neighbors of each sample point p i Find its K nearest neighbors and construct the adjacency relation E: The features of the edges are encoded, e ij The features of the edges are encoded, e ij The features of the edges are encoded, e ij The features of the edges are encoded, e ij The features of the edges are encoded, e ij The features of the edges are encoded, e < e ij = concat(p j -p i ,f j -f i ) where p i , p j are the three-dimensional coordinates of the i, j-th sampling point, f i , f j are the visual features of the corresponding points in the down-sampled point feature matrix F'.

5. The crop three-dimensional point cloud branch and leaf separation method of the semantic prototype-driven graph attention network according to claim 4, characterized in that, After completing the neighborhood topology modeling, the node visual features are iteratively aggregated and updated based on the graph attention mechanism, the node features are updated layer by layer through neighborhood aggregation and attention weighting calculation, and the update formula is: in, Represents the node features of the (l+1)th layer. Represents the neighbor node p in the l-th layer j eigenvectors, W (l) Let σ be the weight matrix of the current layer l, and σ be the non-linear activation function.

6. The crop three-dimensional point cloud branch and leaf separation method of the semantic prototype-driven graph attention network according to claim 5, characterized in that, Based on the attention mechanism, feature transformation is performed to integrate local and global features at different levels and form global semantic representation: where F G represents the final feature matrix after processing, F '(0) , '(1) , '(2) represents the point cloud feature matrix extracted at the 0th, 1st, 2nd layers, Linear(·) represents the dimension reduction operation.

7. The crop three-dimensional point cloud branch and leaf separation method of the semantic prototype-driven graph attention network according to claim 6, characterized in that, The feature decoding module adopts K-neighbor interpolation strategy based on geometric attention weighting, uses the local geometric relationship of point cloud to calculate the weight of each high-resolution point cloud inherited from the low-resolution point cloud, and normalizes it by Softmax; Let the set of point cloud coordinates after downsampling be {p' j }, j = 1, 2, ..., N', each sampling point p' j Having eigenvector f' j ∈F G The set of point cloud coordinates at the original resolution or a higher resolution layer is {p}. i For each high-resolution point p i Using the upsampling index or geometric neighborhood relationship stored in the encoding stage, select the K nearest neighbor points p' in the low-resolution point set. j , construct its neighborhood The interpolation result is obtained by weighted sum of neighborhood features by the following formula Wherein: where f j is the feature vector of point p j in the low-resolution point cloud, f k and p k denote the feature vector and the point coordinate of the k-th neighbor point in the low-resolution space, respectively, and a ij is the attention weight inherited by point p i from point p j , and MLP denotes a multi-layer perceptron.

8. The crop three-dimensional point cloud branch and leaf separation method of the semantic prototype-driven graph attention network according to claim 7, characterized in that, The feature decoding module adopts a skip connection strategy: The output feature of the previous decoding layer is mapped to the current layer by interpolation upsampling, the feature obtained by upsampling is fused with the high-resolution feature corresponding to the encoding stage, and the decoding output feature of the current layer is denoted as The output feature of the previous decoding layer is denoted as The encoding feature corresponding to the current layer is denoted as The fusion weight is α l Normalization is performed: Wherein, UpInterp represents the up-sampling operation based on K-neighbor interpolation, and σ represents a nonlinear activation function.

9. The crop three-dimensional point cloud branch and leaf separation method of the semantic prototype-driven graph attention network according to claim 1, characterized in that, Adopting cross-entropy loss As a base classification loss: where C is the number of classes, n is the class index, t n denotes the predicted value, denotes the true label, minimizing the model's predicted probability distribution gradually approaches the true probability distribution; Total loss function is defined as: Wherein, λ is a weight coefficient.

Images