A Leaf Point Cloud Completion Method Based on Multi-Level Attention Fusion and Edge Structure Guidance

Abstract

This invention provides a leaf point cloud completion method based on multi-layer attention fusion and edge structure guidance, comprising: acquiring and preprocessing a 3D point cloud dataset of plant leaves; establishing a leaf point cloud completion network; inputting the preprocessed 3D point cloud dataset of plant leaves into the established leaf point cloud completion network, and using a preset total loss function to guide the leaf point cloud completion network for iterative training; finally, inputting the plant leaf point cloud to be completed into the trained leaf point cloud completion network for completion; this invention designs an encoder-decoder with an attention mechanism for the overall leaf structure, and integrates multi-layer attention-weighted features through a fusion module to improve the model's perception of leaf structure, enabling the network to better complete the leaf point cloud completion task; for the leaf sheet structure, an edge loss function is designed to guide the model to learn leaf edge information, thereby improving the accuracy of plant leaf morphology research.

Description

Technical Field

[0001] This invention relates to the fields of point cloud processing and deep learning technology, and more specifically, to a leaf point cloud completion method based on multi-layer attention fusion and edge structure guidance. Background Technology

[0002] Smart agriculture represents a significant advancement in agricultural informatization, progressing from digitalization to networking and then to intelligence. It is a milestone in agricultural development and has become a global trend in modern agriculture. Plant leaves, as key organs for plant growth and metabolism, are crucial for understanding plant ecology and growth processes. In recent years, advancements in 3D laser scanning and structured light technologies have made it possible to acquire 3D point cloud data of plant leaves, providing new avenues for in-depth research on plant morphology and growth. However, due to limitations in environment and equipment, the acquired point clouds often contain significant gaps.

[0003] Currently, the commonly used solution is a 3D point cloud completion method based on deep learning. Deep learning leverages its powerful representation learning capabilities to learn the mapping from missing point clouds to complete point clouds. The earliest point cloud shape completion network, PCN, directly completes the original point cloud, generating a coarse-grained sparse complete point cloud and a fine-grained dense complete point cloud in two stages. Later, TopNet introduced a hierarchical tree structure network for point cloud generation, eliminating the need to assume any specific topology on the base point set. In recent years, generative networks have become the mainstream network for point cloud completion. PF-Net utilizes a multi-scale generative network based on feature points to perform hierarchical estimation of partial point clouds, allowing the network to predict the missing parts of the point cloud. In addition, existing patent documents also disclose a point cloud completion method based on deep learning. This method includes: preprocessing the original point cloud set; constructing an encoder-decoder network; inputting the preprocessed point cloud data into the encoder to obtain the feature vector of the input point cloud; inputting the feature vector of the input point cloud into the decoder to obtain the missing part of the point cloud; concatenating the missing part of the point cloud with the input incomplete point cloud to obtain the complete point cloud; setting a loss function, optimizing the network, and obtaining the final optimized output result; and finally classifying based on the PointNet network.

[0004] However, the existing solutions mentioned above only perform well on datasets with regular shapes and strong structures. Since plant leaf point clouds are usually sheet-like structures with uneven surfaces, the above solutions are not very sensitive to the features of plant leaf point clouds, and the shapes are usually blurred at the leaf base and leaf tip after completion. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies in completing plant leaf point clouds, which result in blurred shapes at the leaf base and tip, this invention provides a leaf point cloud completion method based on multi-layer attention fusion and edge structure guidance. This method can improve the leaf completion effect, enhance the accuracy of obtaining plant phenotypic information in agricultural automation, and reduce the error between measured data and real data during automated monitoring.

[0006] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0007] A leaf point cloud completion method based on multi-layer attention fusion and edge structure guidance includes the following steps:

[0008] S1: Obtain the 3D point cloud dataset of plant leaves and perform preprocessing;

[0009] S2: Establish a complete point cloud network for the leaf blades;

[0010] The leaf point cloud completion network includes an encoder and a decoder connected in sequence;

[0011] The encoder includes: an upsampling module, a downsampling module, and a multi-layer attention fusion module connected in sequence;

[0012] S3: Input the preprocessed 3D point cloud dataset of plant leaves into the established leaf point cloud completion network, and use the preset total loss function to guide the leaf point cloud completion network to perform iterative training to obtain the trained leaf point cloud completion network.

[0013] The preset total loss function includes: the edge loss function and the overall loss function;

[0014] S4: Obtain the point cloud of the plant leaf to be completed, input the point cloud of the plant leaf to be completed into the trained leaf point cloud completion network for completion, and complete the plant leaf point cloud completion.

[0015] Preferably, the preprocessing in step S1 includes:

[0016] The plant leaf 3D point cloud dataset is filtered to remove the point clouds of withered and yellow leaves and miscellaneous point sets other than leaves, and the filtered plant leaf 3D point cloud dataset is obtained.

[0017] The plant leaf 3D point cloud dataset after data filtering is subjected to mean filtering and smoothing and point cloud coordinate normalization in sequence to obtain the preprocessed plant leaf 3D point cloud dataset, thus completing the preprocessing.

[0018] Preferably, in step S2, the encoder downsampling module includes: a first local feature extraction submodule SA and a second local feature extraction submodule SA with the same structure and connected in sequence;

[0019] The output of the upsampling module is connected to the input of the first local feature extraction submodule SA, and the outputs of the first local feature extraction submodule SA and the second local feature extraction submodule SA are respectively connected to the input of the multi-layer attention fusion module.

[0020] Preferably, each of the local feature extraction submodules SA includes: a sampling layer, a grouping layer, and a feature encoding layer; the sampling layer uses the farthest point sampling algorithm (FPS) to sample the input point cloud to obtain a set of sampled points; the grouping layer uses the K-nearest neighbor algorithm to cluster the sampled point set to obtain a set of point sets with several local centers; the feature encoding layer uses a multilayer perceptron (MLP) to encode the features of all the point sets with local centers to obtain local features of the point cloud, and uses the local features of the point cloud as the output of the local feature extraction submodule SA.

[0021] Preferably, in step S2, the multi-layer attention fusion module of the encoder includes: a first attention sub-module, a second attention sub-module, and a convolutional fusion sub-module;

[0022] The first attention submodule and the second attention submodule have the same structure and are arranged side by side. The outputs of the first attention submodule and the second attention submodule are respectively connected to the input of the convolution fusion submodule. The input of the first attention submodule is connected to the output of the first local feature extraction submodule SA, and the input of the second attention submodule is connected to the output of the second local feature extraction submodule SA.

[0023] Preferably, each attention submodule includes, in sequence, several structurally identical OffSet-Transformer blocks and a convolutional block; the input of each OffSet-Transformer block is also connected to the input of the convolutional block.

[0024] In each OffSet-Transformer block, the input data is processed through three parallel first multilayer perceptrons (MLPs) to obtain data 1, data 2, and data 3. Data 2 is then matrix-transposed and multiplied with data 1 to obtain data 4. Data 4 is processed by a softmax operation and recorded as data 5. The second dimension of data 5 is summed to obtain data 6. Data 5 and data 6 are then divided by a matrix to obtain data 7. Data 3 and data 7 are then multiplied by a matrix to obtain data 8. The input data and data 8 are then subtracted by a matrix to obtain data 9. Data 9 is then processed through a second multilayer perceptron (MLP) and the first batch of normalized batch normalization (BN) layers to obtain data 10. The input data and data 10 are then matrix-added to obtain the output data of the OffSet-Transformer block. The input data, data 1 to 10, and output data are all three-dimensional vectors.

[0025] The convolutional block comprises, in sequence: a third multilayer perceptron (MLP), a second batch normalized (BN) layer, and a ReLU activation function layer;

[0026] The convolutional fusion submodule includes: a fourth multilayer perceptron (MLP).

[0027] Preferably, each attention submodule contains four OffSet-Transformer blocks.

[0028] Preferably, in step S3, the edge loss function L1 is specifically:

[0029] L1 = EMD1(P g_edge ,P f_edge )

[0030] Among them, P g_edge P represents the edge truth value of the leaf point cloud; f_edge EMD1 represents the predicted edge value of the leaf point cloud after completion; EMD1 represents the bulldozing distance between the predicted edge value and the true edge value of the leaf point cloud after completion.

[0031] The overall loss function L2 is specifically:

[0032] L2=EMD2(P gt ,P fake )

[0033] Among them, P gt P represents the truth value of the leaf point cloud. fake EMD1 represents the predicted value of the leaf point cloud after completion; EMD2 represents the bulldozing distance between the predicted value and the true value of the leaf point cloud after completion.

[0034] The preset total loss function Loss is specifically as follows:

[0035] Loss = αL1 + (1-α)L2

[0036] Where α is a hyperparameter.

[0037] This invention also provides a blade point cloud completion system based on multi-layer attention fusion and edge structure guidance. The system applies the aforementioned blade point cloud completion method based on multi-layer attention fusion and edge structure guidance, and includes:

[0038] Data acquisition and preprocessing unit: used to acquire and preprocess the 3D point cloud dataset of plant leaves;

[0039] Model building unit: used to build the leaf point cloud completion network;

[0040] The leaf point cloud completion network includes an encoder and a decoder connected in sequence;

[0041] The encoder includes: an upsampling module, a downsampling module, and a multi-layer attention fusion module connected in sequence;

[0042] Model training unit: used to input the preprocessed 3D point cloud dataset of plant leaves into the established leaf point cloud completion network, and use a preset total loss function to guide the leaf point cloud completion network to perform iterative training to obtain the trained leaf point cloud completion network;

[0043] The preset total loss function includes: the edge loss function and the overall loss function;

[0044] Leaf point cloud completion unit: Used to acquire the plant leaf point cloud to be completed, input the plant leaf point cloud to be completed into the trained leaf point cloud completion network for completion, and complete the plant leaf point cloud.

[0045] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described method.

[0046] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0047] This invention provides a leaf point cloud completion method based on multi-layer attention fusion and edge structure guidance. First, a 3D point cloud dataset of plant leaves is acquired and preprocessed. Then, a leaf point cloud completion network is established. The preprocessed 3D point cloud dataset of plant leaves is input into the established leaf point cloud completion network, and a preset total loss function guides the network to undergo iterative training to obtain a trained leaf point cloud completion network. Finally, the plant leaf point cloud to be completed is acquired and input into the trained leaf point cloud completion network for completion, thus completing the plant leaf point cloud.

[0048] This invention proposes a plant leaf point cloud completion network based on multi-layer attention fusion, which improves the network's ability to learn features related to plant leaf shape. For the overall leaf structure, an encoder-decoder with an attention mechanism is designed, fusing multi-layer attention-weighted features through a fusion module to enhance the model's perception of leaf structure, enabling the network to better complete the leaf point cloud completion task. For the leaf sheet structure, an edge loss function is designed to guide the model in learning leaf edge information, thereby improving the accuracy of plant leaf morphology research and providing more reliable data support for research in ecology and plant biology, while also providing strong support for the realization of intelligent agriculture. Attached Figure Description

[0049] Figure 1 This is a flowchart of a leaf point cloud completion method based on multi-layer attention fusion and edge structure guidance provided in Example 1.

[0050] Figure 2 This is an architecture diagram of a leaf point cloud completion method based on multi-layer attention fusion and edge structure guidance provided in Example 2.

[0051] Figure 3 This is a structural diagram of the downsampling module and the multi-layer attention fusion module in the leaf point cloud completion network encoder provided in Example 2.

[0052] Figure 4 This is a structural diagram of the leaf point cloud completion network decoder provided in Example 2.

[0053] Figure 5 This is a comparison chart of experimental results between the method provided in Example 2 and three existing point cloud completion methods.

[0054] Figure 6 This is a structural diagram of a leaf point cloud completion system based on multi-layer attention fusion and edge structure guidance provided in Example 3. Detailed Implementation

[0055] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent.

[0056] To better illustrate this embodiment, some parts in the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions;

[0057] It will be understood by those skilled in the art that certain well-known structures and their descriptions may be omitted in the accompanying drawings.

[0058] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0059] Example 1

[0060] like Figure 1 As shown, this embodiment provides a leaf point cloud completion method based on multi-layer attention fusion and edge structure guidance, including the following steps:

[0061] S1: Obtain the 3D point cloud dataset of plant leaves and perform preprocessing;

[0062] S2: Establish a complete point cloud network for the leaf blades;

[0063] The leaf point cloud completion network includes an encoder and a decoder connected in sequence;

[0064] The encoder includes: an upsampling module, a downsampling module, and a multi-layer attention fusion module connected in sequence;

[0065] S3: Input the preprocessed 3D point cloud dataset of plant leaves into the established leaf point cloud completion network, and use the preset total loss function to guide the leaf point cloud completion network to perform iterative training to obtain the trained leaf point cloud completion network.

[0066] The preset total loss function includes: the edge loss function and the overall loss function;

[0067] S4: Obtain the point cloud of the plant leaf to be completed, input the point cloud of the plant leaf to be completed into the trained leaf point cloud completion network for completion, and complete the plant leaf point cloud completion.

[0068] In the specific implementation process, the three-dimensional point cloud dataset of plant leaves is first obtained and preprocessed to obtain the preprocessed dataset, and the preprocessed dataset is divided into training set, validation set and test set.

[0069] A leaf point cloud completion network is established. In this embodiment, an encoder-decoder structure is used as the main network architecture. Two attention mechanisms are introduced in the encoder part. The two attention mechanisms will process features at different levels in sequence and finally perform feature fusion through a convolutional layer.

[0070] The preprocessed 3D point cloud dataset of plant leaves is input into the established leaf point cloud completion network. The preset total loss function guides the leaf point cloud completion network to perform iterative training. The model weights are trained using the built network. After training is repeated several times, a converged model is obtained and saved as the trained leaf point cloud completion network. The performance of the trained model is evaluated using a test set. In this embodiment, the model performance is measured by calculating the chamfer distance (CD) and earth mover's distance (EMD).

[0071] The preset total loss function in this embodiment includes an edge loss function and an overall loss function. To enhance the learning of the blade edge structure, the network is guided by the edge loss function. By extracting the edges of the obtained complete blade and the ground truth respectively, the bulldozing distance between the two is calculated to guide the network to learn edge information. At the same time, in order to prevent the network from processing the point cloud to only the shape of the blade edge in order to obtain the lowest EMD, this embodiment also introduces the overall EMD of the blade as part of the total loss function, balancing the learning ability of the network for the overall blade and the edges.

[0072] Finally, the point cloud of the plant leaf to be completed is obtained, and the point cloud of the plant leaf to be completed is input into the trained leaf point cloud completion network for completion, thus completing the plant leaf point cloud.

[0073] This embodiment provides a leaf point cloud completion method based on multi-layer attention fusion and edge structure guidance, which effectively solves the problem of blurred leaf edge shape completion. By improving the shape completion effect of leaf point cloud, it improves the accuracy of agricultural automated monitoring and measurement.

[0074] Example 2

[0075] This embodiment provides a leaf point cloud completion method based on multi-layer attention fusion and edge structure guidance, including the following steps:

[0076] S1: Obtain the 3D point cloud dataset of plant leaves and perform preprocessing;

[0077] S2: Establish a complete point cloud network for the leaf blades;

[0078] The leaf point cloud completion network includes an encoder and a decoder connected in sequence;

[0079] The encoder includes: an upsampling module, a downsampling module, and a multi-layer attention fusion module connected in sequence;

[0080] S3: Input the preprocessed 3D point cloud dataset of plant leaves into the established leaf point cloud completion network, and use the preset total loss function to guide the leaf point cloud completion network to perform iterative training to obtain the trained leaf point cloud completion network.

[0081] The preset total loss function includes: the edge loss function and the overall loss function;

[0082] S4: Obtain the point cloud of the plant leaf to be completed, input the point cloud of the plant leaf to be completed into the trained leaf point cloud completion network for completion, and complete the completion of the plant leaf point cloud.

[0083] The preprocessing in step S1 includes:

[0084] The plant leaf 3D point cloud dataset is filtered to remove the point clouds of withered and yellow leaves and miscellaneous point sets other than leaves, and the filtered plant leaf 3D point cloud dataset is obtained.

[0085] The plant leaf 3D point cloud dataset after data filtering is subjected to mean filtering and smoothing and point cloud coordinate normalization in sequence to obtain the preprocessed plant leaf 3D point cloud dataset, thus completing the preprocessing.

[0086] In step S2, the encoder downsampling module includes: a first local feature extraction submodule SA and a second local feature extraction submodule SA with the same structure and connected in sequence;

[0087] The output of the upsampling module is connected to the input of the first local feature extraction submodule SA, and the outputs of the first local feature extraction submodule SA and the second local feature extraction submodule SA are respectively connected to the input of the multi-layer attention fusion module.

[0088] Each of the local feature extraction submodules (SA) includes a sampling layer, a grouping layer, and a feature encoding layer. The sampling layer uses the farthest point sampling algorithm (FPS) to sample the input point cloud and obtain a set of sampled points. The grouping layer uses the K-nearest neighbor algorithm to cluster the sampled point set and obtain a set of point sets with several local centers. The feature encoding layer uses a multilayer perceptron (MLP) to encode the features of all the point sets with local centers, obtain the local features of the point cloud, and use the local features of the point cloud as the output of the local feature extraction submodule (SA).

[0089] In step S2, the multi-layer attention fusion module of the encoder includes: a first attention sub-module, a second attention sub-module, and a convolutional fusion sub-module;

[0090] The first attention submodule and the second attention submodule have the same structure and are arranged side by side. The outputs of the first attention submodule and the second attention submodule are respectively connected to the input of the convolution fusion submodule. The input of the first attention submodule is connected to the output of the first local feature extraction submodule SA, and the input of the second attention submodule is connected to the output of the second local feature extraction submodule SA.

[0091] Each attention submodule includes, in sequence, four identical OffSet-Transformer blocks and a convolutional block; the input of each OffSet-Transformer block is also connected to the input of the convolutional block.

[0092] In each OffSet-Transformer block, the input data is processed through three parallel first multilayer perceptrons (MLPs) to obtain data 1, data 2, and data 3. Data 2 is then matrix-transposed and multiplied with data 1 to obtain data 4. Data 4 is processed by a softmax operation and recorded as data 5. The second dimension of data 5 is summed to obtain data 6. Data 5 and data 6 are then divided by a matrix to obtain data 7. Data 3 and data 7 are then multiplied by a matrix to obtain data 8. The input data and data 8 are then subtracted by a matrix to obtain data 9. Data 9 is then processed through a second multilayer perceptron (MLP) and the first batch of normalized batch normalization (BN) layers to obtain data 10. The input data and data 10 are then matrix-added to obtain the output data of the OffSet-Transformer block. The input data, data 1 to 10, and output data are all three-dimensional vectors.

[0093] The convolutional block comprises, in sequence: a third multilayer perceptron (MLP), a second batch normalized (BN) layer, and a ReLU activation function layer;

[0094] The convolutional fusion submodule includes: a fourth multilayer perceptron (MLP);

[0095] In step S3, the edge loss function L1 is specifically:

[0096] L1 = EMD1(P g_edge ,P f_edge )

[0097] Among them, P g_edge P represents the edge truth value of the leaf point cloud; f_edge EMD1 represents the predicted edge value of the leaf point cloud after completion; EMD1 represents the bulldozing distance between the predicted edge value and the true edge value of the leaf point cloud after completion.

[0098] The overall loss function L2 is specifically:

[0099] L2=EMD2(P gt ,P fake )

[0100] Among them, P gt P represents the truth value of the leaf point cloud. fake EMD1 represents the predicted value of the leaf point cloud after completion; EMD2 represents the bulldozing distance between the predicted value and the true value of the leaf point cloud after completion.

[0101] The preset total loss function Loss is specifically as follows:

[0102] Loss = αL1 + (1-α)L2

[0103] Wherein, α is a hyperparameter, and in this embodiment, α = 0.5.

[0104] In the specific implementation process, the three-dimensional point cloud dataset of plant leaves is first acquired and preprocessed to obtain the preprocessed dataset. The preprocessing in this embodiment includes data filtering, smoothing filtering and normalization. Data filtering is to remove withered yellow leaves and other miscellaneous point sets besides leaves. Then, mean filtering is used to smooth the acquired point cloud, and the coordinates of the smoothed point cloud are normalized to scale the coordinate values ​​of its three dimensions to the interval [-1,1].

[0105] After preprocessing, the dataset is divided into three parts: training set, validation set, and test set, with a ratio of 2:1:1.

[0106] Establish a leaf point cloud completion network; such as Figure 2 As shown, this embodiment uses an encoder-decoder structure as the main network architecture. Two attention mechanisms are introduced in the encoder part. The two attention mechanisms will process features at different levels in sequence, and finally perform feature fusion through a convolutional layer.

[0107] like Figure 3 As shown, the encoder includes an upsampling module, a downsampling module, and a multi-layer attention fusion module connected in sequence. First, a convolutional layer is used to increase the feature dimension of the input point cloud, and then the input downsampling module downsamples the point cloud. The downsampling module includes two local feature extraction sub-modules SA (Set Abstraction) with the same structure and connected in sequence, which extract features at different levels step by step. SA is a local feature extraction module proposed in PointNet++.

[0108] Each local feature extraction submodule (SA) comprises three parts: a sampling layer, a grouping layer, and an embedding layer. The sampling layer uses the Farthest Point Sampling (FPS) algorithm to sample the input point cloud, randomly selecting N1 center points from the original point cloud to obtain a set of sampled points. The grouping layer clusters the sampled point sets, using the K-nearest neighbor algorithm to calculate the K nearest neighbors around each center point as the point set of that center point, obtaining a set of N1 local center point sets. The embedding layer uses a Multi-layer Perceptron (MLP) to encode the features of all the local center point sets, obtaining local features of the point cloud, and uses these local features as the output of the local feature extraction submodule (SA).

[0109] In the encoder, local features are extracted layer by layer. This embodiment uses two layers of local feature extraction. In the first layer of local feature extraction module, N1 is 512, forming a set of 512 local center points. In the second layer of local feature extraction module, N1 is 256, that is, 256 center points are extracted again from the set of 512 center points extracted in the first layer to form the second layer of local point set.

[0110] Since the details of the leaf base and tip are difficult for a network to learn, the attention mechanism Transformer block used in this embodiment is the OffSet-Transformer block proposed in the PCT (Point Cloud Transformer) network. Compared with the traditional Transformer, the OffSet-Transformer uses an "offset attention" mechanism, which learns the topology and geometric features of the point cloud based on the offset between points, and captures the spatial details in the point cloud.

[0111] like Figure 3 As shown, the multi-layer attention fusion module in this embodiment includes: a first attention submodule, a second attention submodule, and a convolutional fusion submodule; the first attention submodule and the second attention submodule have the same structure and are arranged side by side, and the outputs of the first attention submodule and the second attention submodule are respectively connected to the input of the convolutional fusion submodule; the input of the first attention submodule is connected to the output of the first local feature extraction submodule SA, and the input of the second attention submodule is connected to the output of the second local feature extraction submodule SA;

[0112] Each attention submodule includes, in sequence, four identical OffSet-Transformer blocks and a convolutional block; the input of each OffSet-Transformer block is also connected to the input of the convolutional block.

[0113] In each OffSet-Transformer block, the input data passes through three parallel first multilayer perceptrons (MLPs) to obtain data 1, data 2, and data 3 respectively. Data 2 is then matrix-transposed and multiplied with data 1 to obtain data 4. The result of the Softmax operation on data 4 is denoted as data 5. The matrix weights of data 5 can be denoted as (b, 512, 512). The second dimension of data 5 is summed to obtain data 6, which can be denoted as (b, 1, 512). Data 5 and data 6 are then divided by a matrix to obtain data 7, which is the sum of each element in the output data 5 from the Softmax operation. Divide each value by the calculated weight sum (data 6) to obtain the attention score (data 7). The calculated attention score is (b, 512, 512), where b is the batch size. Multiply the data 3 and data 7 by a matrix to obtain data 8. Subtract the input data from data 8 by a matrix to obtain data 9. Pass the data 9 through the second multilayer perceptron (MLP) and the first batch of normalized batch normalization (BN) layers to obtain data 10. Add the input data to data 10 by a matrix to obtain the output data of the Offset-Transformer block. The input data, data 1 to 10, and output data are all three-dimensional vectors.

[0114] The convolutional block consists of, in sequence: a third multilayer perceptron (MLP), a second batch normalized (BN) layer, and a ReLU activation function layer; the convolutional fusion submodule consists of: a fourth multilayer perceptron (MLP);

[0115] The decoder used in this embodiment is based on a fully connected network, and its purpose is to reconstruct the point cloud coordinates from the encoder. The decoder's input is a 1024-dimensional vector representing the global features of the point cloud, encoded by the encoder, and its output is a matrix composed of N three-dimensional coordinates, where N represents the number of points in the output point cloud. Figure 4 As shown, the decoder first passes the input vector through a linear layer and a batch normalization layer, using the ReLU activation function to obtain a 2048-dimensional vector. Then, the vector is randomly deactivated by 0.5 to prevent overfitting. The above calculation is repeated to obtain a 4096-dimensional feature vector. Finally, this vector is concatenated with the original input vector and fed into the last linear layer to obtain an N×3 matrix, which is used as the decoder output.

[0116] In the point cloud completion network of this embodiment, after the downsampling module extracts features from two layers with different receptive fields, it stacks four attention layers to help the network obtain a richer and more refined point cloud representation, enhancing the network's expressive and generalization abilities and improving its performance. The multi-layer attention mechanism can progressively extract and fuse information from different levels of input features. This helps the model capture both local and global features of the data and selectively focus on important information through the attention mechanism. On the other hand, the outputs of the two attention modules are fused through convolutional layers, and the model can finally obtain feature information from different receptive fields.

[0117] The preprocessed 3D point cloud dataset of plant leaves is then input into the established leaf point cloud completion network. The preset total loss function guides the leaf point cloud completion network to perform iterative training. The model weights are trained using the constructed network. After training is repeated 1000 times, a converged model is obtained and saved as the trained leaf point cloud completion network. The performance of the trained model is evaluated using a test set. In this embodiment, the model performance is measured by calculating the chamfer distance (CD) and earth mover's distance (EMD).

[0118] The preset total loss function in this embodiment includes an edge loss function and an overall loss function. To enhance the learning of the blade edge structure, the network is guided by the edge loss function. By extracting the edges of the obtained complete blade and the ground truth respectively, the bulldozing distance between the two is calculated to guide the network to learn edge information. At the same time, in order to prevent the network from processing the point cloud to only the shape of the blade edge in order to obtain the lowest EMD, this embodiment also introduces the overall EMD of the blade as part of the total loss function, balancing the learning ability of the network for the overall blade and the edges.

[0119] The edge loss function L1 is as follows:

[0120] L1 = EMD1(P g_edge ,P f_edge )

[0121] Among them, P g_edge P represents the edge truth value of the leaf point cloud; f_edge EMD1 represents the predicted edge value of the leaf point cloud after completion; EMD1 represents the bulldozing distance between the predicted edge value and the true edge value of the leaf point cloud after completion.

[0122] The overall loss function L2 is specifically:

[0123] L2=EMD2(P gt ,P fake )

[0124] Among them, Pgt P represents the truth value of the leaf point cloud. fake EMD1 represents the predicted value of the leaf point cloud after completion; EMD2 represents the bulldozing distance between the predicted value and the true value of the leaf point cloud after completion.

[0125] The formula for calculating bulldozing distance (EMD) is:

[0126]

[0127] The preset total loss function Loss is specifically as follows:

[0128] Loss = 0.5L1 + 0.5L2

[0129] Figure 2 The training process is as follows: First, N×3 point cloud data (N is the number of points, and 3 represents the x, y, z 3D coordinate information) is input into the network; the network first uses an encoder to extract the leaf point cloud features to obtain low-dimensional features; then a decoder is used to decode the features to restore them to the N×3 point coordinate form; then, edge extraction is performed on the ground truth and the network output point cloud respectively, and the bulldozing distance between the two edges is calculated; at the same time, the overall bulldozing distance of the ground truth and the network output point cloud is calculated; then, the edge loss function and the overall loss function are calculated respectively. In this embodiment, the weight of the edge loss function is 0.5, the weight of the overall loss function is 0.5, and the two functions are added together as the total loss function to guide the network update;

[0130] Finally, the point cloud of the plant leaf to be completed is obtained, and the point cloud of the plant leaf to be completed is input into the trained leaf point cloud completion network for completion, thus completing the plant leaf point cloud.

[0131] This embodiment also provides a verification experiment to verify the effectiveness of the method:

[0132] This embodiment trains a network using training images from the point cloud dataset of Caladium leaves. 4000 samples are used as the training set to train the model, which then yields the final leaf completion result. The same dataset is used to train three existing completion methods (PCN, TopNet, and VRCNet). Compared with these methods, this approach achieves optimal bulldozing distance and Hausdorff distance in most cases, and the bulldozing distance does not significantly increase even with a higher missing value, as shown in Table 1.

[0133] Table 1 compares the experimental results of the method in this embodiment with those of three existing point cloud completion methods.

[0134]

[0135]

[0136] The completion results obtained by this method and other methods under the condition of missing proportions are as follows: Figure 5 As shown; by Figure 5 It can be seen that PCN, during the completion process, preserves a certain leaf shape and can distinguish the leaf base or tip, but the leaf structural features are not obvious. TopNet's completion effect is worse than PCN's, with the leaf base and tip shapes blurred and difficult to distinguish after completion. VCRNet's completed point cloud leaves show significant missing shapes at the leaf base. In contrast, the leaf point cloud completed by the proposed method has clearer edges and better preserves the leaf shape and sheet-like structure. Experimental results show that the proposed method, based on the morphological characteristics of plant leaves, can preserve leaf edges to a large extent and improve the completion effect.

[0137] This method proposes a plant leaf point cloud completion network based on multi-layer attention fusion, which improves the network's ability to learn features of plant leaf shape. For the overall leaf structure, an encoder-decoder with an attention mechanism is designed. By fusing multi-layer attention-weighted features through a fusion module, the model's ability to perceive leaf structure is enhanced, enabling the network to better complete the leaf point cloud completion task. For the leaf sheet structure, an edge loss function is designed to guide the model to learn leaf edge information, thereby improving the accuracy of plant leaf morphology research, providing more reliable data support for research in fields such as ecology and plant biology, and providing strong support for the realization of intelligent agriculture.

[0138] Example 3

[0139] like Figure 6 As shown, this embodiment provides a leaf point cloud completion system based on multi-layer attention fusion and edge structure guidance, applying the leaf point cloud completion method based on multi-layer attention fusion and edge structure guidance described in Embodiment 1 or 2, including:

[0140] Data acquisition and preprocessing unit 301: used to acquire and preprocess the three-dimensional point cloud dataset of plant leaves;

[0141] Model building unit 302: Used to build the leaf point cloud completion network;

[0142] The leaf point cloud completion network includes an encoder and a decoder connected in sequence;

[0143] The encoder includes: an upsampling module, a downsampling module, and a multi-layer attention fusion module connected in sequence;

[0144] Model training unit 303: is used to input the preprocessed plant leaf 3D point cloud dataset into the established leaf point cloud completion network, and use a preset total loss function to guide the leaf point cloud completion network to perform iterative training to obtain the trained leaf point cloud completion network.

[0145] The preset total loss function includes: the edge loss function and the overall loss function;

[0146] Leaf point cloud completion unit 304: Used to acquire the plant leaf point cloud to be completed, input the plant leaf point cloud to be completed into the trained leaf point cloud completion network for completion, and complete the plant leaf point cloud.

[0147] In the specific implementation process, firstly, the data acquisition and preprocessing unit 301 acquires and preprocesses the 3D point cloud dataset of plant leaves; the model building unit 302 builds a leaf point cloud completion network; the model training unit 303 inputs the preprocessed 3D point cloud dataset of plant leaves into the established leaf point cloud completion network, and uses a preset total loss function to guide the leaf point cloud completion network to perform iterative training to obtain a trained leaf point cloud completion network; finally, the leaf point cloud completion unit 304 acquires the plant leaf point cloud to be completed, inputs the plant leaf point cloud to be completed into the trained leaf point cloud completion network for completion, and completes the plant leaf point cloud completion.

[0148] This system proposes a plant leaf point cloud completion network based on multi-layer attention fusion, which improves the network's ability to learn features of plant leaf shape. For the overall leaf structure, an encoder-decoder with an attention mechanism is designed. By fusing multi-layer attention-weighted features through a fusion module, the model's perception of leaf structure is enhanced, enabling the network to better complete the leaf point cloud completion task. For the leaf sheet structure, an edge loss function is designed to guide the model to learn leaf edge information, thereby improving the accuracy of plant leaf morphology research, providing more reliable data support for research in fields such as ecology and plant biology, and providing strong support for the realization of intelligent agriculture.

[0149] The same or similar labels correspond to the same or similar parts;

[0150] The terms used to describe positional relationships in the accompanying drawings are for illustrative purposes only and should not be construed as limiting this patent.

[0151] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A leaf point cloud completion method based on multi-layer attention fusion and edge structure guidance, characterized in that, Includes the following steps: S1: Obtain the 3D point cloud dataset of plant leaves and perform preprocessing; S2: Establish a complete point cloud network for the leaf blades; The leaf point cloud completion network includes an encoder and a decoder connected in sequence; The encoder includes: an upsampling module, a downsampling module, and a multi-layer attention fusion module connected in sequence; the multi-layer attention fusion module of the encoder includes: a first attention sub-module, a second attention sub-module, and a convolutional fusion sub-module; Each attention submodule includes, in sequence, several OffSet-Transformer blocks with identical structures, and a convolutional block; the input of each OffSet-Transformer block is also connected to the input of the convolutional block. In each OffSet-Transformer block, the input data is processed through three parallel first multilayer perceptrons (MLPs) to obtain data 1, data 2, and data 3. Data 2 is then matrix-transposed and multiplied with data 1 to obtain data 4. Data 4 is processed by a softmax operation and recorded as data 5. The second dimension of data 5 is summed to obtain data 6. Data 5 and data 6 are then divided by a matrix to obtain data 7. Data 3 and data 7 are then multiplied by a matrix to obtain data 8. The input data and data 8 are then subtracted by a matrix to obtain data 9. Data 9 is then processed through a second multilayer perceptron (MLP) and the first batch of normalized batch normalization (BN) layers to obtain data 10. The input data and data 10 are then matrix-added to obtain the output data of the OffSet-Transformer block. The input data, data 1 to 10, and output data are all three-dimensional vectors. The convolutional block comprises, in sequence: a third multilayer perceptron (MLP), a second batch normalized (BN) layer, and a ReLU activation function layer; The convolutional fusion submodule includes: a fourth multilayer perceptron (MLP); S3: Input the preprocessed 3D point cloud dataset of plant leaves into the established leaf point cloud completion network, and use the preset total loss function to guide the leaf point cloud completion network to perform iterative training to obtain the trained leaf point cloud completion network. The preset total loss function includes: the edge loss function and the overall loss function; S4: Obtain the point cloud of the plant leaf to be completed, input the point cloud of the plant leaf to be completed into the trained leaf point cloud completion network for completion, and complete the plant leaf point cloud completion.

2. The leaf point cloud completion method based on multi-layer attention fusion and edge structure guidance according to claim 1, characterized in that, The preprocessing in step S1 includes: The plant leaf 3D point cloud dataset is filtered to remove the point clouds of withered and yellow leaves and miscellaneous point sets other than leaves, and the filtered plant leaf 3D point cloud dataset is obtained. The plant leaf 3D point cloud dataset after data filtering is subjected to mean filtering and smoothing and point cloud coordinate normalization in sequence to obtain the preprocessed plant leaf 3D point cloud dataset, thus completing the preprocessing.

3. The leaf point cloud completion method based on multi-layer attention fusion and edge structure guidance according to claim 1, characterized in that, In step S2, the encoder downsampling module includes: a first local feature extraction submodule SA and a second local feature extraction submodule SA with the same structure and connected in sequence; The output of the upsampling module is connected to the input of the first local feature extraction submodule SA, and the outputs of the first local feature extraction submodule SA and the second local feature extraction submodule SA are respectively connected to the input of the multi-layer attention fusion module.

4. The leaf point cloud completion method based on multi-layer attention fusion and edge structure guidance according to claim 3, characterized in that, Each of the local feature extraction submodules (SA) includes a sampling layer, a grouping layer, and a feature encoding layer. The sampling layer uses the farthest point sampling algorithm (FPS) to sample the input point cloud and obtain a set of sampled points. The grouping layer uses the K-nearest neighbor algorithm to cluster the sampled point set and obtain a set of point sets with several local centers. The feature encoding layer uses a multilayer perceptron (MLP) to encode the features of all the point sets with local centers, obtain the local features of the point cloud, and use the local features of the point cloud as the output of the local feature extraction submodule (SA).

5. A leaf point cloud completion method based on multi-layer attention fusion and edge structure guidance according to claim 3 or 4, characterized in that, In step S2, the first attention submodule and the second attention submodule have the same structure and are arranged side by side. The outputs of the first attention submodule and the second attention submodule are respectively connected to the input of the convolution fusion submodule. The input of the first attention submodule is connected to the output of the first local feature extraction submodule SA, and the input of the second attention submodule is connected to the output of the second local feature extraction submodule SA.

6. The leaf point cloud completion method based on multi-layer attention fusion and edge structure guidance according to claim 1, characterized in that, Each attention submodule contains four OffSet-Transformer blocks.

7. The leaf point cloud completion method based on multi-layer attention fusion and edge structure guidance according to claim 1, characterized in that, In step S3, the edge loss function Specifically: in, The edge truth value of the leaf point cloud; To complete the edge prediction values ​​of the point cloud of the leaf; The bulldozing distance is used to complete the predicted edge value and the true edge value of the leaf point cloud. The overall loss function Specifically: in, The truth value of the leaf point cloud; To complete the predicted value of the point cloud of the rear leaf; The bulldozing distance is used to complete the predicted value and the true value of the leaf point cloud. The preset total loss function Specifically: in, This is a hyperparameter.

8. A leaf point cloud completion system based on multi-layer attention fusion and edge structure guidance, employing the leaf point cloud completion method based on multi-layer attention fusion and edge structure guidance as described in any one of claims 1 to 7, characterized in that, include: Data acquisition and preprocessing unit: used to acquire and preprocess the 3D point cloud dataset of plant leaves; Model building unit: used to build the leaf point cloud completion network; The leaf point cloud completion network includes an encoder and a decoder connected in sequence; The encoder includes: an upsampling module, a downsampling module, and a multi-layer attention fusion module connected in sequence; Model training unit: used to input the preprocessed 3D point cloud dataset of plant leaves into the established leaf point cloud completion network, and use a preset total loss function to guide the leaf point cloud completion network to perform iterative training to obtain the trained leaf point cloud completion network; The preset total loss function includes: the edge loss function and the overall loss function; Leaf point cloud completion unit: Used to acquire the plant leaf point cloud to be completed, input the plant leaf point cloud to be completed into the trained leaf point cloud completion network for completion, and complete the plant leaf point cloud.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 7.

Images