Lentinus edodes sporocarp point cloud completion method, device and system and storage medium
By constructing an augmented dataset and an improved Transformer model, the occlusion problem in point cloud completion of shiitake mushroom fruit entities was solved, achieving fine completion and local feature enhancement of shiitake mushroom fruit entity point clouds, which is applicable to the complex morphology of agricultural crops.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHEJIANG UNIV
- Filing Date
- 2025-12-30
- Publication Date
- 2026-05-01
AI Technical Summary
Existing technologies are insufficient to effectively complete the point cloud of shiitake mushroom fruiting bodies, especially due to incomplete 3D information acquisition caused by occlusion, which affects the extraction of phenotypic parameters. Furthermore, existing models are not effective in completing local features on agricultural crops.
We constructed an augmented dataset for point cloud completion of shiitake mushroom subentities using data augmentation methods, and designed the AEE-Conv module and a multi-scale shape-aware attention mechanism to improve the AdaPoinTr model. We then performed point cloud completion using a Transformer structure.
It achieves precise completion of point clouds of shiitake mushroom fruiting bodies, improves the accuracy of local feature understanding and overall structure, and adapts to the complex morphology of agricultural crops.
Smart Images
Figure CN121962433A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of information processing technology, specifically relating to a method, apparatus, system, and storage medium for completing point clouds of shiitake mushroom fruit entities. Background Technology
[0002] Shiitake mushrooms, as an important economic crop promoting sustainable agricultural development, possess advantages such as rich nutrition, high yield, high resource utilization, and significant economic benefits. However, currently, the acquisition of phenotypic information still relies mainly on manual measurement, a time-consuming and labor-intensive method that severely restricts the selection and cultivation of high-quality planting resources. The main reason lies in the prominent occlusion problem of shiitake fruiting bodies: on the one hand, fruiting bodies occlude with each other; on the other hand, larger caps hinder the acquisition of three-dimensional information of the cap interior and stipe, resulting in generally incomplete point clouds of fruiting bodies, making it impossible to effectively extract phenotypic parameters. Therefore, it is essential to complete the point cloud of the collected 3D model of shiitake fruiting bodies and develop phenotypic extraction algorithms; however, there is currently no research on point cloud completion for shiitake fruiting bodies.
[0003] 3D phenotypic data completion methods are mainly divided into two categories: traditional methods and deep learning methods. Traditional methods are further subdivided into geometry-based completion methods and matching-based completion methods. Geometry-based methods complete the point cloud based on the geometric shape information of the input data, and are only suitable for completing objects with simple and symmetrical structures, such as strawberries and plant leaves. Matching-based completion methods search for 3D models similar to the input data in a database, and replace or deform the input data to more closely resemble the input model to achieve point cloud completion. However, agricultural crops have a wide variety of varieties and significant morphological differences, making database construction extremely costly. If the input point cloud differs significantly from the database model, the accuracy of the completion result is difficult to guarantee, making it unsuitable for agricultural crops with complex and varied morphologies.
[0004] Learning-based methods mainly fall into two categories: voxelizing the point cloud before completion and directly completing the point cloud. Voxel-based methods typically require high memory and computational costs and may result in resolution loss. Currently, direct point cloud processing has become the mainstream method for point cloud completion. Most point cloud completion models employ an encoder-decoder structure, but due to their limited learning capabilities, they cannot completely complete the point cloud structure and are unsuitable for point clouds with significant incompleteness. PoinTr innovatively introduces a Transformer structure to predict missing parts of the point cloud, treating the point cloud completion problem as a set-to-set translation task. Through the Transformer's self-attention mechanism, this model can effectively integrate global contextual information, thereby better capturing the overall structural features of crop point clouds, significantly improving completion performance, and handling complex point clouds with significant incompleteness well, especially suitable for severely occluded mushroom fruiting body point clouds. AdaPoinTr, as an iteration of PoinTr, introduces an adaptive query mechanism and a dedicated denoising module, making the predicted point cloud more smooth and consistent in density distribution. However, since AdaPoinTr is primarily tested on datasets with a large number of low-frequency features (such as simplified sofas, chairs, etc.), its local feature completion performance is relatively poor when transferred to agricultural datasets. Summary of the Invention
[0005] To address the problems existing in the prior art, this invention provides a method, apparatus, system, and storage medium for completing point clouds of shiitake mushroom fruit entities. It proposes a data augmentation method for model training and designs an AEE-Conv module and a multi-scale shape-aware attention mechanism to improve the AdaPoinTr model, enabling it to optimize local completion details while maintaining the stability of the overall completion structure.
[0006] To achieve the above objectives, the present invention provides the following solution: A method for point cloud completion of shiitake mushroom fruiting bodies, comprising: Step S1: Construct a point cloud completion and enhancement dataset for mushroom subentities; Step S2: Based on the mushroom sub-entity point cloud completion and enhancement dataset, use the AEE-Conv module and the farthest point downsampling to realize the downsampling of the point cloud and the global feature extraction of the residual point cloud, obtain the center point position and local geometric features, and perform position encoding to obtain the point proxy of the residual point cloud. Step S3: Input the residual point cloud proxy sequence into the Transformer structure composed of multi-head self-attention and multi-scale fusion geometric perception attention proposed in this invention to predict the point proxy of the complete point cloud; Step S4: Based on the predicted complete point proxy, gradually generate the target number of point cloud outputs.
[0007] Preferably, step S1 includes: Use a high-precision 3D scanner to obtain complete point clouds of shiitake mushroom fruiting bodies; By randomly rotating, randomly occluding, and downsampling the farthest point of the mushroom sub-entity point cloud, a defective point cloud is generated, and a complete and enhanced dataset of mushroom sub-entity point cloud is constructed.
[0008] Preferably, in step S4, based on the predicted complete point proxy, a layered progressive strategy from coarse to fine is adopted to gradually generate the target number of point cloud outputs based on the FoldingNet architecture.
[0009] The present invention also provides a point cloud completion device for shiitake mushroom fruiting bodies, comprising: The first processing module is used to construct a point cloud completion and enhancement dataset for mushroom sub-entities. The second processing module is used to complete and enhance the dataset based on the point cloud of mushroom fruit entities. It uses the AEE-Conv module and the farthest point downsampling to realize the downsampling of the point cloud and the global feature extraction of the residual point cloud, to obtain the center point position and local geometric features, and performs position encoding to obtain the point proxy of the residual point cloud. The third processing module is used to input the incomplete point cloud point proxy sequence into the Transformer structure composed of multi-head self-attention and the multi-scale fusion geometric perception attention proposed in this invention, and to predict the point proxy of the complete point cloud. The fourth processing module is used to gradually generate the target number of point cloud outputs based on the predicted complete point proxies.
[0010] Preferably, the first processing module includes: The first processing unit is used to acquire complete point clouds of shiitake mushroom fruiting bodies using a high-precision 3D scanner; The second processing unit is used to generate a residual point cloud by randomly rotating, randomly occluding, and downsampling the farthest point of the mushroom sub-entity point cloud, and to construct a complete and enhanced dataset of the mushroom sub-entity point cloud.
[0011] As a preferred option, the fourth processing module generates the target number of point cloud outputs step by step based on the predicted complete point proxy and the FoldingNet architecture, using a coarse-to-fine layered progressive strategy.
[0012] The present invention also provides a point cloud completion system for shiitake mushroom fruit entities, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes a point cloud completion method for shiitake mushroom fruit entities when executed by the processor.
[0013] The present invention also provides a storage medium storing a computer program, which executes a method for completing the point cloud of mushroom sub-entities during runtime.
[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention proposes a model called MushroomNet, based on the AdaPoinTr model, for completing point cloud data of mushroom fruit entities in real-world scenes. First, this invention proposes a point cloud data augmentation method that simulates real-world incompleteness, solving the problems of difficulty in acquiring 3D point cloud data and the challenge of accurately registering real point clouds with incomplete point clouds. Furthermore, the designed AEE-Conv module and multi-scale fusion geometric perception attention mechanism enhance the model's ability to learn high-frequency feature points and improve its understanding of local shape features, thereby addressing the shortcomings of the base model AdaPoinTr in local detail completion and achieving fine completion of mushroom fruit entity point clouds. Attached Figure Description
[0015] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0016] Figure 1 This is a flowchart of the point cloud completion method for shiitake mushroom fruiting bodies according to an embodiment of the present invention; Figure 2 A schematic diagram illustrating the construction of a point cloud completion and augmentation dataset for mushroom fruit entities; Figure 3 A schematic diagram of the AEE-ConV module; Figure 4 A schematic diagram of a multi-scale shape-aware attention module; Figure 5 A point cloud completion diagram of shiitake mushroom fruiting bodies. Detailed Implementation
[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0018] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0019] Example 1 like Figure 1As shown, this invention provides a method for point cloud completion of mushroom fruit entities, applied to the completion of incomplete mushroom fruit entity point clouds, achieving good completion results. It employs MushroomNet, a model capable of finely completing incomplete mushroom fruit entity point clouds in real-world scenes. This model is based on the Transformer architecture, viewing the point cloud completion task as a process of predicting the global features of the complete point cloud from the global features of the incomplete point cloud. Specifically, a feature extraction module extracts the features of the incomplete point cloud as input, the Transformer structure predicts the features of the complete point cloud, and a refinement module optimizes these features, ultimately outputting a finely refined complete point cloud. The method includes: Step S1: Construction of the augmented dataset for mushroom fruit entity point cloud data: as follows Figure 2 As shown, the complete single-mushroom dataset obtained using a scanner is first randomly downsampled to 16438 points. Then, decentering is achieved by subtracting the average coordinates, followed by normalization, which serves as the ground truth. Next, to enhance dataset diversity, each processed sample is randomly rotated around the y-axis (angles uniformly distributed in the range [0, 360°]), generating 100 enhanced versions, which serve as the ground truth. Subsequently, occlusion is simulated on these complete point cloud samples: a point is randomly selected on the sample surface, and an occlusion radius is randomly chosen from the interval [0.425, 0.85]. All points within this spherical region are removed to generate a fragmented point cloud. This fragmented point cloud is then further randomly downsampled to 2048 points, achieving sparsity and fragmentation of the point cloud. Finally, all paired complete and fragmented point cloud samples are divided into training, validation, and test sets in an 8:1:1 ratio.
[0020] Step S2: Point cloud feature extraction: The AEE-Conv module proposed in this invention is used to extract global features from the point cloud, such as... Figure 3 As shown, this module introduces a point-level attention mechanism to adaptively adjust point weights after local features are extracted by EdgeConv. Specifically, firstly, global average pooling is performed on the local features output by EdgeConv, compressing the local representation of each point into a one-dimensional global description. Then, a multilayer perceptron (MLP) and sigmoid activation function are used to perform a non-linear mapping on the compressed features of each point to obtain the attention weights for each point. Finally, these weights are reassigned to the corresponding point features to model the importance of different points, thereby focusing on key points with high-frequency information. Global feature-specific extraction is achieved through a four-layer AEE-Conv combined with farthest point downsampling (FPS). Finally, the obtained sparse point cloud coordinates and global features are encoded using a multilayer perceptron.
[0021] Step S3: Complete Point Cloud Point Proxy Prediction: This invention designs a multi-scale fusion geometric perception attention module, MSFGAAttention, to replace the original geometric perception module in the Transformer structure of the basic model AdaPoinTr. The input of this module is the encoded global sparse point cloud representation and its extracted global features, such as... Figure 4 As shown. First, the number of nearest neighbors is set to K=5. The KNN algorithm is used to retrieve neighboring points, thereby obtaining the coordinates of the center point (3D), the coordinates of neighboring points (15D), the relative coordinates obtained by subtraction (15D), and the Euclidean distance from the neighboring points to the center point (5D). Furthermore, this invention calculates the covariance matrix of the center point's neighborhood using the following formula: , Let the coordinates of the nearest point be (K,3). Then, the eigenvalues of the covariance matrix are solved using `torch.linalg.eigvalsh` and sorted in descending order as λ1, λ2, λ3. The linearity, flatness, and scatter (3D) of the point cloud can be obtained through the following calculations: ; ; A total of 46 features were obtained, and their dimensionality was increased to match the global feature dimension using a multilayer perceptron (MLP). Next, with the nearest neighbor points set to K=10, the EdgeConv operation was performed using sparse coordinates and global features as input, and adjustments were made based on the MLP to ensure that the output feature dimension was consistent with the global feature dimension. The features obtained based on K=5 and K=10 were then added together to form new local features. A Transformer structure incorporating multi-scale fusion geometric awareness attention can predict incomplete point cloud local features as point proxies for complete point clouds.
[0022] Step S4: Generating a Refined and Complete Point Cloud: First, the reconstructed feature (rebuild_feature) is constructed by concatenating the global feature (1024 dimensions), the query feature (384 dimensions), and the coarse point proxy (3 dimensions), with a shape of (512, 1411). A linear layer (reduce_map) is used to reduce its dimensionality to (512, 384). Second, a fully connected decoder (decode_head) is used for further projection and dimensionality reduction to (512, 96), and then reshaped to relative coordinates (512, 32, 3). Finally, this relative coordinate is added to the coarse point proxy, and the first two dimensions are merged to generate a refined and complete point cloud (16384, 3). The final result of the point cloud completion is shown below. Figure 5 As shown.
[0023] The innovation of this invention lies in: 1. This invention provides a new data augmentation dataset for point cloud completion tasks. By simulating the incomplete point cloud generation mechanism in real-world scenarios (such as random occlusion, noise injection, and geometric transformation), it effectively solves the problems of high difficulty and cost in acquiring 3D point cloud data, as well as the difficulty in accurately matching the acquired incomplete data with the complete data.
[0024] 2. The AEE-Conv module proposed in this invention obtains the importance weight of each point through global average pooling and a multilayer perceptron, which enhances the focus on high-frequency feature points, thereby better understanding and capturing the local feature details of point clouds (such as edges, curvature, and texture). Compared with existing convolutional modules (such as standard PointNet or DGCNN), this module significantly improves the accuracy of feature extraction when dealing with complex geometric structures. At the same time, this module is simpler to operate, requiring no additional manual parameter tuning, and is easy to integrate into various point cloud networks.
[0025] 3. The multi-scale fusion geometric perception attention mechanism proposed in this invention, by integrating multi-level features and introducing geometric constraints, can more comprehensively understand the relationship between local and global features of point clouds, avoiding the problem of information loss at a single scale. Compared with existing shape perception attention mechanisms, this mechanism significantly improves the overall accuracy of the completion task.
[0026] Example 2 The present invention also provides a point cloud completion device for shiitake mushroom fruiting bodies, comprising: The first processing module is used to construct a point cloud completion and enhancement dataset for mushroom sub-entities. The second processing module is used to complete and enhance the dataset based on the point cloud of mushroom fruit entities. It uses the AEE-Conv module and the farthest point downsampling to realize the downsampling of the point cloud and the global feature extraction of the residual point cloud, to obtain the center point position and local geometric features, and performs position encoding to obtain the point proxy of the residual point cloud. The third processing module is used to input the incomplete point cloud point proxy sequence into the Transformer structure composed of multi-head self-attention and the multi-scale fusion geometric perception attention proposed in this invention, and to predict the point proxy of the complete point cloud. The fourth processing module is used to gradually generate the target number of point cloud outputs based on the predicted complete point proxies.
[0027] As one embodiment of the present invention, the first processing module includes: The first processing unit is used to acquire complete point clouds of shiitake mushroom fruiting bodies using a high-precision 3D scanner; The second processing unit is used to generate a residual point cloud by randomly rotating, randomly occluding, and downsampling the farthest point of the mushroom sub-entity point cloud, and to construct a complete and enhanced dataset of the mushroom sub-entity point cloud.
[0028] As one embodiment of the present invention, the fourth processing module generates the target number of point cloud outputs step by step based on the predicted complete point proxy and the FoldingNet architecture, using a coarse-to-fine layered progressive strategy.
[0029] Example 3 The present invention also provides a point cloud completion system for shiitake mushroom fruit entities, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes a point cloud completion method for shiitake mushroom fruit entities when executed by the processor.
[0030] Example 4 The present invention also provides a storage medium storing a computer program, which executes a method for completing the point cloud of mushroom sub-entities during runtime.
[0031] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.
Claims
1. A method for completing point clouds of shiitake mushroom fruiting bodies, characterized in that, include: Step S1: Construct a point cloud completion and enhancement dataset for mushroom subentities; Step S2: Based on the mushroom sub-entity point cloud completion and enhancement dataset, use the AEE-Conv module and the farthest point downsampling to realize the downsampling of the point cloud and the global feature extraction of the residual point cloud, obtain the center point position and local geometric features, and perform position encoding to obtain the point proxy of the residual point cloud. Step S3: Input the residual point cloud proxy sequence into the Transformer structure composed of multi-head self-attention and multi-scale fusion geometric perception attention proposed in this invention to predict the point proxy of the complete point cloud; Step S4: Based on the predicted complete point proxy, gradually generate the target number of point cloud outputs.
2. The method for completing the point cloud of shiitake mushroom fruiting bodies as described in claim 1, characterized in that, Step S1 includes: Use a high-precision 3D scanner to obtain complete point clouds of shiitake mushroom fruiting bodies; By randomly rotating, randomly occluding, and downsampling the farthest point of the mushroom sub-entity point cloud, a defective point cloud is generated, and a complete and enhanced dataset of mushroom sub-entity point cloud is constructed.
3. The method for completing the point cloud of shiitake mushroom fruiting bodies as described in claim 2, characterized in that, In step S4, based on the predicted complete point proxy, and using the FoldingNet architecture, a layered progressive strategy from coarse to fine is adopted to gradually generate the target number of point cloud outputs.
4. A point cloud completion device for shiitake mushroom fruiting bodies, characterized in that, include: The first processing module is used to construct a point cloud completion and enhancement dataset for mushroom sub-entities. The second processing module is used to complete and enhance the dataset based on the point cloud of mushroom fruit entities. It uses the AEE-Conv module and the farthest point downsampling to realize the downsampling of the point cloud and the global feature extraction of the residual point cloud, to obtain the center point position and local geometric features, and performs position encoding to obtain the point proxy of the residual point cloud. The third processing module is used to input the incomplete point cloud point proxy sequence into the Transformer structure composed of multi-head self-attention and the multi-scale fusion geometric perception attention proposed in this invention, and to predict the point proxy of the complete point cloud. The fourth processing module is used to gradually generate the target number of point cloud outputs based on the predicted complete point proxies.
5. The mushroom fruiting body point cloud completion device as described in claim 4, characterized in that, The first processing module includes: The first processing unit is used to acquire complete point clouds of shiitake mushroom fruiting bodies using a high-precision 3D scanner; The second processing unit is used to generate a residual point cloud by randomly rotating, randomly occluding, and downsampling the farthest point of the mushroom sub-entity point cloud, and to construct a complete and enhanced dataset of the mushroom sub-entity point cloud.
6. The mushroom fruiting body point cloud completion device as described in claim 5, characterized in that, The fourth processing module generates the target number of point cloud outputs step by step based on the predicted complete point proxy and the FoldingNet architecture, using a coarse-to-fine layered progressive strategy.
7. A point cloud completion system for shiitake mushroom fruiting bodies, characterized in that, include: A memory and a processor, wherein the memory stores a computer program executed by the processor, the computer program performing the mushroom fruit entity point cloud completion method as described in any one of claims 1-3 when executed by the processor.
8. A storage medium, characterized in that, The storage medium stores a computer program, which executes the mushroom fruit entity point cloud completion method as described in any one of claims 1-3 when the computer program is running.