Methods for constructing the DEA_PFNet 3D point cloud completion model of shiitake mushroom and extracting phenotypic parameters

Abstract

This invention discloses a method for constructing a DEA_PFNet 3D point cloud completion model for shiitake mushrooms and extracting phenotypic parameters. It relates to the fields of computer vision and 3D point cloud processing technology, as well as the field of 3D point cloud data completion for plant phenotypic representation. It addresses the problems of existing methods in point cloud completion, such as high computational cost, insufficient local feature extraction, and low accuracy in completing irregular shiitake mushrooms. The model is based on an improved PF-Net, and the construction method includes the following steps: 1. Obtaining standardized shiitake mushroom point cloud datasets from multiple shiitake mushrooms; 2. For the standardized shiitake mushroom point cloud datasets, simulating real-world occlusion to obtain corresponding shiitake mushroom point cloud sample sets; 3. Constructing the DEA_PFNet model, which includes an improved multi-resolution encoder and a point pyramid decoder; the extraction method involves extracting phenotypic parameters based on the 3D point cloud data of shiitake mushrooms. This invention is applicable to fields such as 3D point cloud completion and processing, the application of computer vision and artificial intelligence in agriculture, and accurate measurement of fungal 3D phenotypic characteristics.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and 3D point cloud processing technology, and also to the field of 3D point cloud data completion technology for plant phenotypic data. Background Technology

[0002] Shiitake mushrooms are among the most produced and consumed edible fungi globally. High-throughput, precise acquisition of their phenotypic traits has become an indispensable prerequisite for optimizing breeding strategies and advancing genetic analysis. Currently, obtaining phenotypic parameters of shiitake mushrooms relies on manual measurement and two-dimensional image analysis. Manual measurement suffers from low efficiency, large subjective errors, and significant destructiveness, while two-dimensional image technology is insufficient to comprehensively describe the three-dimensional morphology and spatial structure of shiitake mushrooms. Three-dimensional reconstruction technology based on multi-view images and the SfM-MVS algorithm can acquire three-dimensional point clouds of shiitake mushrooms at low cost and high precision, providing a new approach for phenotypic shape analysis.

[0003] However, in real-world data collection scenarios, due to factors such as mutual occlusion between mushrooms and limited viewing angles of the equipment, the reconstructed point cloud often has local gaps, affecting the accuracy of subsequent segmentation and parameter measurement. Traditional point cloud completion methods are mostly based on geometric symmetry or regular growth models, which have limited applicability to mushrooms with complex shapes and irregular structures. In recent years, deep learning-based point cloud completion methods, such as PF-Net, PCN, and FoldingNet, have shown stronger feature learning and generation capabilities, but they still generally suffer from problems such as high computational cost, insufficient local feature extraction, and low accuracy in completing irregular mushrooms.

[0004] In summary, the current technical challenges in completing 3D point clouds of shiitake mushrooms are that existing completion methods, especially deep learning models, struggle to achieve a good balance between completion accuracy and computational efficiency in real-world data acquisition scenarios due to factors such as mutual occlusion between shiitake mushroom entities, limited device viewing angles, or complex and irregular shapes and structures. Summary of the Invention

[0005] This invention solves the problems of high computational cost, insufficient local feature extraction, and low accuracy in completing irregular mushroom shapes in existing point cloud completion methods.

[0006] The method for constructing the DEA_PFNet 3D point cloud completion model of shiitake mushrooms includes the following steps:

[0007] Step 1: Obtain a standardized point cloud dataset of multiple shiitake mushrooms;

[0008] Step 2: For the standardized mushroom point cloud dataset, simulate real occlusion, remove the part far from the viewpoint according to the preset missing rate, generate incomplete point cloud-corresponding missing ground truth pair data, obtain the corresponding mushroom point cloud sample set, and randomly divide the mushroom point cloud sample set into training set, validation set and test set.

[0009] Step 3: Construct the DEA_PFNet model, which includes an improved multi-resolution encoder and a dot pyramid decoder;

[0010] The improved multi-resolution encoder uses a dual-pooling multilayer perceptron (Dual_MLP) module to perform multi-level feature extraction and fusion from coarse to fine on the input mushroom point cloud sample set, and outputs a comprehensive latent feature vector.

[0011] The improved point pyramid decoder integrates EdgeConv and a multi-head self-attention mechanism to receive the latent feature vector. The EdgeConv is responsible for enhancing the capture of local geometric features, and the multi-head self-attention mechanism is responsible for modeling global shape dependencies. Finally, the corresponding mushroom point cloud completion data is obtained.

[0012] The model uses a joint loss function that combines completion loss and adversarial loss. It is trained and optimized using training, validation and test sets. The optimized DEA_PFNet model serves as the 3D point cloud completion model for mushrooms.

[0013] To further optimize the solution, step 1 involves obtaining the standardized shiitake mushroom point cloud dataset for each shiitake mushroom, which includes the following steps:

[0014] Step 11: Acquire multi-view RGB images of a single shiitake mushroom and use U... 2 The -Net model performs foreground segmentation on the RGB image to obtain an RGB image after removing the background;

[0015] Step 12: Perform 3D reconstruction on the RGB image after background removal based on the SfM-MVS algorithm to obtain the original mushroom point cloud dataset;

[0016] Step 13: Preprocess the original mushroom point cloud dataset. The preprocessing includes scale recovery, coordinate correction and noise removal to obtain a standardized mushroom point cloud dataset.

[0017] In a further optimized scheme, in step 11, the method for acquiring the RGB image of a single shiitake mushroom is as follows: the shiitake mushroom is fixed on a marker, and the marker is placed at the center of a turntable; during data acquisition, the turntable rotates at a constant speed, and during one rotation, a camera device is used to continuously acquire images of the shiitake mushroom at a fixed point to obtain the RGB image of a single shiitake mushroom.

[0018] In a further preferred embodiment, in step 12, the three-dimensional reconstruction process is as follows: using the SIFT algorithm to extract feature points from the RGB image and generate feature descriptors, generating sparse three-dimensional point cloud data based on the feature descriptors using the SfM algorithm, and generating dense three-dimensional point cloud data using the MVS algorithm.

[0019] In a further optimized approach, in step 13, the scale restoration is based on the marker, and... As a scaling factor, scale recovery is performed on the original mushroom point cloud dataset;

[0020] The coordinate correction is based on the random sampling consistency plane fitting algorithm to calculate the normal vector of the marker. With the Z-axis normal vector Determine the axis of rotation and rotation angle The rotation matrix is ​​generated using the Rodrigues rotation formula. Rotation matrix Align the marker plane to the horizontal plane to complete the coordinate correction of the original mushroom point cloud dataset;

[0021] The noise removal process employs pass-through filtering and color filtering to remove surrounding noise, resulting in a standardized mushroom point cloud dataset.

[0022] Further optimization of the scheme: In step 2, the method for obtaining the paired data is as follows: First, create a unit spherical bounding box for each standardized mushroom point cloud data, and set multiple viewpoints on the surface of the sphere to simulate different positions in the real scene; then, eliminate points far from the viewpoints with different missing rates, and use the Iterative Farthest Point Sampling Method (IFPS) to sample and obtain incomplete point cloud data. From the eliminated viewpoints, use the Iterative Farthest Point Sampling Method (IFPS) to sample and obtain the corresponding missing ground truth data.

[0023] In a further optimized scheme, in step 3, the dual-pooling multilayer perceptron (Dual_MLP) module employs GhostConv convolution and a dual-pooling dynamic fusion method.

[0024] In a further optimized scheme, in step 3, the completion loss is calculated based on the chamfer distance; the adversarial loss is achieved by a discriminator that distinguishes the mushroom point cloud data completed by the DEA_PFNet mushroom 3D point cloud completion model from the corresponding missing ground truth values.

[0025] A method for extracting phenotypic parameters based on 3D point cloud data of shiitake mushrooms is described. The method extracts phenotypic parameters from the completed shiitake mushroom point cloud data. The phenotypic parameters of shiitake mushrooms include the horizontal diameter of the cap, the vertical diameter of the cap, the cap thickness, the height of the stipe, and the diameter of the stipe. A region growing algorithm is used to segment the 3D point cloud data of shiitake mushrooms into cap point cloud data and stipe point cloud data for phenotypic parameter extraction.

[0026] A further optimized approach is to extract the phenotypic parameters of shiitake mushrooms as follows:

[0027] The point cloud data of the cap is rotated and aligned using the PCA algorithm and projected onto the XOY plane. The Euclidean distance between the two farthest points is taken as the horizontal diameter of the cap; the longest diameter perpendicular to the horizontal diameter is taken as the vertical diameter of the cap; and the absolute difference between the maximum and minimum values ​​in the Z-axis direction is taken as the cap thickness.

[0028] The stipe point cloud data is rotated and aligned using the PCA algorithm. The absolute difference between the maximum and minimum values ​​of the stipe point cloud data in the X-axis direction is taken as the stipe height. Slices with a thickness of 10% of the total stipe length are cut at 25%, 50%, and 75% of the relative height. The point cloud data of the slices are projected onto the YOZ plane, and a circle is fitted using the least squares method to calculate the diameter. The mean of the three results is taken as the stipe diameter.

[0029] The beneficial effects of this invention compared to the prior art are as follows:

[0030] This study proposes a DEA_PFNet method for constructing a 3D point cloud completion model for shiitake mushrooms. The DEA_PFNet model is based on an improved PF-Net. In the encoder, the original CMLP module is replaced with an improved dual-pooling multilayer perceptron (Dual_MLP) module. The Dual_MLP module uses GhostConv convolution instead of ordinary convolution, effectively reducing computational cost, improving data processing speed, and lowering hardware requirements. Furthermore, to improve completion accuracy while reducing computational cost, this invention uses a dual-pooling dynamic fusion method instead of a single max-pooling method, improving the model's accuracy in completing details. EdgeConv convolution and a multi-head self-attention mechanism are added to the decoder to enhance the model's ability to extract local geometric features and global structural features, thereby improving the overall completion accuracy.

[0031] The DEA_PFNet model constructed using the proposed 3D point cloud completion method for shiitake mushrooms in this invention demonstrates, based on experimental comparisons, that its performance significantly outperforms existing similar technologies, including classic completion models such as FoldingNet, PCN, TopNet, SnowflakeNet, and PointTr. Furthermore, compared to PF-Net:

[0032] The overall point cloud completion method reduced the chamfer distance (CD) by 9.62%, the Earth movement distance (EMD) by 3.65%, improved the F-score by 4.32%, and reduced the number of floating-point operations (FLOPs) by 44.39%. For missing point cloud completion, the chamfer distance (CD) was reduced by 11.96% and the Earth movement distance (EMD) by 4.40%. These experimental comparisons verify that the DEA_PFNet model constructed in the method for constructing the mushroom 3D point cloud completion model based on the improved PF-Net described in this invention significantly reduces the computational load and significantly improves the completion accuracy compared with existing similar technologies.

[0033] The DEA_PFNet model constructed by the proposed method for building a 3D point cloud completion model of shiitake mushrooms in this invention maintains a high correlation between the phenotypic measurement results after completing the real missing shiitake mushroom point cloud and the manually measured values, with a coefficient of determination R0. 2 All values ​​remained above 0.9, verifying that point cloud completion technology has both high reliability and practical potential in the phenotypic measurement of shiitake mushrooms.

[0034] The DEA_PFNet model constructed by the proposed method for building a 3D point cloud completion model of shiitake mushroom has good versatility and can be applied to point cloud completion tasks of other fungi such as king oyster mushroom, Ganoderma lucidum, and oyster mushroom.

[0035] The method for constructing the DEA_PFNet three-dimensional point cloud completion model of shiitake mushroom and the method for extracting phenotypic parameters described in this invention are applicable to fields such as three-dimensional point cloud completion and processing, the application of computer vision and artificial intelligence in agriculture, and the accurate measurement of three-dimensional phenotypic characteristics of fungi. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the Dual_MLP module structure of the present invention;

[0037] Figure 2 This is a schematic diagram of the GhostConv module of the present invention;

[0038] Figure 3 This is a schematic diagram of the dual-pooling dynamic fusion method of the present invention;

[0039] Figure 4 The flowchart of EdgeConv computation of the present invention is shown in (a) a schematic diagram of edge feature computation, and (b) a schematic diagram of EdgeConv visualization.

[0040] Figure 5 This is a structural diagram of the multi-head attention mechanism of the present invention;

[0041] Figure 6 This is a detailed structural diagram of the dot pyramid decoder of the present invention;

[0042] Figure 7 Phenotypic calculations for the present invention include (a) shiitake mushroom dot cloud, (b) cap OBB bounding box, (c) stipe OBB bounding box, (d) cap transverse and longitudinal diameters, (e) cap thickness, (f) stipe height, and (g) stipe diameter.

[0043] Figure 8 The image acquisition diagram for Embodiment Twelve of the present invention is shown in (a) a schematic diagram of three fixed positions of the camera device during image acquisition, and (b) an RGB image acquired during acquisition.

[0044] Figure 9 In response to Figure 8 The RGB image shown in Figure (b) is the RGB image after removing the background;

[0045] Figure 10 This is a schematic diagram of the generation of an incomplete point cloud dataset according to Embodiment Twelve of the present invention. (a) 14 viewpoints are evenly set around the sphere of the mushroom point cloud. (b) Points that are far from the viewpoints are removed as missing values.

[0046] Figure 11 The visualization results of point cloud completion for shiitake mushrooms using different models in Embodiment Twelve of the present invention;

[0047] Figure 12 The image shows the mushroom point cloud completion results of PF-Net and DEA_PFNet under different missing rates in Embodiment Twelve of the present invention. The brown area represents the original input, the pink area represents the actual missing part, and the blue area represents the missing part predicted by the model.

[0048] Figure 13 As described in Embodiment Twelve of the present invention, after completing the point cloud data using the DEA_PFNet model described in the present invention based on the data in the test set, the simulated missing shiitake mushroom phenotypic parameters obtained are: (a) cap transverse diameter, (b) cap longitudinal diameter, (c) cap thickness, (d) stipe height, and (e) stipe diameter.

[0049] Figure 14 As described in Embodiment Twelve of the present invention, the shiitake mushroom point cloud that is truly missing under natural conditions is used as input and output. After the point cloud is completed using the DEA_PFNet model described in the present invention, the shiitake mushroom phenotypic parameter measurement results are obtained as follows: (a) cap transverse diameter, (b) cap longitudinal diameter, (c) cap thickness, (d) stipe height, (e) stipe diameter.

[0050] Figure 15 This is the result of completing the real missing mushroom point cloud based on DEA_PFNet according to Embodiment Twelve of the present invention. The blue part is the model-predicted point cloud. Detailed Implementation

[0051] Various embodiments of the present invention will now be clearly and completely described with reference to the accompanying drawings. The embodiments described with reference to the drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0052] Implementation Method 1: This implementation method provides a method for constructing a DEA_PFNet 3D point cloud completion model of a shiitake mushroom, including the following steps:

[0053] Step 1: Obtain a standardized point cloud dataset of multiple shiitake mushrooms;

[0054] Step 2: For the standardized mushroom point cloud dataset, simulate real occlusion, remove the part far from the viewpoint according to the preset missing rate, generate incomplete point cloud-corresponding missing ground truth pair data, obtain the corresponding mushroom point cloud sample set, and randomly divide the mushroom point cloud sample set into training set, validation set and test set.

[0055] Step 3: Construct the DEA_PFNet model, which includes an improved multi-resolution encoder and a dot pyramid decoder;

[0056] The improved multi-resolution encoder uses a dual-pooling multilayer perceptron (Dual_MLP) module to perform multi-level feature extraction and fusion from coarse to fine on the input mushroom point cloud sample set, and outputs a comprehensive latent feature vector.

[0057] The improved point pyramid decoder integrates EdgeConv and a multi-head self-attention mechanism to receive the latent feature vector. The EdgeConv is responsible for enhancing the capture of local geometric features, and the multi-head self-attention mechanism is responsible for modeling global shape dependencies. Finally, the corresponding mushroom point cloud completion data is obtained.

[0058] The model uses a joint loss function that combines completion loss and adversarial loss. It is trained and optimized using training, validation and test sets. The optimized DEA_PFNet model serves as the 3D point cloud completion model for mushrooms.

[0059] The DEA_PFNet shiitake mushroom 3D point cloud completion model described in this embodiment improves the completion accuracy of incomplete point clouds and reduces the computational load by combining the Dual_MLP module of dual pooling multilayer perceptron, EdgeConv edge convolution and self-attention mechanism, providing an effective solution for accurately obtaining shiitake mushroom phenotypic information.

[0060] Implementation Method 2: This implementation method further defines Implementation Method 1 and provides an example of the method for obtaining the standardized shiitake mushroom point cloud dataset for each shiitake mushroom in step 1.

[0061] The method for obtaining the standardized shiitake mushroom point cloud dataset for each shiitake mushroom includes the following steps:

[0062] Step 11: Acquire multi-view RGB images of a single shiitake mushroom and use U... 2 The -Net model performs foreground segmentation on the RGB image to obtain an RGB image after removing the background;

[0063] Step 12: Perform 3D reconstruction on the RGB image after background removal based on the SfM-MVS algorithm to obtain the original mushroom point cloud dataset;

[0064] Step 13: Preprocess the original mushroom point cloud dataset. The preprocessing includes scale recovery, coordinate correction and noise removal to obtain a standardized mushroom point cloud dataset.

[0065] Implementation Method 3: This implementation method is a further limitation of Implementation Method 2, and provides an example of step 11, which involves collecting an RGB image of a single shiitake mushroom.

[0066] The method for acquiring an RGB image of a single shiitake mushroom is as follows: the shiitake mushroom is fixed on a marker, and the marker is placed at the center of a turntable; during data acquisition, the turntable rotates at a constant speed, and during one rotation, a camera device is used to continuously acquire images of the shiitake mushroom at fixed points to obtain an RGB image of a single shiitake mushroom.

[0067] Implementation Method 4: This implementation method is a further limitation of Implementation Method 2, and provides an example of the three-dimensional reconstruction in step 12.

[0068] The 3D reconstruction process is as follows: the SIFT algorithm is used to extract feature points from the RGB image and generate feature descriptors; based on the feature descriptors, the SfM algorithm is used to generate sparse 3D point cloud data; and the MVS algorithm is used to generate dense 3D point cloud data.

[0069] Implementation Method 5: This implementation method is a further limitation of Implementation Method 2, and provides an example of step 13, namely scale restoration, coordinate correction and noise removal.

[0070] The scale restoration is based on the markers, and... As a scaling factor, the original mushroom point cloud dataset is scaled back using the following formula:

[0071] ,

[0072] ,

[0073] in, , and These represent the actual length, width, and height of the marker, respectively. , and These represent the length, width, and height of the mushroom point cloud after 3D reconstruction, respectively, and the original coordinates. According to the scaling factor Calculate the new coordinates ;

[0074] The coordinate correction is based on the random sampling consistency plane fitting algorithm to calculate the normal vector of the marker. With the Z-axis normal vector Determine the axis of rotation and rotation angle The rotation matrix is ​​generated using the Rodrigues rotation formula. Rotation matrix Align the marker plane to the horizontal plane to complete the coordinate correction of the original mushroom point cloud dataset. The formula is as follows:

[0075] ,

[0076] ,

[0077] ,

[0078] in It is the identity matrix. yes The antisymmetric matrix;

[0079] The noise removal process employs pass-through filtering and color filtering to remove surrounding noise, resulting in a standardized mushroom point cloud dataset.

[0080] Implementation Method Six: This implementation method is a further limitation of Implementation Method One, and provides an example of the pairing data in step 2.

[0081] The method for obtaining the paired data is as follows: First, a unit spherical bounding box is created for each standardized mushroom point cloud data. Multiple viewpoints are set on the surface of the sphere to simulate different positions in the real scene. Then, points far from the viewpoints are eliminated with three different missing rates of 25%, 50%, and 75%. The Iterative Farthest Point Sampling (IFPS) method is used for sampling to obtain incomplete point cloud data. The corresponding missing ground truth data is obtained by sampling from the eliminated viewpoints using the Iterative Farthest Point Sampling (IFPS) method.

[0082] Implementation Method Seven: This implementation method is a further limitation of Implementation Method One, and provides an example of the Dual_MLP module in step 3.

[0083] The Dual_MLP module employs GhostConv convolution and a dual-pooling dynamic fusion method. The structure of the Dual_MLP module is as follows: Figure 1 As shown, the specific workflow is as follows:

[0084] The Dual_MLP module of the dual-pooling multilayer perceptron encodes the input point cloud dataset to be completed into a multidimensional form through multiple convolutions, and extracts multidimensional feature vectors from the outputs of the last four layers using a dual-pooling dynamic fusion method. , and all the aforementioned feature vectors The connections form a potential vector F.

[0085] The GhostConv convolution generates more feature maps with less computation. GhostConv uses a small number of regular convolution kernels to generate essential features, performs inexpensive operations on these features to generate new phantom features, further reducing computation, and then concatenates the two sets of features into a new feature map. The GhostConv structure is as follows: Figure 2 As shown.

[0086] The dual-pooling dynamic fusion method uses weight parameters The two pooling results are weighted and combined. It can adaptively adjust the pooling ratio according to data characteristics and task objectives to achieve dynamic feature fusion. The dual-pooling dynamic fusion structure is as follows: Figure 3 As shown, the formula is as follows:

[0087] ,

[0088] ,

[0089] in, and Representing feature maps respectively The output after max pooling and average pooling It is the sigmoid function. Map to the (0,1) interval and update the weight parameters. .

[0090] The Dual_MLP module of this embodiment uses GhostConv to replace the ordinary convolution in the PF-Net model, reducing the computational cost of the model; and replaces the single max pooling method in the PF-Net model with a dual-pooling dynamic fusion method, enhancing the model's ability to extract local and global features.

[0091] Implementation Method 8: This implementation method is a further limitation of Implementation Method 1, and provides an example of step 3, namely EdgeConv and the multi-head self-attention mechanism.

[0092] The EdgeConv convolution calculates the relationship between each center point and its local neighborhood graph by constructing a local neighborhood graph. The geometric relationships between the nearest neighbors are analyzed using a nonlinear transformation function to extract edge features, and a local aggregation function to update the point features. For each point in the point cloud... EdgeConv uses the KNN algorithm to select edge convolutions based on Euclidean distance. The nearest neighbor points form a local neighborhood; the center point and its neighboring points are calculated. The edge characteristics are given by the following formula:

[0093] ,

[0094] in This represents a learnable nonlinear transformation function, implemented using an MLP that integrates nonlinear activation functions. (Edge features) Perform aggregation to obtain updated point features. The formula is as follows:

[0095] ,

[0096] in, It is a point The domain set. The EdgeConv computation process is as follows: Figure 4 As shown.

[0097] The multi-head attention mechanism captures the dependencies between distant points in the global structure of the point cloud. The multi-head self-attention mechanism obtains the query matrix through three sets of linear transformations. Key matrix Sum matrix Use a weight matrix for each head , , Mapping to the corresponding subspace, performing scaling dot product attention operations on each subspace, and using matrix multiplication for computation. and The dot product is calculated using the softmax function, and then weighted and fused. The outputs of all subspaces are concatenated and subjected to a linear transformation to form a fused representation of the multi-head self-attention mechanism. The formula for calculating multi-head attention is as follows:

[0098] ,

[0099] ,

[0100] ,

[0101] in, This represents the output of each subspace. The number of subspaces For linear transformation, , and This is the weight matrix. For the dimensions of query and key. The structure of the multi-head self-attention mechanism is as follows: Figure 5 As shown.

[0102] The dot pyramid decoder takes the final feature vector V output by the multi-resolution encoder and passes it through three fully connected layers to output three feature layers respectively. ( Different feature layers predict point clouds at different resolutions. High feature layers Predict the rough center point set Size is ; Middle feature layer Predict the set of secondary centroids Through expansion and addition operations, Each point in the middle is used as the center to generate indivual point, Size is Low feature layer Predict the final dense point set Size is .right , EdgeConv is used to extract local geometric information, which is then passed to the next layer through residual connections. During the generation phase, a multi-head self-attention mechanism is introduced to effectively capture the dependencies between distant points.

[0103] The detailed structure of the point pyramid decoder is as follows: Figure 6 As shown.

[0104] Implementation Method Nine: This implementation method is a further limitation of Implementation Method One, and provides an example of step 3, namely, the completion loss and the countermeasure loss.

[0105] The completion loss is calculated based on the chamfer distance; the adversarial loss uses a discriminator to distinguish between the mushroom point cloud data completed by the DEA_PFNet mushroom 3D point cloud completion model and the corresponding missing ground truth values.

[0106] The formula for calculating the chamfer distance is as follows:

[0107] ,

[0108] ,

[0109] ,

[0110] in, Indicates the calculation of predicted point clouds Each point in the cloud is converted to a real point cloud. The average Euclidean distance of the nearest point in the middle. This indicates the calculation of the true point cloud. Each point in the predicted point cloud The average Euclidean distance to the nearest point in the middle.

[0111] The decoder in the model predicts point clouds at three different resolutions, and the completion loss function is derived from... , , It consists of three parts:

[0112] ,

[0113] When epoch < 30, =0.01, 30<=epoch<80, =0.05, epoch>80 =0.1.

[0114] The formula for the adversarial loss function is as follows:

[0115] ,

[0116] in, , S yes , The size of the dataset, and Let and represent the input missing point cloud and the actual missing point cloud, respectively. This indicates that the model maps a portion of the input to the predicted missing regions. Discriminator D and To differentiate, the total loss function formula is as follows:

[0117] .

[0118] The hardware environment for model training is as follows: CPU is Intel(R) Xeon(R) Gold 6246R, GPU is NVIDIA Quadro RTX 8000 GPU with 48GB memory; software environment is Ubuntu 22.04 operating system, deep learning environment is Python 3.8, PyTorch 1.10.0, CUDA 11.3.

[0119] The hyperparameters for model training are configured as follows: epochs = 200, batch size = 24, optimizer = Adam, initial learning rate = 0.0001, weight decay = 0.00001, and a dual-pooling multilayer perceptron (Dual_MLP) is used. The initial value is 0.5.

[0120] Implementation Method 10: This implementation method provides a method for extracting phenotypic parameters based on 3D point cloud data of shiitake mushrooms. The method extracts phenotypic parameters from the completed 3D point cloud data of shiitake mushrooms. The phenotypic parameters of shiitake mushrooms include the cap's transverse diameter, cap's longitudinal diameter, cap thickness, stipe height, and stipe diameter. A region growing algorithm is used to segment the 3D point cloud data of shiitake mushrooms into cap point cloud data and stipe point cloud data for phenotypic parameter extraction.

[0121] Implementation Method Eleven: This implementation method further defines Implementation Method Ten and provides an example of the extraction of shiitake mushroom phenotypic parameters. See [link to implementation method]. Figure 7 This implementation method is described below.

[0122] The phenotypic parameters are extracted as follows:

[0123] right Figure 7 The cap point cloud in (a) is rotated and aligned using the PCA algorithm and projected onto the XOY plane (see [reference]). Figure 7 (b)). The Euclidean distance between the two farthest points is taken as the transverse diameter of the cap (see (b)). Figure 7 (The orange line segment in (d)). The longest diameter perpendicular to the transverse diameter is taken as the longitudinal diameter of the cap (see...). Figure 7 (The blue line segment in (d)). The maximum value in the Z-axis direction (see...) Figure 7 (in the blue dot in (e)) and the minimum value (see Figure 7 The absolute difference between the green dots in (e) is taken as the cap thickness;

[0124] right Figure 7The stipe point cloud in (a) was rotated and aligned using the PCA algorithm (see [reference]). Figure 7 (c) in the figure, the maximum value of the stipe point cloud in the X-axis direction (see [reference]). Figure 7 (in the blue dot in (f)) and the minimum value (see Figure 7 The absolute difference between the green dots in (f) is taken as the stipe height. Sections with a thickness of 10% of the total stipe length are cut at 25%, 50%, and 75% of the relative height. The point cloud of the sections is projected onto the YOZ plane (see...). Figure 7 In (g), the least squares method was used to fit the circle and calculate the diameter. The mean of the three results was taken as the diameter of the stipe.

[0125] Implementation Method Twelve: This implementation method illustrates the beneficial effects of the present invention through specific experimental data.

[0126] In this embodiment, the 509 strain of shiitake mushroom from the Shanghai Academy of Agricultural Sciences was selected as the research object. The shiitake mushroom logs were placed in a growth room with an environment of 20-25℃ and a humidity of 85% and cultivated until the shiitake mushrooms matured.

[0127] Shiitake mushrooms were fixed to a marker measuring 20cm × 10cm × 5cm, which was placed at the center of a turntable with a diameter of 30cm. During data acquisition, the turntable rotated at a constant speed of 33 revolutions per second. Images were acquired using a mobile phone with a resolution of 3024×4032. The phone was mounted on a tripod, with the lens maintained between 20 and 40cm from the shiitake mushroom sample. The image was captured by adjusting the tripod height. The mushroom was filmed at three angles: 45°, 0°, and +45° as it rotated one full turn. The camera's (phone's) fixed position during the filming process is shown in [reference needed]. Figure 8 As shown in (a) above, an RGB image was acquired as follows: Figure 8 As shown in (b), a total of 240 RGB images were collected for a single shiitake mushroom. U... 2 The -Net segmentation model performs foreground segmentation on the acquired RGB image to obtain the RGB image after removing the background, so as to accurately extract the key regions of the shiitake mushroom.

[0128] The background-removed RGB image (e.g.) is processed based on the SfM-MVS algorithm. Figure 9 (As shown) 3D reconstruction is performed to obtain the original mushroom point cloud dataset. The original mushroom point cloud dataset is then preprocessed with scale recovery, coordinate correction and noise removal to obtain a standardized mushroom point cloud dataset.

[0129] Based on the above steps, 220 standardized shiitake mushroom point cloud datasets were obtained and randomly divided into training, validation, and test sets in an 8:1:1 ratio. Phenotypic parameters of shiitake mushrooms were measured using calipers, including cap diameter, cap diameter, cap thickness, stipe height, and stipe diameter.

[0130] A unit spherical bounding box is created for each standardized mushroom point cloud dataset. Fourteen viewpoints are set on the surface of the sphere to simulate different locations in a real-world scene, such as... Figure 10 As shown in (a) of the diagram. To generate an incomplete point cloud, points far from the viewpoint were eliminated with three different missing rates of 25%, 50%, and 75%. The Iterative Farthest Point Sampling (IFPS) method was used to sample the point cloud, generating input data consisting of 2048 points. 512 points were sampled from the eliminated viewpoints using IFPS as the ground truth values ​​for calculating the loss function. Based on the above construction method, a total of 9240 samples were obtained, each consisting of an incomplete point cloud (2048 points) and a corresponding missing ground truth value (512 points), resulting in a point cloud dataset to be completed. A schematic diagram of the point cloud dataset generation is shown below. Figure 10 As shown in (b) of the diagram.

[0131] The point cloud dataset to be completed is input into the DEA_PFNet mushroom 3D point cloud completion model constructed in embodiments one to ten, after training and optimization, to obtain the complete mushroom point cloud data completed by the model.

[0132] To evaluate the completion performance of DEA_PFNet on the point cloud dataset, this study compares DEA_PFNet with classic completion models such as PF-Net, FoldingNet, PCN, TopNet, SnowflakeNet, and PointTr. The results of each model completing the mushroom point cloud dataset on the test set are shown in Table 1.

[0133] Table 1

[0134]

[0135] In Table 1, the F-score is used to evaluate the quality of complete point cloud completion, comprehensively measuring the coverage and matching accuracy of the generated point cloud to the real point cloud; FLOPs refers to the number of floating-point operations, which is a core indicator for measuring the computational complexity and computing power requirements of the computational model; Pred_GT / GT_Pred: Pred_GT represents the average Euclidean distance from each point in the predicted point cloud S1 to the nearest point in the real point cloud S2, and GT_Pred represents the average Euclidean distance from each point in the real point cloud S2 to the nearest point in the predicted point cloud S1. The sum of Pred_GT and GT_Pred equals the chamfer distance CD.

[0136] As shown in Table 1, in the test results of the complete mushroom point cloud data, among the classic completion models, PCN performed poorly, while PF-Net performed better than other models. Compared with PF-Net, DEA_PFNet reduced CD and EMD by 9.62% and 3.65% respectively, increased F-score by 4.32%, reduced FLOPs by 44.39%, and significantly reduced Pred_GT and GT_Pred, resulting in the best overall point cloud completion performance.

[0137] The error in completing the mushroom point cloud data consists of two parts: the prediction error of the missing region and the shape change error of the original part. DEA_PFNet only outputs the point cloud of the missing region without changing the shape of the original part. To make the experimental evaluation more reasonable, this implementation method tests the point cloud with the missing region completion. The test results of the point cloud completion of the missing region are shown in Table 2.

[0138] Table 2

[0139]

[0140] As shown in Table 2, in the missing point cloud completion test results, PCN has the largest completion error, while PointTr has the smallest completion error. Compared with PointTr, DEA_PFNet reduces CD and EMD by 11.84% and 4.19% respectively, and both Pred_GT and GT_Pred are reduced, indicating that it has the best point cloud completion effect in the missing region.

[0141] Using mushroom point cloud data with a missing rate of 25% from different perspectives as input, and the ground truth set as the complete mushroom point cloud, the visualization results of different models completing the mushroom point cloud are shown below. Figure 11 As shown.

[0142] Ablation experiments were conducted to further verify the impact of Dual_MLP (dual pooling multilayer perceptron), EdgeConv (edge ​​convolution), and multi-head self-attention mechanism on the performance of the DEA_PFNet model. The experimental results are shown in Table 3.

[0143] Table 3

[0144]

[0145] As shown in Table 3, after replacing CMLP with Dual_MLP (a dual-pooling multilayer perceptron), CD, EMD, and FLOPs decreased by 5.29%, 1.68%, and 45.31%, respectively, compared with PF-Net. After adding EdgeConv (edge ​​convolution), CD and EMD decreased by 3.63% and 1.23%, respectively, while FLOPs increased by 0.79%, compared with PF-Net. After adding a multi-head self-attention mechanism, CD and EMD decreased by 6.30% and 3.96%, respectively, while FLOPs increased by 0.05%, compared with PF-Net.

[0146] The Dual_MLP proposed in this invention uses GhostConv to replace ordinary convolution and dual pooling to replace max pooling. Further fine-grained ablation experiments are conducted to verify the specific contribution of the GhostConv and dual pooling dynamic fusion method to the model performance. The experimental results are shown in Table 4.

[0147] Table 4

[0148]

[0149] As shown in Table 4, in the Dual_MLP multilayer perceptron, replacing ordinary convolution with GhostConv significantly reduces FLOPs by 45.31% compared to PF-Net. Replacing max pooling with a dual-pooling dynamic fusion method reduces CD and EMD by 2.2% and 0.99% respectively compared to PF-Net. The DEA_PFNet model, which integrates all improvements, achieves a balance between completion accuracy and computational cost.

[0150] To further verify the performance of DEA_PFNet in completing mushroom point clouds with different missing values, this implementation set three different missing values ​​(25%, 50% and 75%) to verify the performance of DEA_PFNet in completing mushroom point clouds with different missing values, and compared it with the baseline model PF-Net. The experimental results are shown in Table 5.

[0151] Table 5

[0152]

[0153] As shown in Table 5, the completion performance of both models decreases with increasing missing rate. DEA_PFNet outperforms PF-Net at all missing rates, indicating that it still has good completion capabilities in severe missing scenarios.

[0154] Figure 12 This demonstrates the point cloud completion effects of two models on parts of shiitake mushrooms under different missing rates. Figure 12It can be seen that PF-Net has many holes and missing details, and the point cloud is dense and unevenly distributed in some areas. DEA_PFNet can better complete the overall outline and local details of the mushroom, and the completion effect is better.

[0155] This implementation uses 20 shiitake mushrooms from the test set as evaluation objects, with 42 incomplete samples for each mushroom. Point cloud completion is performed using DEA_PFNet. To obtain robust phenotypic measurement results, the mean of the phenotypic calculations from the 42 completed samples for each shiitake mushroom is taken as the measured value for that mushroom. The true values ​​are derived from the actual phenotypic parameters measured manually with calipers. At the individual shiitake mushroom level, the measured values ​​are compared with the true values, and the results are as follows: Figure 13 As shown. The phenotypic parameter measurements of the completed shiitake mushroom dot cloud showed high consistency with the manual measurements. The R values ​​for the cap transverse diameter, cap longitudinal diameter, and stipe height are shown. 2 The values ​​were relatively high, at 0.977, 0.972, and 0.961 respectively, for cap thickness and stipe diameter. 2 It is low, but still remains above 0.920.

[0156] To avoid optimism bias, this implementation method uses a hierarchical aggregation method to calculate MAE and RMSE. The absolute error of 42 completed samples of a single shiitake mushroom is calculated and averaged, which is the MAE of the shiitake mushroom. The mean of MAE (MAE_mean) and the mean of RMSE (RMSE_mean) of 20 shiitake mushrooms are taken. The results are shown in Table 6.

[0157] Table 6

[0158]

[0159] As shown in Table 6, the errors in the cap diameter, cap length, cap thickness, and stipe diameter are small, all not exceeding 2 mm. The stipe height is slightly larger, but still within 3 mm.

[0160] This implementation analyzes phenotypic measurements under different deletion rates, calculating R based on three deletion rates: 25%, 50%, and 75%. 2 The calculation results of RMSE_mean and MAE_mean are shown in Table 7.

[0161] Table 7

[0162]

[0163] As shown in Table 7, when the missing rate is 25%, the measured values ​​of the phenotypic parameters of the cap (transverse and vertical diameters), cap (longitudinal diameter), cap (thickness), stipe (height), and stipe (diameter) after completion have small errors compared with the manual measurements.

[0164] When the missing rate is 50%, the phenotypic parameters of the completed cap (transverse and vertical diameters), cap (longitudinal diameter), cap (thickness), stipe (height), and stipe (diameter) show increased errors compared to manual measurements.

[0165] When the missing rate is 75%, the measured values ​​of the phenotypic parameters of the cap (transverse and vertical diameters), cap (longitudinal diameter), cap (thickness), stipe (height), and stipe (diameter) after completion have a large error compared with the manual measurements.

[0166] This implementation method collects 20 real-world missing mushroom point clouds under natural conditions, and uses the proposed DEA_PFNet model for completion. The phenotypic measurements after completion are used as predicted values, and the manually measured results are used as true values. The completed phenotypic measurements are evaluated, and the calculation results are as follows: Figure 14 As shown. The cap diameter and longitudinal diameter showed a high correlation with manual measurements, while the cap thickness, stipe height, and stipe diameter showed a slightly lower correlation with manual measurements.

[0167] The DEA_PFNet model performs poorer phenotypic measurements on real-world missing mushroom point clouds compared to those on simulated missing mushroom point clouds in the test set. Mushroom morphology varies, features are highly variable, and the missing data is more complex and diverse. Furthermore, real-world missing mushroom point clouds contain more noise. Figure 15 This demonstrates the effect of dot cloud completion on partially missing shiitake mushrooms.

Claims

1. A method for constructing a DEA_PFNet 3D point cloud completion model of shiitake mushrooms, characterized in that, The construction method includes the following steps: Step 1: Obtain a standardized point cloud dataset of multiple shiitake mushrooms; Step 2: For the standardized mushroom point cloud dataset, simulate real occlusion, remove the part far from the viewpoint according to the preset missing rate, generate incomplete point cloud-corresponding missing ground truth pair data, obtain the corresponding mushroom point cloud sample set, and randomly divide the mushroom point cloud sample set into training set, validation set and test set. Step 3: Construct the DEA_PFNet model, which includes an improved multi-resolution encoder and a dot pyramid decoder; The improved multi-resolution encoder uses a dual-pooling multilayer perceptron (Dual_MLP) module to perform multi-level feature extraction and fusion from coarse to fine on the input mushroom point cloud sample set, and outputs a comprehensive latent feature vector. The improved point pyramid decoder integrates EdgeConv and a multi-head self-attention mechanism to receive the latent feature vector. The EdgeConv is responsible for enhancing the capture of local geometric features, and the multi-head self-attention mechanism is responsible for modeling global shape dependencies. Finally, the corresponding mushroom point cloud completion data is obtained. The model uses a joint loss function that combines completion loss and adversarial loss. It is trained and optimized using training, validation and test sets. The optimized DEA_PFNet model serves as the 3D point cloud completion model for mushrooms.

2. The method for constructing the DEA_PFNet three-dimensional point cloud completion model of shiitake mushrooms according to claim 1, characterized in that, Step 1, the method for obtaining the standardized shiitake mushroom point cloud dataset for each shiitake mushroom includes the following steps: Step 11: Acquire multi-view RGB images of a single shiitake mushroom and use U... 2 The -Net model performs foreground segmentation on the RGB image to obtain an RGB image after removing the background; Step 12: Perform 3D reconstruction on the RGB image after background removal based on the SfM-MVS algorithm to obtain the original mushroom point cloud dataset; Step 13: Preprocess the original mushroom point cloud dataset. The preprocessing includes scale recovery, coordinate correction and noise removal to obtain a standardized mushroom point cloud dataset.

3. The method for constructing the DEA_PFNet three-dimensional point cloud completion model of shiitake mushrooms according to claim 2, characterized in that, In step 11, the method for acquiring the RGB image of a single shiitake mushroom is as follows: the shiitake mushroom is fixed on a marker, and the marker is placed at the center of a turntable; during data acquisition, the turntable rotates at a constant speed, and during one rotation, a camera device is used to continuously acquire images of the shiitake mushroom at a fixed point to obtain the RGB image of a single shiitake mushroom.

4. The method for constructing the DEA_PFNet three-dimensional point cloud completion model of shiitake mushrooms according to claim 2, characterized in that, In step 12, the three-dimensional reconstruction process is as follows: using the SIFT algorithm to extract feature points from the RGB image and generate feature descriptors, using the SfM algorithm to generate sparse three-dimensional point cloud data based on the feature descriptors, and using the MVS algorithm to generate dense three-dimensional point cloud data.

5. The method for constructing the DEA_PFNet three-dimensional point cloud completion model of shiitake mushrooms according to claim 2, characterized in that, In step 13, the scale restoration is based on the markers, and... As a scaling factor, scale recovery is performed on the original mushroom point cloud dataset; The coordinate correction is based on the random sampling consistency plane fitting algorithm to calculate the normal vector of the marker. With the Z-axis normal vector Determine the axis of rotation and rotation angle The rotation matrix is ​​generated using the Rodrigues rotation formula. Rotation matrix Align the marker plane to the horizontal plane to complete the coordinate correction of the original mushroom point cloud dataset; The noise removal process employs pass-through filtering and color filtering to remove surrounding noise, resulting in a standardized mushroom point cloud dataset.

6. The method for constructing the DEA_PFNet three-dimensional point cloud completion model of shiitake mushrooms according to claim 1, characterized in that, In step 2, the method for obtaining the paired data is as follows: First, a unit spherical bounding box is created for each standardized mushroom point cloud data. Multiple viewpoints are set on the surface of the sphere to simulate different positions in the real scene. Then, points far from the viewpoints are eliminated with different missing rates. The iterative farthest point sampling method (IFPS) is used for sampling to obtain incomplete point cloud data. The corresponding missing ground truth data is obtained by sampling from the eliminated viewpoints using the iterative farthest point sampling method (IFPS).

7. The method for constructing the DEA_PFNet three-dimensional point cloud completion model of shiitake mushrooms according to claim 1, characterized in that, In step 3, the dual-pooling multilayer perceptron (Dual_MLP) module employs GhostConv convolution and a dual-pooling dynamic fusion method.

8. The method for constructing the DEA_PFNet three-dimensional point cloud completion model of shiitake mushrooms according to claim 1, characterized in that, In step 3, the completion loss is calculated based on the chamfer distance; the adversarial loss is achieved by a discriminator that distinguishes the mushroom point cloud data completed by the DEA_PFNet mushroom 3D point cloud completion model from the corresponding missing ground truth values.

9. A method for extracting phenotypic parameters based on 3D point cloud data of shiitake mushrooms, characterized in that, The method extracts phenotypic parameters from the three-dimensional point cloud completion data of shiitake mushroom as described in claim 1. The phenotypic parameters of shiitake mushroom include the horizontal diameter of the cap, the vertical diameter of the cap, the cap thickness, the height of the stipe, and the diameter of the stipe. The method uses a region growing algorithm to segment the three-dimensional point cloud data of shiitake mushroom into cap point cloud data and stipe point cloud data, and then extracts the phenotypic parameters of shiitake mushroom.

10. The method for extracting phenotypic parameters based on 3D point cloud data of shiitake mushrooms according to claim 9, characterized in that, The phenotypic parameters of shiitake mushrooms were extracted as follows: The point cloud data of the cap is rotated and aligned using the PCA algorithm and projected onto the XOY plane. The Euclidean distance between the two farthest points is taken as the horizontal diameter of the cap; the longest diameter perpendicular to the horizontal diameter is taken as the vertical diameter of the cap; and the absolute difference between the maximum and minimum values ​​in the Z-axis direction is taken as the cap thickness. The stipe point cloud data is rotated and aligned using the PCA algorithm. The absolute difference between the maximum and minimum values ​​of the stipe point cloud data in the X-axis direction is taken as the stipe height. Slices with a thickness of 10% of the total stipe length are cut at 25%, 50%, and 75% of the relative height. The point cloud data of the slices are projected onto the YOZ plane, and a circle is fitted using the least squares method to calculate the diameter. The mean of the three results is taken as the stipe diameter.

Images