A method for estimating shiitake mushroom yield based on non-destructive phenotypic parameter extraction based on three-dimensional reconstruction

Abstract

The method for estimating the yield of shiitake mushrooms based on lossless phenotypic parameter extraction based on three-dimensional reconstruction belongs to the field of agricultural information and intelligent technology, and solves the problem of low accuracy in estimating the yield of shiitake mushrooms. The method of the invention comprises: using the region of interest to perform three-dimensional reconstruction of the point cloud of shiitake mushrooms to generate dense point cloud data; preprocessing the dense point cloud data; segmenting the shiitake mushroom point cloud based on the improved CP-PointNet++ model and clustering algorithm, extracting multi-dimensional key phenotypic parameters of the cap diameter, thickness, and stipe height, and covering the main features of the phenotypic information of shiitake mushrooms; the improved PointNet++ network comprises: adding a CBAM module to the MLP layer of the PointNet++ network, and replacing ordinary convolution with partial convolution PConv; using the phenotypic parameters of shiitake mushrooms as input variables of the neural network model to predict the yield of shiitake mushrooms. The invention is suitable for phenotypic trait analysis and breeding research of edible fungi such as shiitake mushrooms.

Description

Technical Field

[0001] The present application relates to the field of agricultural informationization and intelligent technology, and in particular to three-dimensional reconstruction and phenotypic parameter extraction of shiitake mushrooms. Background Art

[0002] Mushrooms are a common fungus in nature and an important crop that is widely used for food and medicine around the world. They have attracted widespread attention for their unique flavor, rich nutritional value, and pharmacological effects such as anti-tumor and cholesterol-lowering. According to a report by Expert Market Research, the global mushroom market size in 2023 is approximately US$68.03 billion, and is expected to continue to grow at a compound annual growth rate of 7.18% between 2024 and 2032. Lentinula edodes, as a unique edible and medicinal crop in China, has become one of the most competitive agricultural products in the domestic and foreign markets. According to data from the Ministry of Agriculture and Rural Affairs of China, China's shiitake mushroom production reached 12.9548 million tons in 2022, accounting for 98.3% of the world's total production.

[0003] The phenotypic parameter characteristics of shiitake mushrooms are of great significance for the selection of high-quality strains, product grading and scientific research. However, the current phenotypic extraction technology has significant limitations, which seriously restricts the study of the relationship between genotype and phenotype of shiitake mushrooms, thereby affecting variety selection and production efficiency. Therefore, the development of a high-throughput, accurate and non-destructive phenotypic parameter extraction method has become an important task in the field of edible fungi research.

[0004] At present, the methods for obtaining phenotypic parameters mainly include manual measurement, two-dimensional image measurement and three-dimensional point cloud extraction:

[0005] 1. Manual measurement: slow, high repetitiveness, strong subjectivity, and easy to damage the sample during the measurement process.

[0006] 2. Two-dimensional image measurement: Only parameters related to the two-dimensional cap such as diameter can be obtained, and spatial properties such as thickness and direction cannot be extracted.

[0007] 3. 3D point cloud extraction: Based on laser measurement, depth camera and multi-view 3D reconstruction technology. Among them, laser measurement has high accuracy, but the equipment is expensive and the acquisition speed is slow; although the depth camera can quickly obtain 3D data, it is limited by resolution, the point cloud quality is low and easily affected by the environment. In contrast, multi-view 3D reconstruction technology has the advantages of low cost, high point cloud density, and is not easily affected by the environment. It is an important research direction for phenotype extraction.

[0008] Research based on multi-view 3D reconstruction has made progress in some crops. However, these studies are mostly aimed at field crops, and there are few studies on edible fungi such as shiitake mushrooms. At the same time, the following problems still exist:

[0009] 1. Background influence: During multi-view acquisition, the background area occupies a large amount of computing resources, reducing the efficiency of 3D reconstruction. Traditional segmentation algorithms (such as threshold segmentation and watershed algorithm) have poor robustness to complex scenes and cannot accurately extract target edges and background shadows.

[0010] 2. Insufficient automation of point cloud segmentation: Most existing point cloud segmentation methods require manual intervention and cannot meet the needs of high-throughput analysis.

[0011] 3. Incomplete extraction of phenotypic parameters: Some studies only extract single or a few phenotypic parameters, which cannot meet the needs of comprehensive analysis. Summary of the invention

[0012] The purpose of the present invention is to solve the problem of low accuracy in estimating mushroom yield caused by the above-mentioned problems, and to provide a method for estimating mushroom yield based on lossless phenotypic parameter extraction based on three-dimensional reconstruction.

[0013] The present invention is achieved through the following technical solutions. On the one hand, the present invention provides a method for estimating shiitake mushroom yield based on three-dimensional reconstruction lossless phenotypic parameter extraction, the method comprising:

[0014] Step 1: Obtain several images covering all angles of shiitake mushrooms; segment the images and classify them into regions of interest and irrelevant regions;

[0015] Using the region of interest, point cloud 3D reconstruction of mushrooms is performed to generate dense point cloud data;

[0016] Step 2: Preprocess the dense point cloud data in step 1, wherein the preprocessing includes downsampling, scale recovery, point cloud filtering and coordinate correction;

[0017] Step 3: Segment the dense point cloud data preprocessed in step 2, including:

[0018] Step 3.1: Use the improved PointNet++ network to segment the dense point cloud data preprocessed in step 2 to segment three types of point clouds: cap, stipe and stick. The improved PointNet++ network includes: adding a CBAM module to the MLP layer of the PointNet++ network, and using partial convolution PConv to replace ordinary convolution;

[0019] Step 3.2: Use the region growing algorithm to segment the cap individuals; use the fast Euclidean clustering algorithm to segment the stipe set;

[0020] Step 4: extracting phenotypic parameters of shiitake mushrooms, wherein the phenotypic parameters include: cap transverse diameter, cap longitudinal diameter, cap thickness, stipe height, stipe diameter and minimum bounding box volume;

[0021] Step 5: Use the phenotypic parameters in step 4 as input variables of the neural network model to predict the yield of shiitake mushrooms.

[0022] Furthermore, in step 1, obtaining a plurality of images covering all angles of shiitake mushrooms specifically includes:

[0023] The camera was used to perform multi-angle cross-surround shooting of the mushroom sticks to obtain multiple images covering all angles of the mushrooms; the camera aperture was f1.5, the exposure speed was 10ms, and the resolution was 3024*4032; the camera was fixed on a tripod, 30cm-50cm away from the mushroom sticks and the marker; the length, width and height of the marker were 30cm, 10cm and 5cm respectively; the camera was at the same height as the mushroom stick to take a picture in a straight line, and twice in a downward direction; when looking down, the camera was 25cm and 50cm higher than the mushroom stick, respectively, and the depression angles were 30° and 60°, respectively.

[0024] Furthermore, in step 1, the image is segmented and classified into regions of interest and irrelevant regions, specifically comprising: using YOLOv8 to first segment the collected digital image, and the image is classified into regions of interest and irrelevant regions, wherein plant pixels are assigned a value of 255 and marked as white.

[0025] Furthermore, in step 1, the three-dimensional reconstruction of the mushroom point cloud to generate dense point cloud data specifically includes:

[0026] Using the 3D reconstruction COLMAP software, SIFT algorithm is used to extract key points from the input region of interest image and generate feature descriptors for the key points;

[0027] Pair the key points of images from different perspectives to find the correspondence between images;

[0028] The Structure from Motion (SfM) algorithm is used to estimate the camera position and posture corresponding to each image through matching feature points, and a sparse 3D point cloud is generated on this basis to describe the preliminary geometry of the scene.

[0029] Based on the sparse 3D point cloud, a multi-view stereo matching algorithm is used to generate a dense 3D point cloud.

[0030] Further, step 2 includes:

[0031] Step 2.1: Perform voxel downsampling on the dense point cloud data in step 1;

[0032] Step 2.2: After calculating the scale based on the landmark, restore the point cloud coordinates to the actual size. The calculation formula is as follows:

[0033]

[0034] Among them, lreal 、w real 、h real are the actual length, width and height of the landmark, respectively. reco 、w reco 、h reco are the length, width and height of the landmark in the reconstructed 3D point cloud, (x, y, z) are the original coordinates, and (x`, y`, z`) are the converted coordinate values;

[0035] Step 2.3: Find the centroid of the point cloud, translate all points to move the centroid to the origin (0,0,0); use a statistical filter to remove outlier noise points in the reconstructed point cloud data;

[0036] Step 2.4: Calculate the whiteboard normal vector using the random sampling consensus algorithm ,Depend on and the z-axis normal vector Generate rotation axis and rotation angle;

[0037] The straight-through filtering is used to remove the markers and white board below the rotated point cloud to obtain the preprocessed mushroom stick point cloud.

[0038] Further, step 4 includes:

[0039] The PCA algorithm is applied to the cap to determine the main direction of the point cloud for rotation; the transformed cap is projected onto the XOY plane, and the Euclidean distance between the two farthest points is calculated as the horizontal diameter, and the longest diameter perpendicular to the horizontal diameter is the vertical diameter; the cap thickness is the absolute value of the difference between the maximum and minimum values ​​of the z-axis in the cap;

[0040] After rotating the stipe using the PCA algorithm, three slices were taken at 25%, 50%, and 75% positions in the main direction and with a parallel YOZ thickness of 10% of the stipe height. The least squares method was used to fit the points of the three slices to calculate the diameter, and the average of the three diameters was taken as the stipe diameter. The absolute value of the difference between the maximum and minimum values ​​on the x-axis of the stipe was taken as the stipe height.

[0041] Furthermore, the neural network model in step 5 is a generalized regression neural network model.

[0042] In a second aspect, the present invention provides a mushroom yield estimation system based on three-dimensional reconstruction lossless phenotypic parameter extraction, the system comprising:

[0043] The point cloud data generation module is used to obtain several images covering all angles of the mushrooms; segment the images and classify them into areas of interest and irrelevant areas;

[0044] Using the region of interest, point cloud 3D reconstruction of mushrooms is performed to generate dense point cloud data;

[0045] A point cloud data preprocessing module, used for preprocessing the dense point cloud data in the point cloud data generation module, wherein the preprocessing includes downsampling, scale recovery, point cloud filtering and coordinate correction;

[0046] The segmentation module is used to segment the dense point cloud data preprocessed in the point cloud data preprocessing module, specifically including:

[0047] The preprocessed dense point cloud data is segmented using an improved PointNet++ network to segment three types of point clouds: cap, stipe and stick. The improved PointNet++ network includes: adding a CBAM module to the MLP layer of the PointNet++ network, and replacing ordinary convolution with partial convolution PConv;

[0048] The region growing algorithm is used to segment the cap individuals; the fast Euclidean clustering algorithm is used to segment the stipe set;

[0049] A phenotypic parameter extraction module is used to extract phenotypic parameters of shiitake mushrooms, wherein the phenotypic parameters include: cap transverse diameter, cap longitudinal diameter, cap thickness, stipe height, stipe diameter and minimum bounding box volume;

[0050] The shiitake mushroom yield prediction module is used to use the phenotypic parameters in the phenotypic parameter extraction module as input variables of the neural network model to predict the shiitake mushroom yield.

[0051] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the steps of the method for estimating shiitake mushroom yield based on three-dimensional reconstruction and lossless phenotypic parameter extraction as described above are executed.

[0052] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein a plurality of computer instructions are stored in the computer-readable storage medium, and the plurality of computer instructions are used to enable a computer to execute a method for estimating shiitake mushroom yield based on three-dimensional reconstruction and lossless phenotypic parameter extraction as described above.

[0053] Beneficial effects of the present invention:

[0054] The method for extracting phenotypic parameters of shiitake mushrooms based on SfM-MVS three-dimensional reconstruction and improved CP-PointNet++ model, as well as the technical solution for yield estimation using GRNN, proposed in the present invention, have the following beneficial effects:

[0055] 1. High throughput and high efficiency: The present invention realizes automated point cloud segmentation and phenotypic parameter extraction, can quickly process a large number of shiitake mushroom samples, greatly improves the efficiency of phenotypic data acquisition, and is suitable for high-throughput agricultural research and production practice.

[0056] 2. High precision and comprehensiveness: Based on the improved CP-PointNet++ model and clustering algorithm, the point cloud of shiitake mushrooms is accurately segmented, avoiding the problems of background interference and misjudgment of complex shapes in traditional methods, and extracting multi-dimensional key phenotypic parameters such as cap diameter, thickness, and stipe height, covering the main characteristics of phenotypic information of shiitake mushrooms.

[0057] 3. Non-destructive and repeatable: The use of 3D reconstruction technology and point cloud analysis eliminates the need for destructive operations on mushroom samples, preserves the integrity of the samples, and improves the repeatability and stability of the measurement data.

[0058] 4. Low cost and universality: SfM-MVS technology relies on ordinary camera equipment to complete 3D reconstruction, which significantly reduces hardware costs and has strong universality and promotion.

[0059] The method of the present invention is highly robust and has strong adaptability to changes in environmental illumination, sample placement, etc., ensuring the stability and reliability of the three-dimensional reconstruction and phenotypic extraction process. The yield estimation is highly accurate. The GRNN model is used in combination with multiple phenotypic parameters to fully utilize the nonlinear associations between data, significantly improving the accuracy of shiitake mushroom yield estimation, and providing a scientific basis for shiitake mushroom cultivation management and yield optimization. The present invention can assist in mushroom breeding and management, providing a non-destructive, fast, and accurate method for acquiring shiitake mushroom phenotypic data and predicting yield, providing technical support for high-quality strain screening, precision agricultural management, product grading, and market pricing, etc., and is of great significance to promoting mushroom breeding and industrial development.

[0060] In summary, the technical solution of the present invention has significant advantages in efficiency, cost, accuracy and application scope, breaking through the bottleneck of the existing technology and having broad application value and development prospects. The present invention provides scientific support and technical tools for the growth characteristic research, genetic improvement and variety breeding of edible fungi through high-throughput, non-destructive and fast point cloud data processing and phenotype extraction technology.

[0061] The invention is suitable for phenotypic trait analysis and breeding research of edible fungi such as shiitake mushrooms. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0063] Figure 1 It is a schematic diagram of multi-view image acquisition method and an example of acquired images;

[0064] Figure 2Schematic diagram of the 3D reconstruction process;

[0065] Figure 3 is the preprocessed point cloud image;

[0066] Figure 4 Schematic diagram for phenotypic parameter calculation;

[0067] Figure 5 It is the CP-PointNet++ network structure;

[0068] Figure 6 It is a CBAM module structure;

[0069] Figure 7 It is a partial convolution module structure;

[0070] Figure 8 This is the segmentation result of the mushroom point cloud;

[0071] Fig. 9 Results are calculated for phenotypic parameters. DETAILED DESCRIPTION

[0072] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0073] Embodiment 1: A method for estimating shiitake mushroom yield based on three-dimensional reconstruction and lossless phenotypic parameter extraction, the method comprising:

[0074] Step 1: Obtain several images covering all angles of shiitake mushrooms; segment the images and classify them into regions of interest and irrelevant regions;

[0075] Using the region of interest, point cloud 3D reconstruction of mushrooms is performed to generate dense point cloud data;

[0076] Step 2: Preprocess the dense point cloud data in step 1, wherein the preprocessing includes downsampling, scale recovery, point cloud filtering and coordinate correction;

[0077] Step 3: Segment the dense point cloud data preprocessed in step 2, including:

[0078] Step 3.1: Use the improved PointNet++ network to segment the dense point cloud data preprocessed in step 2 to segment three types of point clouds: cap, stipe and stick. The improved PointNet++ network includes: adding a CBAM module to the MLP layer of the PointNet++ network, and using partial convolution PConv to replace ordinary convolution;

[0079] Step 3.2: Use the region growing algorithm to segment the cap individuals; use the fast Euclidean clustering algorithm to segment the stipe set;

[0080] Step 4: extracting phenotypic parameters of shiitake mushrooms, wherein the phenotypic parameters include: cap transverse diameter, cap longitudinal diameter, cap thickness, stipe height, stipe diameter and minimum bounding box volume;

[0081] Step 5: Use the phenotypic parameters in step 4 as input variables of the neural network model to predict the yield of shiitake mushrooms.

[0082] This implementation realizes automated point cloud segmentation and phenotypic parameter extraction, can quickly process a large number of shiitake mushroom samples, greatly improves the efficiency of phenotypic data acquisition, and is suitable for high-throughput smart agricultural research and production practice.

[0083] Based on the improved CP-PointNet++ model and clustering algorithm, the shiitake mushroom point cloud is accurately segmented, avoiding the problems of background interference and misjudgment of complex shapes in traditional methods, and extracting multi-dimensional key phenotypic parameters such as cap diameter, thickness, and stipe height, covering the main characteristics of shiitake mushroom phenotypic information.

[0084] The use of 3D reconstruction technology and point cloud analysis eliminates the need for destructive operations on shiitake mushroom samples, preserves the integrity of the samples, and improves the repeatability and stability of the measurement data.

[0085] Embodiment 2: This embodiment further limits the method for estimating shiitake mushroom yield based on three-dimensional reconstruction and lossless phenotypic parameter extraction as described above. In this embodiment, the step 1 of obtaining a plurality of images covering all angles of shiitake mushrooms is further limited, specifically including:

[0086] In step 1, the step of obtaining a plurality of images covering all angles of shiitake mushrooms specifically includes:

[0087] The camera was used to perform multi-angle cross-surround shooting of the mushroom sticks to obtain multiple images covering all angles of the mushrooms; the camera aperture was f1.5, the exposure speed was 10ms, and the resolution was 3024*4032; the camera was fixed on a tripod, 30cm-50cm away from the mushroom sticks and the marker; the length, width and height of the marker were 30cm, 10cm and 5cm respectively; the camera was at the same height as the mushroom stick to take a picture in a straight line, and twice in a downward direction; when looking down, the camera was 25cm and 50cm higher than the mushroom stick, respectively, and the depression angles were 30° and 60°, respectively.

[0088] Embodiment 3: This embodiment further limits the method for estimating shiitake mushroom yield based on lossless phenotypic parameter extraction based on three-dimensional reconstruction as described above. In this embodiment, the segmentation and classification of the image into regions of interest and irrelevant regions in step 1 are further limited, specifically including:

[0089] In step 1, the image is segmented and classified into regions of interest and irrelevant regions, specifically including: YOLOv8 is used to segment the collected digital image first, and the image is classified into regions of interest and irrelevant regions, wherein plant pixels are assigned a value of 255 and marked as white.

[0090] In this embodiment, during data acquisition, the collected multi-view RGB images contain a large amount of background information, which greatly interferes with the accuracy and efficiency of subsequent 3D reconstruction. Therefore, it is essential to remove the background information. As an end-to-end network architecture, YOLOv8 has demonstrated excellent performance in the field of semantic segmentation and is suitable for distinguishing subtle semantic differences in complex environments.

[0091] Embodiment 4: This embodiment further limits the method for estimating mushroom yield based on three-dimensional reconstruction and lossless phenotypic parameter extraction as described above. In this embodiment, in step 1, the three-dimensional reconstruction of the point cloud of the mushroom to generate dense point cloud data is further limited, specifically including:

[0092] In step 1, the three-dimensional reconstruction of the mushroom point cloud is performed to generate dense point cloud data, which specifically includes:

[0093] Using the 3D reconstruction COLMAP software, SIFT algorithm is used to extract key points from the input region of interest image and generate feature descriptors for the key points;

[0094] Pair the key points of images from different perspectives to find the correspondence between images;

[0095] The Structure from Motion (SfM) algorithm is used to estimate the camera position and posture corresponding to each image through matching feature points, and a sparse 3D point cloud is generated on this basis to describe the preliminary geometry of the scene.

[0096] Based on the sparse 3D point cloud, a multi-view stereo matching algorithm is used to generate a dense 3D point cloud.

[0097] In this embodiment, compared with laser scanning and depth camera equipment, the adopted SfM-MVS technology only relies on ordinary camera equipment to complete three-dimensional reconstruction, which significantly reduces the hardware cost, has a high tolerance for the acquisition environment, and has strong universality and scalability.

[0098] Embodiment 5: This embodiment further limits the method for estimating shiitake mushroom yield based on three-dimensional reconstruction and lossless phenotypic parameter extraction as described above. In this embodiment, step 2 is further limited, specifically including:

[0099] Step 2 includes:

[0100] Step 2.1: Perform voxel downsampling on the dense point cloud data in step 1;

[0101] Step 2.2: After calculating the scale based on the landmark, restore the point cloud coordinates to the actual size. The calculation formula is as follows:

[0102]

[0103] Among them, l real 、w real 、h real are the actual length, width and height of the landmark, respectively. reco 、w reco 、h reco are the length, width and height of the landmark in the reconstructed 3D point cloud, (x, y, z) are the original coordinates, and (x`, y`, z`) are the converted coordinate values;

[0104] Step 2.3: Find the centroid of the point cloud, translate all points to move the centroid to the origin (0,0,0); use a statistical filter to remove outlier noise points in the reconstructed point cloud data;

[0105] Step 2.4: Calculate the whiteboard normal vector using the random sampling consensus algorithm ,Depend on and the z-axis normal vector Generate rotation axis and rotation angle;

[0106] The straight-through filtering is used to remove the markers and white board below the rotated point cloud to obtain the preprocessed mushroom stick point cloud.

[0107] In this embodiment, because the amount of dense point cloud data reconstructed by the SfM-MVS method is too large, in order to save the time of subsequent algorithms processing data, a voxel downsampling method is used to streamline the point cloud data.

[0108] The SfM algorithm usually selects the camera shooting position coordinates of the first image as the coordinate origin of the point cloud. For multiple data, the coordinate system is chaotic and inconvenient for subsequent point cloud processing. Therefore, it is necessary to perform a point cloud centroid removal operation.

[0109] The three-dimensional point cloud data reconstructed from the multi-view images after YOLOv8 segmentation has a small number of outlier noise points. However, due to interference from hardware equipment, human operation, etc., the reconstructed point cloud still has some outlier noise points, so a statistical filter (SOR) is needed to remove them.

[0110] Embodiment 6: This embodiment further limits the method for estimating shiitake mushroom yield based on three-dimensional reconstruction and lossless phenotypic parameter extraction as described above. In this embodiment, step 4 is further limited, specifically including:

[0111] Step 4 includes:

[0112] The PCA algorithm is applied to the cap to determine the main direction of the point cloud for rotation; the transformed cap is projected onto the XOY plane, and the Euclidean distance between the two farthest points is calculated as the horizontal diameter, and the longest diameter perpendicular to the horizontal diameter is the vertical diameter; the cap thickness is the absolute value of the difference between the maximum and minimum values ​​of the z-axis in the cap;

[0113] After rotating the stipe using the PCA algorithm, three slices were taken at 25%, 50%, and 75% positions in the main direction and with a parallel YOZ thickness of 10% of the stipe height. The least squares method was used to fit the points of the three slices to calculate the diameter, and the average of the three diameters was taken as the stipe diameter. The absolute value of the difference between the maximum and minimum values ​​on the x-axis of the stipe was taken as the stipe height.

[0114] This embodiment provides an automated phenotypic parameter extraction method that covers the main features of Lentinus edodes phenotypic information, can quickly process a large number of Lentinus edodes samples, and greatly improves the efficiency of phenotypic data acquisition.

[0115] Embodiment 7: This embodiment further limits the method for estimating shiitake mushroom yield based on three-dimensional reconstruction and lossless phenotypic parameter extraction as described above. In this embodiment, the neural network model in step 5 is further limited, specifically including:

[0116] The neural network model in step 5 is a generalized regression neural network model.

[0117] This embodiment achieves high-precision estimation of shiitake mushroom yield through a generalized regression neural network (GRNN) model, uses the extracted phenotypic parameters as input variables, and combines historical sample data for model training and prediction.

[0118] This embodiment combines the characteristics of the multiple phenotypic parameters extracted in step 4 with a generalized regression neural network (GRNN) model to predict shiitake mushroom yield, which can: (1) High-precision nonlinear fitting: GRNN can capture the complex nonlinear relationship between phenotypic parameters and shiitake mushroom yield, and its prediction accuracy is better than that of traditional linear regression models.

[0119] (2) Adaptability to small sample training: Compared with deep neural networks, GRNN is more suitable for regression analysis under small sample conditions.

[0120] (3) Fast calculation: Both model training and prediction are highly efficient and meet the needs of practical applications.

[0121] In summary, this implementation method significantly improves the reliability and comprehensiveness of yield prediction by combining multiple key phenotypic parameters; and makes full use of the nonlinear correlation between data to significantly improve the accuracy of shiitake mushroom yield estimation, providing a scientific basis for shiitake mushroom cultivation management and yield optimization.

[0122] Embodiment 8: This embodiment is an example of a method for estimating mushroom yield based on three-dimensional reconstruction and lossless phenotypic parameter extraction as described above, and specifically includes:

[0123] Step 1: Multi-view image acquisition, image segmentation and 3D reconstruction

[0124] 1. Use a smartphone to perform multi-angle cross-surround photography of shiitake mushroom sticks to obtain multiple images covering all angles of the mushrooms. The camera aperture is f1.5, the exposure speed is 10 ms, and the resolution is 3024*4032. Fix the device on a tripod, 30 cm-50 cm away from the mushroom sticks and the marker. The length, width and height of the marker are 30 cm, 10 cm and 5 cm respectively. The camera is at the same height as the mushroom stick and takes one photo in a horizontal position, and two photos in a downward position. When looking down, the camera is 25 cm and 50 cm higher than the mushroom stick, respectively, and the depression angles are 30° and 60°, respectively. The image acquisition method is as follows: Figure 1 As shown in (a), there are about 150 images collected from multiple angles. The image data is as follows: Figure 1 (b) shown.

[0125] 2. Semantic Segmentation of Multi-View Images

[0126] In data acquisition, the collected multi-view RGB images contain a lot of background information, which greatly interferes with the accuracy and efficiency of subsequent 3D reconstruction. Therefore, it is essential to remove background information. As an end-to-end network architecture, YOLOv8 has demonstrated excellent performance in the field of semantic segmentation and is suitable for distinguishing subtle semantic differences in complex environments. Here, YOLOv8 is used to segment the collected digital images first, and the images are classified into regions of interest (ROI) and irrelevant regions. Among them, the plant pixels are assigned a value of 255 and marked as white.

[0127] 3. 3D reconstruction of mushroom point cloud

[0128] For 3D reconstruction, COLMAP (Ver.3.9.1, https: / / colmap.github.io / ) was selected. First, the SIFT algorithm was used to extract key points from the input segmented multi-view RGB image and generate feature descriptors for the key points. Then, the key points of the images under different viewpoints were paired to find the correspondence between the images. The Structure from Motion (SfM) algorithm was further used to estimate the camera position and posture corresponding to each image through the matched feature points. On this basis, a sparse 3D point cloud was generated to describe the preliminary geometry of the scene. Finally, based on the sparse point cloud, the Multi-View Stereo (MVS) algorithm was used to generate a dense 3D point cloud, such as Figure 2 As shown, a is the extraction and matching of mushroom feature points; b is SfM: motion structure; c is MVS: multi-view stereo.

[0129] Step 2: Point cloud preprocessing and segmentation

[0130] 1. Downsampling and scale restoration

[0131] The amount of dense point cloud data reconstructed by SfM-MVS is too large. In order to save the time of subsequent algorithm processing data, the voxel downsampling method is used to simplify the point cloud data. First, the point cloud space is gridded, also known as voxelization. Each grid after gridding is called a voxel. These extremely small grids contain some points, and then these points are averaged to get a point to replace all the points in the original grid. Setting the voxel size to 0.008 can remove 82% of the points on average, and the outer contour of the point cloud does not change, which does not affect the calculation of phenotypic parameters.

[0132] Because the point cloud information obtained by the SfM method does not have real size information, it is necessary to correct the coordinate ratio of the reconstructed 3D point cloud to obtain the real size of the mushroom stick. After calculating the ratio based on the marker, the point cloud coordinates are restored to the real size. The calculation formula is as follows:

[0133]

[0134] Among them, l real 、w real 、h real are the actual length, width and height of the landmark, respectively. reco 、w reco 、h reco are the length, width and height of the landmark in the reconstructed 3D point cloud, (x, y, z) are the original coordinates, and (x`, y`, z`) are the converted coordinate values.

[0135] 2. Point cloud filtering and coordinate correction

[0136] The SfM algorithm usually selects the camera shooting position coordinates of the first image as the coordinate origin of the point cloud. For multiple copies of data, the coordinate system is chaotic and inconvenient for subsequent processing of the point cloud. Therefore, it is necessary to perform a point cloud centroid removal operation. The centroid represents the geometric center position of the point cloud. The centroid of the point cloud is calculated, and all points are translated to move the centroid to the origin (0,0,0). There are a few outlier noise points in the three-dimensional point cloud data reconstructed from multi-view images after YOLOv8 segmentation. However, due to interference from hardware equipment, human operation, etc., there are still some outlier noise points in the reconstructed point cloud. A statistical filter (SOR) is used to remove them. SOR is a common filter used to remove outliers in point cloud processing. The algorithm determines whether the point is an outlier based on the local neighborhood statistical characteristics of each point. Among them, the parameters K=15, n=0.75, and there is no obvious change in the main body of the point cloud after denoising.

[0137] The random sampling consensus algorithm (RANSAC) is used to calculate the whiteboard normal vector .Depend on and the z-axis normal vector Generates the rotation axis and rotation angle.

[0138] The straight-through filter is used to remove the markers and white board below the rotated point cloud, and the preprocessed mushroom stick point cloud is obtained as shown below. Figure 3 shown.

[0139] 3. CP-PointNet++ network and clustering method to automate the segmentation process

[0140] The CP-PointNet++ model based on PointNet++ (Qi et al., 2017) is used for point cloud segmentation. PointNet++ introduces a hierarchical feature extraction method, which enhances the model's understanding of local structures by sampling and grouping point cloud data at different scales. Through layer-by-layer feature transformation and pooling, rich local feature representations can be extracted without losing global structural information. The PointNet++ segmentation network adopts a multi-level architecture, and the core structure is composed of set abstraction (SA module). In each SA module, there are three layers: Sampling Layer, Grouping Layer, and MLP Layer. The PointNet++ segmentation network model adopts an Encoder-Decoder structure, and the features are connected to the following modules through skip link concatenation. Feature Propagation consists of Interpolate and unit PointNet. Interpolate obtains the features that are ignored during the downsampling process of the SA module. Unit PointNet is composed of MLP + ReLU, which continues to extract features from the data.

[0141] Aiming at the problems of insufficient feature extraction capability, low segmentation accuracy and high memory usage of PointNet++ model, we improved it and proposed a novel CP-PointNet++ model. We added CBAM module to the MLP layer of the original PointNet++ network to improve the model's feature extraction capability. We used partial convolution (PConv) to replace ordinary convolution to reduce memory usage during model training.

[0142] CP-PointNet++ segmented three types of point clouds: cap, stipe and stick. Further segmentation of cap and stipe sets is required. Caps are usually arc-shaped or circular, with a smooth surface and obvious curvature characteristics. The normal directions of adjacent points show a relatively consistent change trend. The region growing algorithm is used to segment individual caps.

[0143] The stipes are spatially separable, and occlusion during data collection leads to different point cloud densities generated by 3D reconstruction. Fast Euclidean clustering is a clustering algorithm based on distance metric. It assumes that points of the same class are close in space. The algorithm is efficient and computationally simple. Based on the point cloud characteristics of the stipe set, the fast Euclidean clustering algorithm is used to segment the stipe set.

[0144] Step 3: Extraction of phenotypic parameters

[0145] For mushrooms, Figure 4 As shown, Figure 4 In the figure, (a) is the measurement principle of the cap transverse and longitudinal diameters, (b) is the measurement principle of the cap thickness, (c) is the measurement principle of the stipe diameter, (d) is the measurement principle of the stipe height, and (e) is the measurement principle of the bounding box. The phenotypic parameters of Lentinus edodes to be calculated are as follows: cap transverse diameter, cap longitudinal diameter, cap thickness, stipe height, stipe diameter and minimum bounding box (OBB) volume (the sum of the cap bounding box volume and the stipe bounding box volume).

[0146] The PCA algorithm is applied to the cap to determine the main direction of the point cloud for rotation. The transformed cap is projected onto the XOY plane, and the Euclidean distance between the two farthest points is calculated as the horizontal diameter, and the longest diameter perpendicular to the horizontal diameter is the vertical diameter; the cap thickness is the absolute value of the difference between the maximum and minimum values ​​of the z-axis in the cap.

[0147] After rotating the stipe using the PCA algorithm, three slices were taken at 25%, 50%, and 75% positions in the main direction and with a YOZ thickness of 10% of the stipe height. The least squares method was used to fit the points of the three slices to obtain the diameter, and the average of the three diameters was taken as the stipe diameter. The absolute value of the difference between the maximum and minimum values ​​of the x-axis of the stipe was taken as the stipe height.

[0148] Step 4: Production Estimation

[0149] The high-precision estimation of shiitake mushroom yield is achieved through the generalized regression neural network (GRNN) model, using the extracted phenotypic parameters as input variables and combining historical sample data for model training and prediction. The specific process is as follows:

[0150] The six phenotypic parameters extracted were used as model inputs: pileus transverse diameter, pileus longitudinal diameter, pileus height, stipe height, stipe diameter, and bounding box volume (OBB). The model output was shiitake yield.

[0151] Advantages of the GRNN model:

[0152] High-precision nonlinear fitting: GRNN can capture the complex nonlinear relationship between phenotypic parameters and shiitake mushroom yield, and its prediction accuracy is better than that of traditional linear regression models.

[0153] Adaptable to small sample training: Compared with deep neural networks, GRNN is more suitable for regression analysis under small sample conditions.

[0154] Fast calculation: Both model training and prediction are highly efficient and meet the needs of practical applications.

[0155] Multi-parameter synergy: By combining multiple key phenotypic parameters, the reliability and comprehensiveness of yield prediction are significantly improved.

[0156] The present invention aims to improve the PointNet++ model, which has the problems of insufficient feature extraction capability, low segmentation accuracy and high memory usage, and proposes a novel CP-PointNet++ model. Figure 5 As shown in the figure, in the mushroom point cloud, similar features between categories will interfere with the accuracy of model segmentation. Here, the CBAM module is added to the MLP layer to improve the model feature extraction capability. PConv is used to replace ordinary convolution to reduce the memory usage during model training. The CBMA module and PConv convolution structure are shown in Figure 1. Figure 6 and Figure 7 shown.

[0157] 1. Evaluation indicators:

[0158] (1) Point cloud segmentation evaluation indicators

[0159] The overall accuracy (OA) and mean IoU (mIoU) are used to evaluate the performance of the trained PointNet++ model in segmenting point clouds. The formula is as follows:

[0160]

[0161]

[0162] Among them, k represents the number of categories, TPi represents the true positive of the i-th category, FPi represents the false positive of the i-th category, and FNi represents the false negative of the i-th category.

[0163] (2) Phenotypic parameter calculation and yield estimation evaluation indicators

[0164] Mean absolute percentage error (MAPE), root mean square error (RMSE), normalized root mean square error (nRMSE) and coefficient of determination (R²) were selected as evaluation indicators. RMSE, nRMSE and R² were also used as evaluation indicators for the yield estimation model.

[0165]

[0166]

[0167]

[0168]

[0169] Where n is the number of samples, y i Measured value, =f(xi ) is the calculated value, is the mean of the measured values.

[0170] 2. Point Cloud Segmentation Model Effect

[0171] Table 1 compares the performance of the CP-PointNet++ proposed in this paper and the original PointNet++ (BASE) network. The original PointNet++ has a low segmentation accuracy (OA=89.86%, mIoU=75.14%), while adding the CBAM module to the model greatly improves the ability to distinguish the features of each category of the point cloud (OA=96.92%, mIoU=87.62%). PConv replaces the ordinary convolution in PointNet++ and significantly reduces the memory occupied during training (3.5G vs. 2.5G), and slightly improves the feature extraction ability of the model compared to the PointNet++ network (OA: 89.86% vs.94.56%, mIoU: 75.14% vs. 83.55%). The improved CP-PointNet++ has the best segmentation ability (OA=96.92%, mIoU=87.62%) and lower memory usage during training (Memory: 2.6G).

[0172] Table 1 Network performance

[0173]

[0174] The region growing algorithm and the fast Euclidean clustering algorithm are used to segment the cap and the stipe. The parameters of the region growing algorithm are: the number of domain search points is 400, the smoothing threshold is 20 degrees, and the curvature threshold is 0.045, which can segment the independent cap. The parameters of the fast Euclidean clustering algorithm are: the maximum number of points in the neighborhood search is 500, the search radius of the nearest neighbor search is 2, the minimum number of points required for clustering is 250, and the maximum number of points is 10,000, which can further segment the independent stipe. Figure 8 As shown, the segmentation results of some mushroom point clouds are shown.

[0175] 3. Effect of phenotypic parameter extraction

[0176] For the extraction of phenotypic parameters of Lentinula edodes, Fig. 9As shown, each picture is a comparison of the calculated values and measured values of the phenotypic parameters of Lentinula edodes: (a) transverse diameter of the pileus; (b) longitudinal diameter of the pileus; (c) thickness of the pileus; (d) diameter of the stipe; (e) height of the stipe. [Jiali 1] Based on the comparison of the calculated values of the phenotypic parameters of Lentinula edodes with the manually measured values from three-dimensional point clouds, each point in the image represents a Lentinula edodes. The abscissa is the manually measured value, and the ordinate is the calculated value by the algorithm. The closer the points in the image are to the diagonal line, the closer the calculated value is to the manually measured value. The calculated transverse diameter of the pileus (R2 = 0.92, RMSE = 3.30 mm, nRMSE = 0.07, MAPE = 6.79%), longitudinal diameter (R2 = 0.86, RMSE = 3.76 mm, nRMSE = 0.09, MAPE = 8.16%) and stipe diameter (R2 = 0.79, RMSE = 1.38 mm, nRMSE = 0.12, MAPE = 9.96%) have relatively small differences from the manually measured values. The calculated value of the pileus height (R2 = 0.53, RMSE = 3.26 mm, nRMSE = 0.17, MAPE = 16.33%) and the calculated value of the stipe height (R2 = 0.65, RMSE = 4.98 mm, nRMSE = 0.15, MAPE = 13.01%) have relatively large differences from the manually measured values.

Claims

1. A method for estimating shiitake mushroom yield based on lossless phenotypic parameter extraction using three-dimensional reconstruction, characterized in that: The method comprises: Step 1: Obtain several images covering all angles of shiitake mushrooms; segment the images and classify them into regions of interest and irrelevant regions; Using the region of interest, point cloud 3D reconstruction of mushrooms is performed to generate dense point cloud data; Step 2: Preprocess the dense point cloud data in step 1, wherein the preprocessing includes downsampling, scale recovery, point cloud filtering and coordinate correction; Step 3: Segment the dense point cloud data preprocessed in step 2, including: Step 3.1: Use the improved PointNet++ network to segment the dense point cloud data preprocessed in step 2 to segment three types of point clouds: cap, stipe and stick. The improved PointNet++ network includes: adding a CBAM module to the MLP layer of the PointNet++ network, and using partial convolution PConv to replace ordinary convolution; Step 3.2: Use the region growing algorithm to segment the cap individuals; use the fast Euclidean clustering algorithm to segment the stipe set; Step 4: extracting phenotypic parameters of shiitake mushrooms, wherein the phenotypic parameters include: cap transverse diameter, cap longitudinal diameter, cap thickness, stipe height, stipe diameter and minimum bounding box volume; Step 5: Use the phenotypic parameters in step 4 as input variables of the neural network model to predict the yield of shiitake mushrooms.

2. The method for estimating shiitake mushroom yield based on three-dimensional reconstruction and lossless phenotypic parameter extraction according to claim 1, characterized in that: In step 1, the step of obtaining a plurality of images covering all angles of shiitake mushrooms specifically includes: The camera was used to perform multi-angle cross-surround shooting of the mushroom sticks to obtain multiple images covering all angles of the mushrooms; the camera aperture was f1.5, the exposure speed was 10ms, and the resolution was 3024*4032; the camera was fixed on a tripod, 30cm-50cm away from the mushroom sticks and the marker; the length, width and height of the marker were 30cm, 10cm and 5cm respectively; the camera was at the same height as the mushroom stick to take a picture in a straight line, and twice in a downward direction; when looking down, the camera was 25cm and 50cm higher than the mushroom stick, respectively, and the depression angles were 30° and 60°, respectively.

3. The method for estimating shiitake mushroom yield based on three-dimensional reconstruction and lossless phenotypic parameter extraction according to claim 2, characterized in that: In step 1, the image is segmented and classified into regions of interest and irrelevant regions, specifically including: YOLOv8 is used to segment the collected digital image first, and the image is classified into regions of interest and irrelevant regions, wherein plant pixels are assigned a value of 255 and marked as white.

4. The method for estimating shiitake mushroom yield based on three-dimensional reconstruction and lossless phenotypic parameter extraction according to claim 3 is characterized in that: In step 1, the three-dimensional reconstruction of the mushroom point cloud is performed to generate dense point cloud data, which specifically includes: Using the 3D reconstruction COLMAP software, SIFT algorithm is used to extract key points from the input region of interest image and generate feature descriptors for the key points; Pair the key points of images from different perspectives to find the correspondence between images; The Structure from Motion (SfM) algorithm is used to estimate the camera position and posture corresponding to each image through matching feature points, and a sparse 3D point cloud is generated on this basis to describe the preliminary geometry of the scene. Based on the sparse 3D point cloud, a multi-view stereo matching algorithm is used to generate a dense 3D point cloud.

5. The method for estimating shiitake mushroom yield based on three-dimensional reconstruction and lossless phenotypic parameter extraction according to claim 1, characterized in that: Step 2 includes: Step 2.1: Perform voxel downsampling on the dense point cloud data in step 1; Step 2.2: After calculating the scale based on the landmark, restore the point cloud coordinates to the actual size. The calculation formula is as follows: Among them, l real 、w real 、h real are the actual length, width and height of the landmark, respectively. reco 、w reco 、h reco are the length, width and height of the landmark in the reconstructed 3D point cloud, (x, y, z) are the original coordinates, and (x`, y`, z`) are the converted coordinate values; Step 2.3: Find the centroid of the point cloud, translate all points to move the centroid to the origin (0,0,0); use a statistical filter to remove outlier noise points in the reconstructed point cloud data; Step 2.4: Calculate the whiteboard normal vector using the random sampling consensus algorithm ,Depend on and the z-axis normal vector Generate rotation axis and rotation angle; The straight-through filtering is used to remove the markers and white board below the rotated point cloud to obtain the preprocessed mushroom stick point cloud.

6. The method for estimating shiitake mushroom yield based on three-dimensional reconstruction and lossless phenotypic parameter extraction according to claim 1, characterized in that: Step 4 includes: The PCA algorithm is applied to the cap to determine the main direction of the point cloud for rotation; the transformed cap is projected onto the XOY plane, and the Euclidean distance between the two farthest points is calculated as the horizontal diameter, and the longest diameter perpendicular to the horizontal diameter is the vertical diameter; the cap thickness is the absolute value of the difference between the maximum and minimum values ​​of the z-axis in the cap; After rotating the stipe using the PCA algorithm, three slices were taken at 25%, 50%, and 75% positions in the main direction and with a parallel YOZ thickness of 10% of the stipe height. The least squares method was used to fit the points of the three slices to calculate the diameter, and the average of the three diameters was taken as the stipe diameter. The absolute value of the difference between the maximum and minimum values ​​on the x-axis of the stipe was taken as the stipe height.

7. The method for estimating shiitake mushroom yield based on three-dimensional reconstruction and lossless phenotypic parameter extraction according to claim 1, characterized in that: The neural network model in step 5 is a generalized regression neural network model.

8. A mushroom yield estimation system based on three-dimensional reconstruction and lossless phenotypic parameter extraction, characterized in that: The system comprises: The point cloud data generation module is used to obtain several images covering all angles of the mushrooms; segment the images and classify them into areas of interest and irrelevant areas; Using the region of interest, point cloud 3D reconstruction of mushrooms is performed to generate dense point cloud data; A point cloud data preprocessing module, used for preprocessing the dense point cloud data in the point cloud data generation module, wherein the preprocessing includes downsampling, scale recovery, point cloud filtering and coordinate correction; The segmentation module is used to segment the dense point cloud data preprocessed in the point cloud data preprocessing module, specifically including: The preprocessed dense point cloud data is segmented using an improved PointNet++ network to segment three types of point clouds: cap, stipe and stick. The improved PointNet++ network includes: adding a CBAM module to the MLP layer of the PointNet++ network, and replacing ordinary convolution with partial convolution PConv; The region growing algorithm is used to segment the cap individuals; the fast Euclidean clustering algorithm is used to segment the stipe set; A phenotypic parameter extraction module is used to extract phenotypic parameters of shiitake mushrooms, wherein the phenotypic parameters include: cap transverse diameter, cap longitudinal diameter, cap thickness, stipe height, stipe diameter and minimum bounding box volume; The shiitake mushroom yield prediction module is used to use the phenotypic parameters in the phenotypic parameter extraction module as input variables of the neural network model to predict the shiitake mushroom yield.

9. A computer device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor runs the computer program stored in the memory, the steps of the method according to any one of claims 1 to 7 are performed.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of computer instructions, and the plurality of computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.

Images