Automatic agaricus bisporus phenotypic method based on machine vision

Abstract

The invention discloses an agaricus bisporus automatic phenotypic method based on machine vision. The method comprises the following steps: acquiring an original mushroom cap image of agaricus bisporus by using a depth camera in a vertical overlook manner, performing data enhancement and marking, and constructing a training set; constructing a target segmentation model containing a star network, and inputting the training set into the model for training; obtaining a mushroom cap image of the agaricus bisporus to be phenotyped, inputting the mushroom cap image into the agaricus bisporus segmentation model, and processing and outputting three target segmentation results; and obtaining a sporocarp image of each target segmentation result, obtaining the diameter, roundness and whiteness of a mushroom cap of the agaricus bisporus by using an agaricus bisporus phenotypic characteristic quantification method, and determining the color spot condition of the mushroom cap so as to realize automatic phenotypic of the agaricus bisporus. According to the method, high precision and low complexity balance can be realized during automatic detection of the agaricus bisporus phenotypic parameters, the phenotypic parameter acquisition requirement before picking can be met, progress of agaricus bisporus phenotypic research is promoted, and a more accurate and efficient tool is provided for edible mushroom production and quality evaluation.

Description

Technical Field

[0001] This invention relates to a mushroom phenotyping method, specifically to an automated phenotyping method for Agaricus bisporus based on machine vision. Background Technology

[0002] In the current industrialized production process of button mushrooms, the determination of the specific harvesting time still relies on the experience of harvesting workers, i.e., manual phenotypic observation. However, this method is highly subjective, lacks quantitative standards, and reduces product quality. Furthermore, manual phenotyping is too costly, severely reducing production efficiency, and its inability to accurately match the optimal harvesting time leads to quality loss and yield reduction. Therefore, developing a rapid and accurate method to identify the growth status of button mushrooms and obtain their key phenotypic parameters has become one of the core technologies for improving the yield and quality of industrialized button mushroom production. Traditional morphological image processing methods generally have a significant limitation: they are easily affected by changes in ambient light, overlapping mushroom fruiting bodies, and complex scenarios such as clustered mushrooms. This significantly reduces the robustness of these algorithms, resulting in poor applicability during promotion and difficulty in meeting actual production needs. Existing network-based algorithms for edible fungi segmentation mostly focus on improving segmentation accuracy, leading to large model weight files, excessive parameters, and high floating-point computation costs, neglecting the deployment costs in a mushroom industrialization environment. Meanwhile, research on the measurement of mushroom parameters after segmentation is relatively limited and warrants further in-depth investigation. Summary of the Invention

[0003] To address the problems existing in the background technology, this invention provides an automated phenotypic method for Agaricus bisporus based on machine vision. Addressing the practical needs of automated phenotypic identification in the industrialized growth of Agaricus bisporus, this invention proposes a lightweight instance segmentation model based on YOLOv11n-Seg. This model improves upon the original model's high computational cost, enhances cap feature recognition capabilities, and increases feature map reconstruction efficiency. While maintaining a lightweight architecture, it significantly improves detection accuracy and can effectively adapt to deployment scenarios with limited computing resources, such as embedded devices and edge computing terminals. Furthermore, this invention systematically constructs a quantification method for Agaricus bisporus phenotypic features, providing a theoretical basis for subsequent model optimization and parameter calibration.

[0004] The technical solution adopted in this invention is: The machine vision-based automated phenotypic method for Agaricus bisporus of this invention includes: Step 1) Use a depth camera to obtain several raw images of the caps of button mushrooms from a vertical top-down view, and then perform data augmentation and labeling to construct a training set.

[0005] Step 2) Construct a target segmentation model that includes StarNet, input the training set into the target segmentation model for training, and obtain the trained mushroom segmentation model.

[0006] Step 3) Use a depth camera to obtain a vertical top-down image of the cap of the mushroom to be phenotyped and input it into the mushroom segmentation model. After processing, the mushroom segmentation model outputs three target segmentation results.

[0007] Step 4) Obtain the sub-entity image of each target segmentation result, and use the button mushroom phenotypic feature quantification method to obtain the diameter, roundness and whiteness of the mushroom cap, and determine the color spot of the mushroom cap, so as to realize the automated phenotypicing of button mushroom.

[0008] In step 1), for each initial image of the mushroom cap and its data-enhanced image, the edges of the cap structure of each button mushroom in the image are marked, and labels of large, medium and small are marked according to the size of the cap structure.

[0009] In step 2), the StarNet network includes a first convolutional layer Conv connected in sequence and four Star units. Each Star unit includes a Star module StarBlock and a second convolutional layer Conv connected in sequence. Each Star module StarBlock includes a first depthwise convolutional layer DWConv, a batch normalization (BN) operation, and a first fully connected layer FC. The input to the StarBlock module is first processed by the first depthwise convolutional layer DWConv and the first batch normalization BN operation, and then input into the first and second fully connected layers FC respectively. The output of the second fully connected layer FC is processed by the activation function ReLU6 and then multiplied by the element-wise Star operation with the output of the first fully connected layer FC. The activation function ReLU6 can also be connected after the first fully connected layer FC. Then, it is processed by the third convolutional layer Conv, the second batch normalization BN operation, and the second depthwise convolutional layer DWConv before being output as the output of the StarBlock module.

[0010] In step 2), the target segmentation model includes StarNet, Spatial Pyramid Pooling Fast (SPPF), Cross-Channel Pyramid Squeeze Attention (C2PSA), Dynamic Upsampling (DySample), and Multi-scale Attention (C3K2-MAB). The target segmentation model consists of three modules: Block 1, the second dynamic upsampling DySample module, the second super-resolution feature extraction C3K2-MAB module 1, the fourth convolutional layer Conv, the third super-resolution feature extraction C3K2-MAB module 1, the fifth convolutional layer Conv, the super-resolution feature extraction C3K2-MAB module 2, the first segmentation head Segment, the second segmentation head Segment, and the third segmentation head Segment. The input of the target segmentation model is processed sequentially by StarNet, the Fast Spatial Pyramid Pooling (SPPF) module, the Cross-Stage Partial Fusion (C2PSA) module, and the first dynamic upsampling DySample module. The output of the first dynamic upsampling DySample module and the output of the third Star unit of StarNet are fused using a concat operation. Then, the output is processed sequentially by the first super-resolution feature extraction C3K2-MAB module 1 and the second dynamic upsampling DySample module. The second dynamic upsampling DySample module and the output of the third Star unit of StarNet are fused using a concat operation. The output of the second Star unit of Net is fused and concatted, and then processed sequentially by the second super-resolution feature extraction C3K2-MAB module and the first segmentation head Segment, and the output is used as the large target segmentation result. The output of the second super-resolution feature extraction C3K2-MAB module is processed by the fourth convolutional layer Conv and then fused and concatted with the output of the first super-resolution feature extraction C3K2-MAB module. Then, it is processed sequentially by the third super-resolution feature extraction C3K2-MAB module and the second segmentation head Segment, and the output is used as the medium target segmentation result. The output of the third super-resolution feature extraction C3K2-MAB module is processed by the fifth convolutional layer Conv and then fused and concatted with the output of the cross-stage partial fusion module C2PSA. Then, it is processed sequentially by the super-resolution feature extraction C3K2-MAB module II and the third segmentation head Segment, and the output is used as the small target segmentation result. Finally, three target segmentation results are obtained: large, medium and small.

[0011] Each of the super-resolution feature extraction C3K2-MAB modules includes a sixth convolutional layer (Conv), a first split operation, a feature extraction C3K-MAB unit, and a seventh convolutional layer (Conv) connected in sequence. The outputs of the sixth convolutional layer (Conv), the first split operation, and the feature extraction C3K-MAB unit are fused using a concat operation, and then input into the seventh convolutional layer (Conv) for processing. The output of the fused convolutional layer (Conv) is then used as the output of the super-resolution feature extraction C3K2-MAB module. The feature extraction C3K-MAB unit includes several feature extraction C3K-MAB layers connected in sequence. Each feature extraction C3K-MAB layer includes an eighth convolutional layer (Conv) and a ninth convolutional layer (Conv), a multi-scale attention MAB unit, and a tenth convolutional layer (Conv) connected in sequence. The output of the ninth convolutional layer (Conv) is input into the eighth convolutional layer (Conv) and the multi-scale attention MAB unit for processing. The outputs of the eighth convolutional layer (Conv) and the multi-scale attention MAB unit are fused using a concat operation, and then input into the tenth convolutional layer (Conv) for processing. The output of the fused convolutional layer (Conv) is then used as the output of the feature extraction C3K-MAB layer.

[0012] The C3K2-MAB super-resolution feature extraction module comprises an eleventh convolutional layer (Conv), a second split operation, a feature extraction C3k unit, and a twelfth convolutional layer (Conv) connected in sequence. The outputs of the eleventh convolutional layer (Conv), the second split operation, and the feature extraction C3k unit are fused using a concat operation, and then input into the twelfth convolutional layer (Conv) for further processing. The output is then used as the output of the C3K2-MAB super-resolution feature extraction module. The feature extraction C3k unit comprises several sequentially connected feature extraction C3k layers. Each C3k layer includes a thirteenth convolutional layer (Conv) and a fourteenth convolutional layer (Conv), a bottleneck unit (Bottleneck unit), and a fifteenth convolutional layer (Conv) connected in sequence. The output of the fourteenth convolutional layer (Conv) is fed into the thirteenth convolutional layer (Conv) and the bottleneck unit (Bottleneck unit) for processing. The outputs of the thirteenth convolutional layer (Conv) and the bottleneck unit (Bottleneck unit) are fused using a concat operation, and then fed into the fifteenth convolutional layer (Conv) for processing. The output of the fused convolutional layer (Bottleneck unit) is then used as the output of the feature extraction C3k layer. The bottleneck unit consists of several bottleneck layers connected in sequence. Each bottleneck layer consists of a sixteenth convolutional layer (Conv) and a seventeenth convolutional layer (Conv) connected in sequence. The input of the bottleneck layer (Bottleneck unit) and the output of the seventeenth convolutional layer (Conv) are added together, and the output of the sum is used as the output of the bottleneck layer (Bottleneck unit).

[0013] In step 4), for each target segmentation result, the edges of the target segmentation result are first represented using polygon coordinates. Then, the edge information is extracted using the Canny edge detection algorithm. Then, based on pixel topology and edge connectivity, connected regions of the same color are identified and boundary contours are extracted. That is, the disconnected positions in the extracted edge information are connected to obtain closed boundary contours. Then, a completely black canvas is created based on the cap image of the button mushroom to be phenotyped. The boundary contours are overlaid on the completely black canvas and filled with white to generate a mask of the fruiting body of a single button mushroom in the cap image of the button mushroom to be phenotyped. Then, morphological operations are performed on the fruiting body mask and the cap image of the button mushroom to be phenotyped. Based on the fruiting body mask, the original image region in the area where the fruiting body mask is located is extracted from the cap image of the button mushroom to be phenotyped as the fruiting body image.

[0014] In step 4), the specific quantification method for the phenotypic features of Agaricus bisporus for each sub-entity image is as follows: Step 4.1) Extract the contour point set of each mushroom cap from the sub-entity image, obtain the convex hull based on the contour point set, and then obtain the diameter of the mushroom cap of the button mushroom.

[0015] Step 4.2) Based on the sub-entity image, the roundness of the mushroom cap of the button mushroom is obtained using the optimized Minimum Zone Circle (MZC) algorithm.

[0016] Step 4.3) Based on the RGB values ​​of each pixel of the cap in the fruiting body image, use the Hunter whiteness formula to obtain the whiteness of the button mushroom.

[0017] Step 4.4) Based on the S-channel saturation values ​​of each pixel in the mushroom cap in the sub-entity image, determine whether there are color spots in the mushroom cap and the area where the color spots are located, and finally determine the color spot situation of the mushroom cap.

[0018] In step 4.1), the convex hull of the contour point set is obtained using the Andrew monotonic chain convex hull algorithm, and then the cap diameter of the button mushroom is obtained using the method based on the maximum distance between the convex hull vertices.

[0019] In step 4.2), the coordinates of each pixel of the mushroom cap in the sub-entity image are processed using the Minimum Region Circle (MZC) method, and the Particle Swarm Optimization (PSO) algorithm is used to optimize the Minimum Region Circle (MZC) method to finally obtain the roundness of the mushroom cap in the sub-entity image.

[0020] In step 4.4), the standard deviation of the S-channel saturation value of each pixel in the mushroom cap in the sub-entity image is obtained. When the standard deviation is greater than a preset threshold, there is a color spot in the mushroom cap; otherwise, there is no color spot. When there is a color spot in the mushroom cap, the K-Means clustering algorithm is used to divide the mushroom cap into three parts: background, mushroom body, and color spot, based on the S-channel saturation value of each pixel. Then, the color spot part is subjected to morphological processing, specifically, erosion and dilation operations are performed sequentially to obtain the processed color spot part. If the cleaned color spot part is not white, the area where the current cleaned color spot part is located is determined to be a color spot area; otherwise, the current cleaned color spot part is not a color spot. Finally, the color spot situation of the mushroom cap is determined.

[0021] The beneficial effects of this invention are: The model of this invention achieves improved accuracy on a self-built dataset compared to the original model, while reducing the number of parameters, floating-point operations, and memory usage. It can achieve a balance between high accuracy and low complexity in automated detection of button mushrooms. The phenotypic feature quantification method proposed in this invention meets the requirements for obtaining phenotypic parameters before harvesting, thus advancing the research on button mushroom phenotypic characteristics and providing a more accurate and efficient tool for edible fungi production and quality assessment. Attached Figure Description

[0022] Figure 1 This is a network structure diagram of the StarBlock module of the present invention; Figure 2 This is a network structure diagram of the target segmentation model including StarNet of the present invention; Figure 3 This is a partial sub-entity mask diagram of the present invention; Figure 4 This is a comparison chart of the three phenotypic parameters of the present invention, wherein, Figure 4 (a) is a scatter plot of the diameter of the button mushroom. Figure 4 (b) is a box plot of the diameter of the button mushroom. Figure 4 (c) is a violin diagram showing the diameter of the button mushroom. Figure 4 (d) is a scatter plot of the roundness of the button mushroom. Figure 4 (e) is a box plot of the roundness of the button mushroom. Figure 4 (f) is a violin diagram of the roundness of the button mushroom. Figure 4 (g) is a scatter plot of the whiteness of Agaricus bisporus. Figure 4 (h) is a box plot of the whiteness of Agaricus bisporus. Figure 4 (i) is a violin diagram of the whiteness of the button mushroom; Figure 5 This is a schematic diagram of the stain detection results of the present invention. Detailed Implementation

[0023] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0024] The automated phenotypic method for Agaricus bisporus based on machine vision of the present invention is as follows: Step 1) Acquire several raw images of button mushroom caps using a depth camera viewed from above, and then perform data augmentation and labeling to construct a training set. In this invention, the button mushroom images are acquired inside the mushroom growing room using a Microsoft Azure Kinect DK TOF depth sensor camera viewed from above. This camera is suitable for the confined space of the mushroom bed and, when used with a ring light source, can achieve uniform, shadow-free illumination. Due to perspective errors caused by the object distance, this invention, through calibration experiments, ultimately selected a camera height of 15cm for subsequent acquisition, capturing images from a top-down angle. Then, for each initial image of the mushroom cap and its augmented image, the edges of the cap structure of each button mushroom in the image are labeled, and labeled as large, medium, and small according to the size of the cap structure. In practice, the Augmentor tool is used to expand the training set to enhance mushroom features, avoid overfitting, and improve the model's generalization ability. The data augmentation methods specifically include reducing brightness, left-right mirroring, vertical mirroring, Gaussian noise, and Gaussian blur. After data augmentation of each initial image of a sub-entity, several images are obtained, which are then combined with the initial images to construct a training set to expand the data. The final training set contains thousands of images.

[0025] The experimental dataset of this invention contains 800 original images of Agaricus bisporus, with a resolution of 640 pixels × 480 pixels. Image annotation was performed using the ISAM (Image Segmentation Annotation Tool), a semi-automatic image segmentation and annotation tool. Considering that the small amount of the original dataset might lead to overfitting of the model on the training data, after data augmentation, the final training set contains 1920 images, the test set contains 480 images, and the validation set contains 480 images. The operating system used is Windows 10, and the deep learning framework is PyTorch. Detailed environment settings are as follows: CPU: Intel(R) Core(TM) i5-10600KF CPU @ 4.10GHz; Graphics Card: NVIDIA GeForce RTX 3080; Torch version: 2.3.0; Programming language: Python 3.8.12. The hyperparameters for the training phase are as follows: input image size is 640 pixels × 640 pixels, batch size is 32, no pre-trained weights are loaded, the optimizer used is stochastic gradient descent (SGD), the training epochs are 300 epochs, and online mosaic augmentation is used for 290 epochs during training. The initial learning rate is 0.02, and the momentum and weight decay values ​​are 0.9 and 0.0005, respectively.

[0026] Step 2) Construct a target segmentation model incorporating StarNet. For example... Figure 1 and Figure 2 As shown, the StarNet network includes a first convolutional layer (Conv) and four Star units connected in sequence. Each Star unit includes a Star module (StarBlock) and a second convolutional layer (Conv) connected in sequence. Each Star module (StarBlock) includes a first depthwise convolutional layer (DWConv), a first batch normalized BN operation, a first fully connected layer (FC), a second fully connected layer (FC), an activation function (ReLU6), an element-wise multiplication Star operation, a third convolutional layer (Conv), a second batch normalized BN operation, and a second depthwise convolutional layer (DWConv). The input of the Star module (StarBlock) is first processed by the first depthwise convolutional layer (DWConv) and the first batch normalized BN operation, and then input into the first and second fully connected layers (FC) respectively. The output of the second fully connected layer (FC) is processed by the activation function (ReLU6) and then element-wise multiplied by the output of the first fully connected layer (FC) using the Star operation. The activation function (ReLU6) can also be connected after the first fully connected layer (FC). Then, the output is processed by the third convolutional layer (Conv), the second batch normalized BN operation, and the second depthwise convolutional layer (DWConv) before being output as the output of the Star module (StarBlock).

[0027] In practice, the StarNet input size is 640×640×3. After processing by the first convolutional layer (Conv), the size becomes 320×320×8. Then, after processing by four Star units, the sizes become 160×160×8, 80×80×16, 40×40×32, and 20×20×64, respectively. The StarBlock module uses element-wise multiplication to effectively increase its implicit dimensionality, capturing richer and more complex feature information with a relatively low number of feature channels. The StarBlock module first extracts one-dimensional input features using a 7×7 depthwise convolution (DWConv). Then, a batch normalization (BN) operation is used to efficiently standardize the output, making the data distribution more stable and aiding in model training and convergence. Next, two 1×1 convolutions with a scaling factor of 4 are used to increase the dimensionality of the features to four dimensions. One of these two convolutions is activated using the ReLU6 activation function to introduce non-linearity and enhance the model's expressive power. Afterward, an element-wise multiplication (Star) operation is performed on the two outputs. This operation not only achieves the interactive fusion of global features but also exponentially amplifies the implicit dimensionality of the features, thus better capturing the relationships between them. Finally, 1×1 convolutions, BN operations, and a 7×7 DWConv operation are sequentially used to reduce the dimensionality of the features and further extract them, resulting in a more compact and effective feature representation.

[0028] The target segmentation model comprises StarNet, a fast spatial pyramid pooling module (SPPF), a cross-stage partial fusion module (C2PSA), a first dynamic upsampling module (DySample), a first super-resolution feature extraction module (C3K2-MAB), a second dynamic upsampling module (DySample), a third super-resolution feature extraction module (C3K2-MAB), a fourth convolutional layer (Conv), a third super-resolution feature extraction module (C3K2-MAB), a fifth convolutional layer (Conv), a second super-resolution feature extraction module (C3K2-MAB), a first segmentation head (Segment), a second segmentation head (Segment), and a third segmentation head (Segment). The input to the target segmentation model is processed sequentially by StarNet, SPPF, C2PSA, and the first dynamic upsampling module (DySample). The output of the first dynamic upsampling module (DySample) and the output of the third Star unit of StarNet are fused using a concat operation. Then, the output is sequentially processed by the first super-resolution feature extraction module (C3K2-MAB) and the second dynamic upsampling module (DySample). The DySample module processes the data, and the outputs of the second dynamic upsampling DySample module and the second Star unit of StarNet are fused and concatted. Then, the outputs are processed sequentially by the second super-resolution feature extraction C3K2-MAB module and the first segmentation head Segment, and the output is used as the large target segmentation result. The output of the second super-resolution feature extraction C3K2-MAB module is processed by the fourth convolutional layer Conv and fused and concatted with the output of the first super-resolution feature extraction C3K2-MAB module. Then, the outputs are processed sequentially by the third super-resolution feature extraction C3K2-MAB module and the second segmentation head Segment, and the output is used as the medium target segmentation result. The output of the third super-resolution feature extraction C3K2-MAB module is processed by the fifth convolutional layer Conv and fused and concatted with the output of the cross-stage partial fusion module C2PSA. Finally, the outputs are processed sequentially by the super-resolution feature extraction C3K2-MAB module and the third segmentation head Segment, and the output is used as the small target segmentation result. Finally, the large, medium and small target segmentation results are obtained.

[0029] In specific implementation, the output size of the Fast Spatial Pyramid Pooling (SPPF) module and the cross-stage partial fusion module C2PSA is 20×20×256, the output size of the first dynamic upsampling (DySample) module is 40×40×256, and the fused size becomes 40×40×288. The output size of the first super-resolution feature extraction (C3K2-MAB) module is 40×40×128, the output size of the second dynamic upsampling (DySample) module is 80×80×128, and the fused size becomes 80×80×144. The output size of the first module of the C3K2-MAB super-resolution feature extraction is 80×80×64, the output size of the fourth convolutional layer Conv is 40×40×64, and the fused size becomes 40×40×192. The output size of the first module of the C3K2-MAB super-resolution feature extraction is 40×40×128, the output size of the fifth convolutional layer Conv is 20×20×128, and the fused size becomes 20×20×384. The output size of the second module of the C3K2-MAB super-resolution feature extraction is 20×20×256.

[0030] The results from StarNet are input into the Fast Spatial Pyramid Pooling (SPPF) module. By sequentially performing multiple pooling layers at different scales on the feature maps, SPP's multi-scale feature extraction capabilities are preserved while reducing redundant computations and improving inference speed. The results from SPPF are then input into the Cross-Stage Partial Fusion (C2PSA) module. Through a cross-stage feature map splitting-processing-reassembly strategy, combined with Pyramid Squeeze Attention (PSA), the information flow interaction of multi-scale features is optimized, further enhancing the model's performance in detecting complex targets. The results from C2PSA are then input into the Dynamic Upsampling (DySample) module. By dynamically adjusting the sampling strategy, the feature map resolution is improved while more accurately restoring the edge contours and texture details of the button mushroom. Its adaptive weight allocation mechanism effectively avoids the drawbacks of traditional methods and significantly enhances the model's ability to perceive subtle features. The Concat operation is used to connect two or more tensors together along a certain dimension to generate a larger tensor. This can be used to increase the number of channels or feature dimensions in deep learning models in order to better capture the relationships between different features in subsequent layers.

[0031] Each super-resolution feature extraction C3K2-MAB module comprises a sixth convolutional layer (Conv), a first split operation, a feature extraction C3K-MAB unit, and a seventh convolutional layer (Conv) connected in sequence. The outputs of the sixth convolutional layer (Conv), the first split operation, and the feature extraction C3K-MAB unit are fused using a concat operation, and then fed into the seventh convolutional layer (Conv) for further processing. The output of the fused convolutional layer (Conv) is then used as the output of the super-resolution feature extraction C3K2-MAB module. The feature extraction C3K-MAB unit comprises several sequentially connected feature extraction... In practice, there can be 3-5 C3K-MAB layers. Each feature extraction C3K-MAB layer includes an eighth convolutional layer (Conv) and a ninth convolutional layer (Conv), a multi-scale attention MAB unit, and a tenth convolutional layer (Conv) connected in sequence. The output of the ninth convolutional layer (Conv) is input into the eighth convolutional layer (Conv) and the multi-scale attention MAB unit for processing. The outputs of the eighth convolutional layer (Conv) and the multi-scale attention MAB unit are fused using the Concat operation and then input into the tenth convolutional layer (Conv) for processing. The output of the fused C3K-MAB layer is then used as the output of the feature extraction C3K-MAB layer.

[0032] The C3K2-MAB two-module super-resolution feature extraction consists of an eleventh convolutional layer (Conv), a second split operation, a feature extraction C3k unit, and a twelfth convolutional layer (Conv) connected in sequence. The outputs of the eleventh convolutional layer (Conv), the second split operation, and the feature extraction C3k unit are fused using a concat operation, and then fed into the twelfth convolutional layer (Conv) for further processing. The output of the twelfth convolutional layer (Conv) is then used as the output of the C3K2-MAB two-module super-resolution feature extraction. The feature extraction C3k unit consists of several sequentially connected feature extraction C3k layers, which can be set to 3-5 in practice. Each C3k layer includes a thirteenth convolutional layer (Conv) and a fourteenth convolutional layer (Conv), a bottleneck unit (Bottleneck unit), and a fifteenth convolutional layer (Conv) connected in sequence. The output of the fourteenth convolutional layer Conv is fed into the thirteenth convolutional layer Conv and the bottleneck unit for processing. The outputs of the thirteenth convolutional layer Conv and the bottleneck unit are fused by a concat operation, and then fed into the fifteenth convolutional layer Conv for processing. The output is then used as the output of the feature extraction C3k layer. The bottleneck unit consists of several bottleneck layers connected in sequence. In specific implementations, there can be 3-5 bottleneck layers. Each bottleneck layer consists of the sixteenth convolutional layer Conv and the seventeenth convolutional layer Conv connected in sequence. The input of the bottleneck layer and the output of the seventeenth convolutional layer Conv are added together and the output is used as the output of the bottleneck layer.

[0033] The training set is then input into the target segmentation model for training, resulting in a trained Agaricus bisporus segmentation model. After training, during the validation phase, the dataset contains sub-entities with varying cap sizes, appearances, and colors, making it representative. Thirty images from the dataset were selected, and the diameter of the caps of 358 Agaricus bisporus mushrooms was measured using calipers. To evaluate the accuracy of the algorithm, this invention uses the coefficient of determination (R-squared), root mean square error (RMSE), and mean absolute error (MAE) as evaluation metrics. A comparison of phenotypic parameters between manual measurement and system estimation is shown below. Figure 4 As shown, this includes scatter plots, box plots, and violin plots.

[0034] like Figure 4 of (a) Figure 4 (b) and Figure 4 As shown in (c), the average measured diameter of the mushroom cap by manual measurement was 3.32 cm, with an average error of 3.82%. This result indicates that the algorithm used can effectively predict the mushroom cap diameter. The main reason for the error is that the mushroom cap is arc-shaped, and during manual measurement, the vernier caliper cannot be bent and can only be placed against the surface of the cap, resulting in unavoidable errors due to visual inspection. Since the error is within expectations, it is acceptable in practical applications.

[0035] like Figure 4 of (d), Figure 4 (e) and Figure 4 As shown in (f), in the measurement of mushroom cap roundness, the average measured value of the manually measured mushroom cap roundness is 1.16 mm, with an average error of 4.17%. This error is closely related to the limitations of the instance segmentation algorithm. There is adhesion between the fruiting bodies of Agaricus bisporus, and their color and texture differences are small, making accurate segmentation difficult and resulting in the extraction mask edges not being completely smooth. Furthermore, manual measurement also introduces some error, but since this error is within expectations, it is acceptable in practical applications.

[0036] like Figure 4 of (g), Figure 4 of (h) and Figure 4 As shown in (i), in the measurement of mushroom cap whiteness, the average measured value of manually measured mushroom cap whiteness was 81.971, with an average error of 1.74%. This error mainly stems from the effects of uneven lighting and shadows. Agaricus bisporus grows in clusters, and due to mutual shading, the brightness distribution on the mushroom surface is uneven. The presence of shadows reduces the brightness of certain parts of the image, thus affecting the whiteness value of these areas. The low brightness of shadowed areas also leads to a decrease in measurement results, causing the actual whiteness level to be underestimated.

[0037] Step 3) Use a depth camera to obtain a vertical top-down image of the cap of the mushroom to be phenotyped and input it into the mushroom segmentation model. After processing, the mushroom segmentation model outputs three target segmentation results.

[0038] Step 4) Obtain the sub-entity image of each target segmentation result, and use the Agaricus bisporus phenotypic feature quantization method to obtain the cap diameter, roundness, and whiteness of Agaricus bisporus, while determining the color spots on the cap, to achieve automated phenotypic identification of Agaricus bisporus. For each target segmentation result, firstly, the edges of the target segmentation result are represented using polygon coordinates, then the edge information is extracted using the Canny edge detection algorithm, and then, based on pixel topological relationships and edge connectivity, the same-color connected regions are identified and the boundary contours are extracted, that is, the disconnected positions in the extracted edge information are connected to obtain closed boundary contours; then, a completely black canvas is created based on the cap image of Agaricus bisporus to be phenotyped, the boundary contours are overlaid on the completely black canvas and filled with white, generating a sub-entity mask of a single Agaricus bisporus in the cap image of Agaricus bisporus to be phenotyped, and then morphological operations are performed on the sub-entity mask and the cap image of Agaricus bisporus to be phenotyped, based on the sub-entity mask, such as Figure 3 The image shown is a masked image of a single fruiting body. The original image region containing the fruiting body mask is extracted from the cap image of the mushroom to be phenotyped, and used as the fruiting body image.

[0039] In practice, the method for quantifying the phenotypic features of Agaricus bisporus for each sub-entity image is as follows: Step 4.1) Extract the contour point set of each mushroom cap from the sub-entity image, use the Andrew monotonic chain convex hull algorithm to obtain the convex hull of the contour point set, and then use the method based on the maximum distance between the convex hull vertices to obtain the diameter of the mushroom cap of the button mushroom.

[0040] Step 4.2) Based on the coordinates of each pixel of the mushroom cap in the sub-entity image, the minimum region circle (MZC) method is used for processing. At the same time, the particle swarm optimization algorithm (PSO) is used to optimize the minimum region circle (MZC) method, and finally the roundness of the mushroom cap in the sub-entity image is obtained.

[0041] The roundness of mushrooms in mushroom fruiting body images is a geometric concept used to measure the geometric deviation between the edge shape of the cap of *Agaricus bisporus* and a perfect circle. A method based on the Minimum Zone Circle (MZC) approach is used to evaluate the roundness of the cap. The core idea of ​​the MZC method is to determine a pair of concentric circles such that all measurement points lie within the annular region formed by these circles, and among many pairs of concentric circles satisfying this condition, the difference in their radii must be minimized. This minimum radius difference is defined as the roundness of the *Agaricus bisporus* cap. When this radius difference is 0, it means that all measurement points are exactly on the same circumference, and the roundness of the cap is 0. Determining roundness is essentially a minimization-maximization problem. The MZC algorithm is optimized based on the Particle Swarm Optimization (PSO) algorithm.

[0042] Step 4.3) Based on the RGB values ​​of each pixel on the cap in the fruiting body image, the whiteness of the button mushroom is obtained using the Hunter whiteness formula. The whiteness of the button mushroom is an important indicator for evaluating its freshness and quality, and is affected by many factors such as the growth environment, contamination by other fungi, and storage environment. To accurately describe the whiteness, the Hunter whiteness formula is proposed to be used for whiteness measurement.

[0043] Step 4.4) Based on the S-channel saturation values ​​of each pixel in the mushroom cap image, obtain the standard deviation of the S-channel saturation values ​​of each pixel. If the standard deviation is greater than a preset threshold, then there are color spots in the mushroom cap; otherwise, there are no color spots. When there are color spots in the mushroom cap, use the K-Means clustering algorithm to divide the mushroom cap into three parts: background, mushroom body, and color spots, based on the S-channel saturation values ​​of each pixel. Then, perform morphological processing on the color spot parts, specifically by performing erosion and dilation operations in sequence to obtain the processed color spot parts. If the cleaned color spot parts are not white, then the area where the current cleaned color spot parts are located is determined to be a color spot area; otherwise, the current cleaned color spot parts are not color spots. Finally, determine the color spot situation of the mushroom cap.

[0044] Extensive observation and experimental analysis revealed that the saturation of the mushroom cap spots was significantly higher than the background in the S-channel (saturation channel). Therefore, the S-channel was selected as the key channel for subsequent cluster analysis, using saturation differences to distinguish the spots from normal mushroom cap areas. Analysis of multiple sets of experimental data showed that the standard deviation of the S-channel can serve as an effective indicator of the presence of mushroom cap spots. When the standard deviation of the S-channel exceeds a set threshold, it indicates a significant color difference on the mushroom cap surface, i.e., the presence of spots.

[0045] To further accurately identify the color spot regions, the K-Means clustering algorithm is used, which can automatically divide the data into different clusters, corresponding to the three main parts of the image: the background, the mushroom body, and the color spots.

[0046] To further improve the accuracy of spot detection and reduce noise interference, morphological processing is performed on the binary image obtained after clustering. In this morphological processing, erosion followed by dilation is performed, which effectively cleans up small-area noise in the image. In mushroom images, some small noise points or small non-spot areas may be misidentified due to interference from color or other features. The above operations can remove these small areas, preserving the true spot areas, thus greatly improving the accuracy of spot detection and enabling us to more accurately identify spots on the mushroom cap surface.

[0047] like Figure 5 The image shows the results of the color spot detection. Color spot detection was performed on 186 Agaricus bisporus caps in the dataset. A total of 83 mushroom images with color spots were detected, while 12 images with color spots were missed. The main reason for the missed detections was that the color spots were relatively light, and the detection algorithm was not sensitive enough to areas similar to the cap color. In addition, some color spots in the images were very small, making them difficult to distinguish clearly at low resolution.

[0048] The above description and embodiments are merely preferred examples of the present invention and do not constitute any limitation on the present invention. Obviously, those skilled in the art, after understanding the content and design principles of the present invention, may make various modifications and changes in form and detail based on the principles and structure of the present invention. However, these modifications and changes based on the ideas of the present invention are still within the protection scope of the claims of the present invention.

Claims

1. An automated phenotypic method for *Agaricus bisporus* based on machine vision, characterized in that, include: Step 1) Use a depth camera to obtain several raw images of the caps of Agaricus bisporus from a vertical top-down view, and then perform data augmentation and labeling to construct a training set; Step 2) Construct a target segmentation model containing StarNet, input the training set into the target segmentation model for training, and obtain the trained mushroom segmentation model; Step 3) Use a depth camera to obtain the cap image of the mushroom to be phenotyped from a vertical top view and input it into the mushroom segmentation model. After processing, the mushroom segmentation model outputs three target segmentation results. Step 4) Obtain the sub-entity image of each target segmentation result, and use the button mushroom phenotypic feature quantification method to obtain the diameter, roundness and whiteness of the mushroom cap, and determine the color spot of the mushroom cap, so as to realize the automated phenotypicing of button mushroom.

2. The automated phenotypic method for Agaricus bisporus based on machine vision according to claim 1, characterized in that: In step 1), for each initial image of the mushroom cap and its data-enhanced image, the edges of the cap structure of each button mushroom in the image are marked, and labels of large, medium and small are marked according to the size of the cap structure.

3. The automated phenotypic method for Agaricus bisporus based on machine vision according to claim 1, characterized in that: In step 2), the StarNet network includes a first convolutional layer Conv and four Star units connected in sequence. Each Star unit includes a Star module StarBlock and a second convolutional layer Conv connected in sequence. Each Star module StarBlock includes a first depthwise convolutional layer DWConv, a first batch normalization BN operation, a first fully connected layer FC, a second fully connected layer FC, an activation function ReLU6, an element-wise multiplication Star operation, a third convolutional layer Conv, a second batch normalization BN operation, and a second depthwise convolutional layer DWConv. The input of the Star module StarBlock is first processed by the first depthwise convolutional layer DWConv and the first batch normalization BN operation, and then input into the first fully connected layer FC and the second fully connected layer FC respectively. The output of the second fully connected layer FC is processed by the activation function ReLU6 and then element-wise multiplied with the output of the first fully connected layer FC using the Star operation. Then it is processed by the third convolutional layer Conv, the second batch normalization BN operation, and the second depthwise convolutional layer DWConv in sequence before being output as the output of the Star module StarBlock.

4. The automated phenotypic method for Agaricus bisporus based on machine vision according to claim 3, characterized in that: In step 2), the target segmentation model includes StarNet, a fast spatial pyramid pooling module (SPPF), a cross-stage partial fusion module (C2PSA), a first dynamic upsampling module (DySample), a first super-resolution feature extraction module (C3K2-MAB), a second dynamic upsampling module (DySample), a third super-resolution feature extraction module (C3K2-MAB), a fourth convolutional layer (Conv), a third super-resolution feature extraction module (C3K2-MAB), a fifth convolutional layer (Conv), a second super-resolution feature extraction module (C3K2-MAB), a first segmentation head (Segment), a second segmentation head (Segment), and a third segmentation head (Segment). The input of the target segmentation model is processed sequentially by StarNet, SPPF, C2PSA, and the first dynamic upsampling module (DySample). The output of the first dynamic upsampling module (DySample) and the output of the third Star unit of StarNet are fused using a concat operation, and then sequentially processed by the first super-resolution feature extraction module (C3K2-MAB) and the second segmentation head (Segment). The second dynamic upsampling module (DySample) processes the data. The output of the second dynamic upsampling module (DySample) and the output of the second Star unit of StarNet are fused and concatted. Then, the data is processed sequentially by the second super-resolution feature extraction module (C3K2-MAB) and the first segmentation head (Segment), and the output is used as the segmentation result for large targets. The output of the second super-resolution feature extraction module (C3K2-MAB) is processed by the fourth convolutional layer (Conv) and then fused and concatted with the output of the first super-resolution feature extraction module (C3K2-MAB). This process is then repeated sequentially by the third super-resolution feature extraction module (C3K2-MAB) and the second segmentation head (Segment), and the output is used as the segmentation result for medium targets. The output of the third super-resolution feature extraction module (C3K2-MAB) is processed by the fifth convolutional layer (Conv) and then fused and concatted with the output of the cross-stage partial fusion module (C2PSA). Finally, the data is processed sequentially by the second super-resolution feature extraction module (C3K2-MAB) and the third segmentation head (Segment), and the output is used as the segmentation result for small targets. This process yields the segmentation results for large, medium, and small targets.

5. The automated phenotypic method for Agaricus bisporus based on machine vision according to claim 4, characterized in that: Each of the super-resolution feature extraction C3K2-MAB modules includes a sixth convolutional layer (Conv), a first split operation, a feature extraction C3K-MAB unit, and a seventh convolutional layer (Conv) connected in sequence. The outputs of the sixth convolutional layer (Conv), the first split operation, and the feature extraction C3K-MAB unit are fused using a concat operation, and then input into the seventh convolutional layer (Conv) for processing. The output of the fused convolutional layer (Conv) is then used as the output of the super-resolution feature extraction C3K2-MAB module. The feature extraction C3K-MAB unit includes several feature extraction C3K-MAB layers connected in sequence. Each feature extraction C3K-MAB layer includes an eighth convolutional layer (Conv) and a ninth convolutional layer (Conv), a multi-scale attention MAB unit, and a tenth convolutional layer (Conv) connected in sequence. The output of the ninth convolutional layer (Conv) is input into the eighth convolutional layer (Conv) and the multi-scale attention MAB unit for processing. The outputs of the eighth convolutional layer (Conv) and the multi-scale attention MAB unit are fused using a concat operation, and then input into the tenth convolutional layer (Conv) for processing. The output of the fused convolutional layer (Conv) is then used as the output of the feature extraction C3K-MAB layer. The C3K2-MAB super-resolution feature extraction module comprises an eleventh convolutional layer (Conv), a second split operation, a feature extraction C3k unit, and a twelfth convolutional layer (Conv) connected in sequence. The outputs of the eleventh convolutional layer (Conv), the second split operation, and the feature extraction C3k unit are fused using a concat operation, and then input into the twelfth convolutional layer (Conv) for further processing. The output is then used as the output of the C3K2-MAB super-resolution feature extraction module. The feature extraction C3k unit comprises several sequentially connected feature extraction C3k layers. Each C3k layer includes a thirteenth convolutional layer (Conv) and a fourteenth convolutional layer (Conv), a bottleneck unit (Bottleneck unit), and a fifteenth convolutional layer (Conv) connected in sequence. The output of the fourteenth convolutional layer (Conv) is fed into the thirteenth convolutional layer (Conv) and the bottleneck unit (Bottleneck unit) for processing. The outputs of the thirteenth convolutional layer (Conv) and the bottleneck unit (Bottleneck unit) are fused using a concat operation, and then fed into the fifteenth convolutional layer (Conv) for processing. The output of the fused convolutional layer (Bottleneck unit) is then used as the output of the feature extraction C3k layer. The bottleneck unit consists of several bottleneck layers connected in sequence. Each bottleneck layer consists of a sixteenth convolutional layer (Conv) and a seventeenth convolutional layer (Conv) connected in sequence. The input of the bottleneck layer (Bottleneck unit) and the output of the seventeenth convolutional layer (Conv) are added together, and the output of the sum is used as the output of the bottleneck layer (Bottleneck unit).

6. The automated phenotypic method for Agaricus bisporus based on machine vision according to claim 1, characterized in that: In step 4), for each target segmentation result, the edges of the target segmentation result are first represented using polygon coordinates. Then, the edge information is extracted using the Canny edge detection algorithm. The disconnected positions in the extracted edge information are connected to obtain a closed boundary contour. Then, a completely black canvas is created based on the cap image of the button mushroom to be phenotyped. The boundary contour is covered onto the completely black canvas and filled with white to generate a mask of the fruiting body of a single button mushroom in the cap image of the button mushroom to be phenotyped. Then, based on the fruiting body mask, the original image region in the area where the fruiting body mask is located is extracted from the cap image of the button mushroom to be phenotyped as the fruiting body image.

7. The automated phenotypic method for Agaricus bisporus based on machine vision according to claim 1, characterized in that: In step 4), the specific quantification method for the phenotypic features of Agaricus bisporus for each sub-entity image is as follows: Step 4.1) Extract the contour point set of each mushroom cap from the sub-entity image, obtain the convex hull based on the contour point set, and then obtain the diameter of the mushroom cap of the button mushroom. Step 4.2) Based on the sub-entity image, the optimized Minimum Circle Coefficient (MZC) algorithm is used to obtain the roundness of the mushroom cap of the button mushroom; Step 4.3) Based on the RGB values ​​of each pixel of the cap in the fruiting body image, use the Hunter whiteness formula to obtain the whiteness of the button mushroom; Step 4.4) Based on the S-channel saturation values ​​of each pixel in the mushroom cap in the sub-entity image, determine whether there are color spots in the mushroom cap and the area where the color spots are located, and finally determine the color spot situation of the mushroom cap.

8. The automated phenotypic method for Agaricus bisporus based on machine vision according to claim 7, characterized in that: In step 4.1), the convex hull of the contour point set is obtained using the Andrew monotonic chain convex hull algorithm, and then the cap diameter of the button mushroom is obtained using the method based on the maximum distance between the convex hull vertices.

9. The automated phenotypic method for Agaricus bisporus based on machine vision according to claim 7, characterized in that: In step 4.2), the minimum circle size (MZC) method is used to process the mushroom caps based on the coordinates of each pixel in the sub-entity image. At the same time, the particle swarm optimization algorithm (PSO) is used to optimize the minimum circle size (MZC) method, and finally the roundness of the mushroom caps in the sub-entity image is obtained.

10. The automated phenotypic method for Agaricus bisporus based on machine vision according to claim 7, characterized in that: In step 4.4), the standard deviation of the S-channel saturation value of each pixel in the mushroom cap in the sub-entity image is obtained. When the standard deviation is greater than a preset threshold, there is a color spot in the mushroom cap; otherwise, there is no color spot. When there is a color spot in the mushroom cap, the K-Means clustering algorithm is used to divide the mushroom cap into three parts: background, mushroom body, and color spot, based on the S-channel saturation value of each pixel. Then, the color spot part is subjected to morphological processing, specifically, erosion and dilation operations are performed sequentially to obtain the processed color spot part. If the cleaned color spot part is not white, the area where the current cleaned color spot part is located is determined to be a color spot area; otherwise, the current cleaned color spot part is not a color spot. Finally, the color spot situation of the mushroom cap is determined.

Images