Method and system for identifying color change degree of shiitake mushroom sticks based on deep learning

Abstract

The present invention relates to the field of agricultural information technology, and in particular to a method and system for identifying the color change degree of shiitake mushroom sticks based on deep learning. The present invention aims to provide a method and system for identifying the color change degree of shiitake mushroom sticks based on deep learning. Through the steps of data collection, data enhancement, image annotation, model construction and model training, a deep learning model is used to realize automatic recognition of the color change degree of shiitake mushroom sticks, quantify the color change degree, and improve recognition accuracy and efficiency.

Description

Technical Field

[0001] The present invention relates to the field of agricultural information technology, and in particular to a method and system for identifying the color change degree of shiitake mushroom sticks based on deep learning. Background Art

[0002] In recent years, my country's edible fungi industry has developed rapidly and is one of the most important innovative industries in my country's agricultural economy. Among them, shiitake mushrooms have multiple pharmacological effects such as enhancing human immunity, improving nutritional anemia, and preventing and treating tumors. They are well-known edible and medicinal mushrooms and are known as the "king of mushrooms." At the same time, shiitake mushrooms are the largest-volume variety in the edible fungi industry. Because of their nutritional and medicinal values, they are gradually recognized by consumers and are gradually widely used in daily diet and medicine.

[0003] In the process of shiitake mushroom cultivation, the color change of mycelium after ripening is the key stage of shiitake mushroom fruiting. The maturity of shiitake mushrooms is achieved through three stages: mycelium growth, differentiation and fruiting body formation. During the mycelium growth and differentiation stages, suitable environmental conditions such as temperature, light and humidity help promote the color change of shiitake mushroom sticks. The degree of color change reflects the metabolic activity and the degree of material transformation inside the shiitake mushroom sticks. It is related to a variety of environmental factors and is an important indicator for measuring the maturity of the mushroom sticks, which directly affects the yield and quality of the later shiitake mushrooms. At present, with the rise of precision agriculture and modern agriculture, environmental control and information collection in the production process of shiitake mushrooms have achieved remarkable results. On the other hand, considerable breakthroughs have also been made in the field of information technology such as picking robots and special sensors. However, despite the progress in the application of information technology for shiitake mushrooms, the identification of the color change degree of shiitake mushroom sticks still relies on the subjective experience and judgment of the growers, which requires the growers to have rich experience, and there are problems such as low efficiency and poor accuracy. An intelligent and accurate color change degree calculation method is needed.

[0004] Therefore, this application studies a method for identifying the color change degree of shiitake mushroom sticks based on deep learning, and develops a shiitake mushroom stick color change degree identification system and APP, which are used to scientifically, quickly and accurately determine the color change degree of the mushroom sticks, solve the problems of low recognition efficiency and poor accuracy in color change degree assessment, provide decision-making basis for determining the maturity of mushroom sticks during the spawning period and environmental regulation, and further improve the efficiency and quality of shiitake mushroom production. Summary of the invention

[0005] The purpose of the present invention is to provide a method and system for identifying the color change degree of shiitake mushroom sticks based on deep learning. Through the steps of data collection, data enhancement, image annotation, model construction and model training, a deep learning model is used to realize automatic identification of the color change degree of shiitake mushroom sticks, quantify the color change degree, and improve the recognition accuracy and efficiency.

[0006] To achieve the above object, the present invention provides a method for identifying the color change degree of shiitake mushroom sticks based on deep learning, and the steps are as follows:

[0007] - Data collection: Use a camera to collect image data of shiitake mushroom sticks during the color change period;

[0008] - Data enhancement: Brightness enhancement and reduction processing of collected image data to expand the data set and simulate different lighting conditions;

[0009] - Image annotation: A method combining manual annotation and HSV color space extraction is used to annotate images at the pixel level and generate an annotation dataset;

[0010] - Model construction: Optimize and improve the U-Net model, including replacing the backbone network, adding multi-scale convolutional attention (MSCA), modifying the skip connection structure, adding channel attention (CSE), and optimizing the loss function to build a DMS-UNet model for the segmentation of shiitake mushroom stick color change images;

[0011] - Model training: Use the constructed labeled dataset to train and test the DMS-UNet model and store the optimal weight file;

[0012] - Color change degree identification: Use the trained DMS-UNet model to perform pixel-level segmentation on the mushroom stick image, calculate the proportion of pixel values ​​of different categories based on the segmentation results, and then obtain the proportion of color change area and quantify the color change degree.

[0013] Furthermore, the mushroom stick images collected in the data collection step during the spawning period include mushroom stick images with a color change area accounting for 0-25%, mushroom stick images accounting for 25-50%, mushroom stick images accounting for 50-75%, and mushroom stick images accounting for 75-100%.

[0014] Furthermore, the data enhancement step includes enhancing and reducing the brightness of each image by 30% to simulate different lighting conditions.

[0015] Furthermore, the image annotation step combines manual annotation with color extraction, specifically including:

[0016] - Manual annotation: Perform USM sharpening on the original image to improve image quality and details; use the "magnetic lasso" tool to draw the selection of the mushroom sticks and the inoculation holes of the mushroom sticks, and fill them with different colors;

[0017] - Color extraction: Convert the image to HSV color space, extract the uncolored area with a color range of (100, 43, 46) to (124,255, 255), and fill it with color. The remaining pixels are classified as the colored area.

[0018] Furthermore, the model building step specifically includes:

[0019] - Use the first 13 layers of the VGG16 network as the feature extraction network of U-Net to extract features of different levels in the color change image of shiitake mushroom sticks;

[0020] - The MSCA module is introduced at the bottom layer of the encoder to improve the segmentation performance of the model by aggregating local deep convolutions, multi-branch deep strip convolutions that capture multi-scale contextual information, and 1×1 convolutions that model the relationship between different channels;

[0021] - Use SE (Squeeze and Excitation) module to enhance feature extraction effect;

[0022] - Optimization of loss function: The combination of two loss functions is called FD Loss, which specifically includes the combination of Focal Loss and Dice Loss.

[0023] The calculation formula of Focal Loss function is as follows:

[0024] ;

[0025] in, represents the model prediction sample, The probability of belonging to a positive sample, Representation sample The weight of Indicates the adjustment factor of sample difficulty. In this paper, γ is set to 2.0;

[0026] The Dice loss function calculation formula is as follows:

[0027] ;

[0028] in, represents the intersection of sets X and Y, represents the number of elements of the true label, Represents the number of elements of the prediction mask, Represents the adjustment parameter to avoid the denominator being zero. Set to 1× ;

[0029] The FD Loss loss function calculation formula is as follows:

[0030] .

[0031] Furthermore, the model training step includes:

[0032] - The constructed dataset is randomly divided into training set, test set and validation set in a ratio of 7:2:1;

[0033] - Use Adam optimizer for model training;

[0034] - Through the cross-validation method, select the optimal training parameters and store the optimal weight file.

[0035] Furthermore, the calculation formula of the color change degree is as follows:

[0036] ;

[0037] in, Represents the color change degree of shiitake mushroom sticks; Represents the total number of uncolored pixel values; Represents the total number of pixel values ​​that have changed color.

[0038] A system for identifying the color change degree of shiitake mushroom sticks based on the above method, the system comprising:

[0039] - Front-end system: developed based on the Vue framework, used for user interface interaction;

[0040] - Backend system: developed based on SpringBoot framework, used for data processing and model calling;

[0041] - Database: used to store the historical records and results of the color change degree identification of shiitake mushroom sticks;

[0042] - Mobile application: Developed on the Android platform, the model is deployed using the NCNN framework to identify the degree of color change at the production site.

[0043] Furthermore, the web page uploads the image of the mushroom sticks through the front-end page, and the back-end system is responsible for receiving the uploaded image data from the web page, and using the DMS-UNet model to perform image segmentation, calculate the color change area ratio, and upload the results to the database.

[0044] The research object of this technical solution is the shiitake mushroom sticks during the color change period. The photos are taken in the culture shed using a smartphone. The phone is fixed on a tripod at a certain distance above the shiitake mushroom sticks. The image resolution is 3000 pixels × 4000 pixels, and the format is JPG. In order to ensure the diversity of the data and avoid overfitting of the model due to a large proportion of a certain type of data, the collected images of mushroom sticks during the spawning period include images of mushroom sticks with a color change area of ​​0-25%, images of mushroom sticks of 25-50%, images of mushroom sticks of 50-75%, and images of mushroom sticks of 75-100%, totaling 785 images. The collected images of the shiitake mushroom sticks during the color change stage were classified and sorted to ensure the balance of the training data in order to improve the generalization and robustness of the model. Samples of shiitake mushroom sticks with different degrees of color change during the color change period are as follows: Figure 1 shown.

[0045] In the task of identifying the color change degree of shiitake mushroom sticks, in order to reduce the impact of lighting changes on the model recognition results, the brightness enhancement and reduction methods were used to enhance the original image data, simulate images under different lighting conditions, and expand the color change data set so that the model can better adapt to images under different lighting conditions. After data enhancement, the data set was expanded to 2355. Among them, the images of mushroom sticks with a color change area of ​​0-25%, 25-50%, 50-75%, and 75-100% were 3 times the number of pictures taken. The enhanced data was used to train the segmentation model to improve the generalization ability and robustness of the model. The data enhancement results are shown in Figure 2.

[0046] Before training the image segmentation model of shiitake mushroom stick color change, it is necessary to annotate the mushroom stick color change image to produce label data. However, due to the large amount of data and the small and large number of targets in the color change area of ​​the mushroom stick, pixel-level annotation is time-consuming and laborious. Therefore, this paper uses a combination of manual annotation and color extraction to complete data annotation.

[0047] (1) Manual labeling

[0048] In order to distinguish different categories and fill in custom colors, mapping software was used for annotation. The manual annotation results of shiitake mushroom sticks are shown in Figure 3.

[0049] Before pixel-level annotation, USM sharpening is performed to improve the quality and details of the mushroom stick image. Through repeated attempts and adjustments to the USM sharpening parameters, it is ensured that the sharpening effect is suitable for the color-changing image of the mushroom stick and does not introduce obvious noise. After sharpening, the "magnetic lasso" tool is used to draw the selection of the mushroom stick and the inoculation hole of the mushroom stick. This tool can automatically identify the edge of the background and foreground color in the image, and is suitable for images with a single background color or clear edges and strong contrast. After the selection is drawn, the background and inoculation holes are filled with different colors to achieve different categories, preparing for the next step of HSV color space extraction.

[0050] (2) HSV color space extraction

[0051] When processing the color-changing images of shiitake mushroom sticks, the surface color of the sticks shows certain hue changes and saturation differences, rather than a single pure color. The traditional RGB color space is difficult to accurately capture this complex color change, so the HSV color space is used for color extraction. By adjusting the hue, saturation, and brightness parameters, the HSV color space can more accurately define the range and characteristics of the color, thereby more effectively extracting and segmenting the target. According to the extraction results of the non-color-changing area and the above-mentioned manual annotation results, all the remaining pixels are classified as the color-changing area and filled with color, and finally an image that distinguishes the four categories of background, color-changing, non-color-changing, and inoculation holes is obtained. The final labeling result is as follows: Figure 4 shown.

[0052] By combining manual annotation with HSV color space extraction, the workload of manual annotation is reduced, thereby improving annotation efficiency. The processed original images and labeled images are randomly divided into training sets, test sets, and validation sets in a ratio of 7:2:1 to ensure that the data samples are evenly distributed and prepare for subsequent model training.

[0053] Model construction: Optimize and improve the U-Net model, including replacing the backbone network, adding multi-scale convolutional attention (MSCA), modifying the skip connection structure, adding channel attention (CSE), and optimizing the loss function to build a DMS-UNet model for the segmentation of shiitake mushroom stick color change images;

[0054] - Model training: Use the constructed labeled dataset to train and test the DMS-UNet model and store the optimal weight file;

[0055] - Color change degree recognition: Use the trained DMS-UNet model to recognize the color change degree of mushroom stick images, calculate the color change area ratio, and quantify the color change degree.

[0056] This technical solution aims to solve the problem of lack of intelligent recognition of the color change degree of shiitake mushroom sticks. It studies the data processing method of shiitake mushroom stick color change, builds a shiitake mushroom stick color change image segmentation model based on semantic segmentation algorithm, calculates the color change rate using the model segmentation results, and designs and develops a shiitake mushroom stick color change degree recognition system and APP based on the shiitake mushroom stick color change image segmentation model, achieving the purpose of automatic recognition of the color change degree of shiitake mushroom sticks. The main research conclusions are as follows:

[0057] (1) A dataset of shiitake mushroom stick color change was constructed. The sample data of shiitake mushroom stick color change images were collected by field shooting, and the data was enhanced by brightness enhancement and attenuation to reduce the impact of lighting changes on the model recognition results. The image annotation method was studied, and the pixel-level annotation of the color change area, non-color change area, inoculation hole and background of the shiitake mushroom stick was completed by combining manual annotation and HSV color extraction. A dataset of 2355 images of shiitake mushroom stick color change was successfully constructed, which provided reliable data support for the subsequent training of the shiitake mushroom stick color change image segmentation model.

[0058] (2) A shiitake mushroom stick color change image segmentation model based on a semantic segmentation algorithm was constructed. By studying the U-Net model, replacing the model's backbone network, modifying the skip connection structure, adding multi-scale convolutional attention and channel attention modules, and adjusting the loss function, the model recognition accuracy was further improved. The results of the comparative test showed that the model performed best in the shiitake mushroom stick color change image segmentation task, with an accuracy rate and average pixel precision of 95.76% and 98.48% respectively. The shiitake mushroom stick color change image segmentation model can accurately segment the color change area, non-color change area, inoculation hole and background of the shiitake mushroom stick at the pixel level, providing a good basis for further obtaining the color change degree of the shiitake mushroom stick.

[0059] (3) A system for identifying the color change degree of shiitake mushroom sticks was designed and implemented. The overall architecture design of the system was completed through analysis of the system's target users, functional requirements, and non-functional requirements. The web page development was completed based on the Vue framework and SpringBoot framework. Based on the constructed shiitake mushroom stick color change image segmentation model, the functions of identifying the color change degree of shiitake mushroom sticks, storing the identification results, and managing the color change degree identification information were implemented. Through the application of the system, the color change degree of mushroom sticks was accurately and quickly identified, providing auxiliary decision-making for the determination of the maturity of mushroom sticks and environmental regulation during the spawning period. At the same time, a mobile phone APP for identifying the color change degree of shiitake mushroom sticks was developed based on JAVA and C++, and the model was deployed using the NCNN framework, which facilitated users to use smartphones to accurately identify the color change degree of shiitake mushroom sticks at the production site.

[0060] The beneficial effect of the present invention is that it provides an efficient and accurate method and system for identifying the color change degree of shiitake mushroom sticks, realizes the quantitative evaluation of the color change degree of mushroom sticks, and provides a scientific basis for the determination of the maturity of mushroom sticks during the spawning period and environmental regulation. Through automated identification and evaluation, the efficiency and quality of shiitake mushroom cultivation are improved, the dependence on manual experience is reduced, and it is helpful to realize the intelligentization and modernization of the shiitake mushroom cultivation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 These are samples of shiitake mushroom sticks with different degrees of color change during the color change period;

[0062] Figure 2 It is a schematic diagram of data enhancement results;

[0063] Figure 3 It is the result of manual annotation;

[0064] Figure 4 This is a schematic diagram of the final label result extracted from the HSV color space;

[0065] Figure 5 This is a schematic diagram of the DMS-UNet model structure;

[0066] Figure 6 This is a schematic diagram of the VGG16 network structure;

[0067] Figure 7 This is a schematic diagram of the network structure of the MSCA module;

[0068] Figure 8 This is a schematic diagram comparing the skip connection structures of U-Net and D-UNet;

[0069] Fig. 9 is the network structure of the SE module;

[0070] Fig.10 It is the comparison result of different network model experiments;

[0071] Fig.11 It is a schematic diagram of the main interface of the system;

[0072] Fig.12 It is a schematic diagram of the system recognition effect;

[0073] Fig.13 It is a schematic diagram of the management of color change degree identification records;

[0074] Fig.14 This is a schematic diagram of the mobile terminal display. DETAILED DESCRIPTION

[0075] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the technical solution of the invention is further explained below in conjunction with specific implementation methods.

[0076] In the description of the present invention, it should be noted that the terms "upper", "lower", "inner", "outer", "two ends", "one end", "the other end" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0077] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "provided with", "connected", etc. should be understood in a broad sense. For example, "connected" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0078] Construction of a dataset for the color change of shiitake mushroom sticks

[0079] Data collection

[0080] The research object of this embodiment is the shiitake mushroom sticks during the color change period. The Xiaomi 11 mobile phone was used to shoot in the culture shed. The phone was fixed on a tripod, about 50 cm above the shiitake mushroom sticks. The image resolution was 3000 pixels × 4000 pixels, and the format was JPG. In order to ensure the diversity of the data and avoid overfitting of the model due to a large proportion of a certain type of data, the collected images of mushroom sticks during the spawning period include 195 images of mushroom sticks with a color change area of ​​0-25%, 192 images of mushroom sticks with a color change area of ​​25-50%, 200 images of mushroom sticks with a color change area of ​​50-75%, and 198 images of mushroom sticks with a color change area of ​​75-100%, totaling 785 images. The collected pictures of the shiitake mushroom sticks during the color change stage were classified and sorted to ensure the balance of the training data in order to improve the generalization and robustness of the model. Samples of shiitake mushroom sticks with different degrees of color change during the color change period are as follows: Figure 1 shown.

[0081] Data Augmentation

[0082] In the task of identifying the color change degree of shiitake mushroom sticks, in order to reduce the impact of lighting changes on the model recognition results, the brightness enhancement and reduction methods were used to perform data enhancement on the original images, simulate images under different lighting conditions, and expand the color change data set so that the model can better adapt to images under different lighting conditions. After data enhancement, the data set was expanded to 2355 images. Among them, there are 585 mushroom stick images with a color change area of ​​0-25%, 561 mushroom stick images with a color change area of ​​25-50%, 600 mushroom stick images with a color change area of ​​50-75%, and 594 mushroom stick images with a color change area of ​​75-100%. The enhanced data is used to train the segmentation model to improve the generalization ability and robustness of the model. The data enhancement results are shown in the figure below. Figure 2 shown.

[0083] Image Annotation

[0084] Before training the image segmentation model of shiitake mushroom stick color change, it is necessary to annotate the mushroom stick color change image to produce label data. However, due to the large amount of data and the small and large number of targets in the color change area of ​​the mushroom stick, pixel-level annotation is time-consuming and laborious. Therefore, this paper uses a combination of manual annotation and color extraction to complete data annotation.

[0085] (1) Manual labeling

[0086] In order to distinguish different categories and fill in custom colors, Adobe Photoshop CC 2018 was used for annotation. The manual annotation results of shiitake mushroom sticks are shown in Figure 3.

[0087] Before pixel-level annotation, USM sharpening is performed to improve the quality and details of the mushroom stick image. Through repeated attempts and adjustment of USM sharpening parameters, this paper sets the radius of USM sharpening to 40 pixels, the amount to 60%, and the threshold to 0 to ensure that the sharpening effect is suitable for the color change image of the mushroom stick and does not introduce obvious noise. After sharpening, the "magnetic lasso" tool is used to draw the selection of the mushroom stick and the mushroom stick inoculation hole. This tool can automatically identify the edge of the background and foreground color in the image, and is suitable for images with a single background color or clear edges and strong contrast. After the selection is drawn, the background and inoculation holes are filled with different colors to achieve different categories, preparing for the next step of HSV color space extraction.

[0088] (2) HSV color space extraction

[0089] When processing the color change images of shiitake mushroom sticks, the surface color of the mushroom sticks shows certain hue changes and saturation differences, rather than a single pure color. The traditional RGB color space is difficult to accurately capture this complex color change, so the HSV color space is used for color extraction. The HSV color space can more accurately define the range and characteristics of colors by adjusting the hue, saturation, and brightness parameters, thereby more effectively extracting and segmenting targets.

[0090] This embodiment uses the OpenCV library provided by Python to convert the image from the original RGB color space to the HSV color space. Since the color-changing part of the mushroom stick has more colors, after converting the image to the HSV color space, the non-color-changing area with a more single color is extracted, and the extracted content is filled with color. Through continuous attempts and adjustment of color parameters, this paper sets the color threshold range from (100, 43, 46) to (124, 255, 255) to extract the non-color-changing area of ​​the mushroom stick image. According to the extraction results of the non-color-changing area and the above-mentioned manual annotation results, all the remaining pixels are classified as the color-changing area, and color filling is performed, and finally an image that distinguishes four categories: background, color change, non-color change, and inoculation hole is obtained. The final labeling result is shown in Figure 4.

[0091] By combining manual annotation with HSV color space extraction, the workload of manual annotation is reduced, thereby improving annotation efficiency. In order to meet the needs of model training, the data set uses the VOC2007 format, processes the annotated images, and uniformly produces them into 8-bit PNG format labels. The processed original images and labeled images are randomly divided into training sets, test sets, and validation sets in a ratio of 7:2:1 to ensure that the data samples are evenly distributed and prepare for subsequent model training.

[0092] Construction of Segmentation Model for Color Change Image of Shiitake Mushroom Sticks

[0093] This embodiment is based on the optimization of the U-Net model. In view of the uneven distribution of different categories of the mushroom stick color change images and the rich color and texture details of the mushroom stick surface, the model is improved by replacing the backbone network of the model, adding multi-scale convolutional attention, modifying the skip connection structure, adding channel attention, and optimizing the model loss function to establish a segmentation model suitable for the mushroom stick color change images. The improved model is named DMS-UNet, and its network structure is as follows: Figure 5 shown.

[0094] Encoder

[0095] (1) Backbone network

[0096] In order to obtain deeper texture and edge feature information in the image segmentation task of the color change of shiitake mushroom sticks, the first 13 layers of the VGG16 network are used as the U-Net feature extraction network. As a representative deep learning model, VGG16 has a deeper network structure and can effectively capture the features of different levels in the color change image. It has advantages in processing the texture, edge and other detailed features involved in the color change image of the shiitake mushroom stick. Its lower network level and progressive structure make VGG16 suitable for the training of small-scale data sets, and more suitable for the image segmentation task of the color change of shiitake mushroom sticks. It can effectively extract image features, thereby improving the accuracy and performance of the segmentation model.

[0097] Specifically, a 512×512×3 image of a mushroom stick changing color is used as input. The first layer uses two 64-channel 3x3 convolutions and ReLU activation functions to extract low-level features of the image and obtain a preliminary effective feature map of 512×512×64. Through 2×2 maximum pooling, the size of the feature map is reduced, and important feature information is retained to obtain a feature map of 256×256×64. The second layer uses two 128-channel 3x3 convolutions and ReLU activation functions to obtain a preliminary effective feature map of 256×256×128. Through 2×2 maximum pooling, a feature map of 128×128×128 is obtained. The third layer uses three 256-channel 3x3 convolutions and ReLU activation functions to obtain a preliminary effective feature map of 128×128×128. Through 2×2 maximum pooling, a feature map of 64×64×256 is obtained. The fourth layer uses three 512-channel 3x3 convolutions and ReLU activation functions to obtain a preliminary effective feature map of 64×64×256. Through 2×2 maximum pooling, a feature map of 32×32×512 is obtained. The fifth layer uses three 512-channel 3x3 convolutions and ReLU activation functions to obtain a preliminary effective feature map of 32×32×512. Through the above steps, multiple convolution and pooling operations are performed to increase the depth and complexity of the network, and five preliminary effective feature maps are obtained. The VGG16 network structure is as follows Figure 6 shown.

[0098] (2) Multi-scale convolutional attention

[0099] In order to better capture the multi-scale information in the color-changing image, the MSCA (Multi-Scale Convolution Attention) module proposed by the SegNeXt network is introduced at the bottom layer of the encoder to improve the model's perception of the multi-scale detail features in the mushroom stick image and improve the performance in the color-changing image segmentation task. The MSCA module mainly consists of three parts: deep convolution for aggregating local information, multi-branch deep strip convolution for capturing multi-scale contextual information, and 1×1 convolution for modeling the relationship between different channels. The MSCA network structure diagram is shown in Figure 7.

[0100] The MSCA module takes the 32×32×512 feature map obtained by the feature extraction network as input. First, the local aggregation deep convolution part uses a 5×5 convolution kernel to perform group convolution on the input feature map to obtain a preliminary feature map. Then, the multi-branch deep strip convolution part performs three groups of group convolution operations of different sizes on the preliminary feature map, extracting features of different levels and scales without introducing too many parameters, and capturing receptive fields of different sizes. The 1×7 and 7×1 convolution kernels are used to capture the local features and details of the color-changing image, and the 1×21 and 21×1 convolution kernels are used to capture the global features and semantic information of the color-changing image. Finally, the 1×1 convolution used to model the relationship between different channels performs a convolution operation on the obtained features, integrates and fuses the color-changing image features, improves the quality of the fused feature representation, and generates the final feature map.

[0101] Decoder

[0102] (1) Skip Connections

[0103] In the U-NET model, the feature map in the encoding process is connected with the feature map in the decoding process through the jump connection, and the model can obtain the feature information of the color change image of the mushroom stick from both the shallow features and the deep features. The jump connection of U-Net is shown in Figure 8 (a). The image feature of the color change of the mushroom stick mainly comes from the change of the color of the hyphae, which shows obvious irregularity according to the growth cycle, covering the color change of a very fine single hyphae to the color change of a large area of ​​hyphae. However, in the study of the color change degree recognition method of shiitake mushroom sticks based on deep learning, since the jump connection mainly relies on upsampling and downsampling operations, the deep features may not be able to retain enough high-frequency detail information after multiple downsampling. In this case, for the color change mushroom stick image, some subtle color change features may not be accurately segmented. In addition, with the downsampling operation of the network, the resolution of the feature map gradually decreases, which may cause the details of the color change image segmentation result to be unclear, resulting in blurred or misaligned boundaries between different categories of mushroom sticks. To this end, the present invention proposes a new connection method DC (Dual-scale Skip Connection), called D-UNet.

[0104] The proposed DC connects the feature map of each encoder layer with the feature map of the corresponding decoder layer and the corresponding feature map of the previous layer, realizing the transmission and fusion of two scale features. Compared with the traditional U-Net jump connection, it makes fuller use of the underlying details and high-level semantic information, bridges the gap between different features, and thus more comprehensively understands the image context, restores the details of the discolored image, and improves the segmentation accuracy of the boundary details of the discolored area. The D-UNet structure is shown in Figure 8 (b).

[0105] (2) Channel Attention

[0106] In order to improve the segmentation accuracy of the U-Net model, the SE (Squeeze-and-Excitation, channel attention) module is introduced in the last layer of the U-Net decoder. The SE module can adaptively focus on the color-changing features and suppress irrelevant features, so that the network can globally integrate the contextual information of the input image before generating the final output and enhance the representation of the target features. This method ensures that the network's representation of features in the final stage is more accurate and targeted, improving the accuracy of the segmentation results. The SE network structure is shown in Figure 9.

[0107] The SE module implements adaptive feature weighting through two key steps. First, the input feature map is compressed into a 1×1×C vector through a global average pooling operation. Then, the 1×1×C vector is processed through two fully connected layers to generate channel weight values. Finally, the generated weight vector is assigned to the feature map to obtain the feature map.

[0108] Loss Function

[0109] In the original U-Net model, the cross entropy loss function is used to measure the difference between the model's classification prediction and the true segmentation label. The cross entropy loss function quantifies the accuracy of the prediction by calculating the logarithmic difference between the predicted probability and the true label for each pixel. The calculation formula of the cross entropy loss function includes the number of positive and negative samples as well as the predicted value and the true value. By minimizing the cross entropy loss, the model can better learn the distinguishing features between different categories, thereby improving classification accuracy. The calculation formula of the cross entropy loss function is shown below.

[0110] ;

[0111] Among them, M represents the number of positive samples, N represents the number of negative samples, yi represents the true value, and pi represents the predicted value.

[0112] However, when dealing with the color change segmentation task of shiitake mushroom sticks, due to the obvious differences in pixel levels of different categories, the cross-entropy loss function may cause the model to have less impact on minority categories and make it difficult to balance category weights, which in turn leads to a bias towards the background that accounts for a larger proportion and ignores the categories that account for a smaller proportion, affecting segmentation accuracy and generalization ability.

[0113] To solve this problem, this paper adopts the combined loss function FD Loss which combines Focal Loss and Dice Loss.

[0114] (1) Focal Loss

[0115] Focal Loss is a loss function for class imbalance. It reduces the weight of easy-to-classify samples and makes the model pay more attention to difficult-to-classify samples, thereby increasing the sensitivity to minority categories. The calculation formula of Focal Loss loss function is as follows.

[0116] ;

[0117] in, represents the model prediction sample, The probability of belonging to a positive sample, Representation sample The weight of Indicates the adjustment factor of sample difficulty. In this paper, γ is set to 2.0;

[0118] (2) Dice Loss

[0119] Dice Loss is used to process pixel-level segmentation tasks. Dice Loss measures the accuracy of segmentation by calculating the similarity between the predicted result and the true label. The Dice loss function has advantages in dealing with noise and fine structures, which can reduce the risk of overfitting. By minimizing the Dice loss, the model can better handle details and improve the accurate recognition of the color change area. The Dice Loss loss function calculation formula is shown below.

[0120] 𝐷𝐿 ;

[0121] in, represents the intersection of sets X and Y, represents the number of elements of the true label, Represents the number of elements of the prediction mask, Represents the adjustment parameter to avoid the denominator being zero. Set to 1× .

[0122] (3) FD Loss

[0123] This paper combines the advantages of the two loss functions and uses FD Loss for the color image segmentation task to solve the problems of small areas and detail features and class imbalance. The calculation formula of FD Loss loss function is as follows.

[0124] ;

[0125] Model training:

[0126] The constructed labeled dataset is used to train and test the DMS-UNet model and store the optimal weight file.

[0127] This embodiment uses the PyTorch deep learning framework combined with the Python programming language to build an experimental environment. The computer configuration includes: Windows 10 operating system, 64GB system memory, Intel® Core™ i9-12900 2.40GHz CPU, NVIDIA GeForce RTX 3090 graphics card, and 24GB video memory. In this environment, the transfer learning method is used to initialize the network model backbone network using pre-trained weights. First, freeze training is performed to freeze the model backbone and only fine-tune the network; then, unfreeze training is performed, and all network parameters are changed. The number of training iterations is set to 100 times, the optimizer selects Adam, the kinetic energy is set to 0.9, the initial learning rate is set to 1e-4, and the input size of the training image is 512×512. The model structure and weight file obtained by training are saved in the logs folder. When the model is tested later, the weight file with the smallest loss is selected as the weight file for predicting the color change degree of the mushroom stick.

[0128] Evaluation indicators

[0129] Evaluation indicators are quantitative standards used to describe, measure and compare the performance of different models or algorithms. In machine learning, deep learning and other related fields, evaluation indicators are important tools for testing and evaluating algorithm performance. Different evaluation indicators reflect different perspectives for evaluating model performance, which can help researchers understand the performance of algorithms on different tasks from different perspectives and dimensions, thereby providing a strong basis and suggestions for model optimization and improvement.

[0130] The evaluation indicators used in this paper include mean intersection over union (mIoU), mean precision (mPrecision), mean recall (mRecall) and mean pixel accuracy (mPA). The formula is as follows, where i is the number of categories, Represents the positive sample with correct classification of category i; Represents the negative sample with correct classification of category i; Represents negative samples of category i that are misclassified as positive samples; Represents positive samples of category i that are misclassified as negative samples.

[0131] (1) Average intersection-union ratio

[0132] The average intersection-over-union ratio is an indicator used to evaluate the performance of image segmentation models. The intersection-over-union ratio is the ratio of the intersection and union of the model's prediction results for a certain category and the true value. It is used to measure the overlap between the prediction and the true annotation. The average intersection-over-union ratio is obtained by averaging the intersection-over-union ratios of all categories. The formula for calculating the average intersection-over-union ratio is shown below.

[0133] ;

[0134] (2) Average Precision

[0135] Average precision is an indicator used to evaluate the performance of classification models. Precision is the ratio of predicted values ​​to total predicted values. It is used to measure how many pixels in the model's prediction of this category are correct. The average precision is obtained by averaging the precision of all categories.

[0136] ;

[0137] (3) Average recall

[0138] The average recall rate is an indicator used to evaluate the performance of classification models. The recall rate calculates the ratio between the number of positive samples detected by the model and the number of positive samples that actually exist. It is used to measure the model's ability to correctly identify all instances of a certain category. The average recall rate is obtained by averaging the recall rates of all categories.

[0139] ;

[0140] (4) Average pixel accuracy

[0141] The average pixel accuracy is an indicator used to evaluate the performance of image segmentation models. The average pixel accuracy calculates the average proportion of pixels correctly predicted by the model to the total number of pixels. It is used to measure the proportion of pixels whose predicted labels match the true labels. The average pixel accuracy is obtained by averaging the recall rates of all categories.

[0142]

[0143] Analysis of ablation test results

[0144] In order to verify the effect of U-Net with different module pairs on segmentation performance, a segmentation performance ablation test was conducted on a self-made mushroom stick color change data set to study the mushroom stick color change degree recognition method based on deep learning. The ablation test uses the original U-Net model as the baseline model, and uses the VGG module, MSCA module, DC module, and SE module as components, and integrates the components into the U-Net model according to a certain combination rule. The input size of the training image is 512×512, and the ablation test results are shown in the following table.

[0145]

[0146] The results in the above table show that combining different modules into the U-Net model can significantly improve its segmentation performance. Compared with the U-Net model, the U-Net model with different components shows better segmentation performance. Using VGG16 as the U-Net backbone feature extraction network has obvious accuracy advantages over the original in the task of segmenting the color-changing image of shiitake mushroom sticks, with mIoU increased by 1.9% and mPrecision increased by 0.86%. Therefore, subsequent studies use VGG16 as the model feature extraction network for experiments. After the MSCA module is introduced in the last layer of the feature extraction network, compared with VGGUNet, mIoU is increased by 0.69% and mPrecision is increased by 0.43%, which effectively enhances the model's perception of local features and multi-scale context features. In the improvement of the decoder, the effects of the dual-scale jump connection and SE module on the model are compared. The results of Scheme 6 show that the proposed dual-scale jump connection can effectively solve the task of segmenting the color-changing image of shiitake mushroom sticks and improve the segmentation accuracy of the model. The results of Scheme 7 verify the effectiveness of integrating decoder feature information with the SE module. When the MSCA module and the SE module are added at the same time, all evaluation indicators are improved, and the segmentation performance of the model is optimal. Compared with the original model, MIoU is improved by 3.56%, precision is improved by 1.89%, recall is improved by 1.83%, and pixel accuracy is improved by 0.51%. In summary, the combination of different modules can effectively improve the performance of the U-Net model in the color change area segmentation task of shiitake mushroom sticks.

[0147] Comparative test results analysis

[0148] In order to demonstrate the segmentation effect of the model proposed in this paper, U-Net, UNet+++, PSPNet and DeepLabv3+ were selected for comparison with the DMS-UNet network. In order to ensure the comparability of the segmentation network, the data set used the exact same shiitake mushroom stick color change data set, and four images were randomly selected for display. The comparison results of the different network model tests are shown in Figure 10.

[0149] Fig.10The first two columns show the original images of the shiitake mushroom sticks during the color change period and the corresponding annotated images. The third column is the segmentation result of the PSPNet model, the fourth column is the segmentation result of the DeepLabv3+ model, the fifth column is the segmentation result of the U-Net model, the sixth column is the segmentation result of the UNet+++ model, and the seventh column is the segmentation result of the DMS-UNet model proposed in this paper. According to the results shown in the figure, we can observe the differences between different network models in the shiitake mushroom stick color change segmentation task. The segmentation result of the PSPNet model is poor, with obvious omissions and misclassifications. The segmentation results of the DeepLabv3+ model are improved compared to PSPNet, but there are still some omissions and misclassifications. Compared with the annotated images, the U-Net, UNet+++ and DMS-UNet models have almost no obvious omissions and misclassifications.

[0150] However, visual observation alone cannot fully understand the differences between models. For the mushroom stick color change segmentation task, the evaluation index can comprehensively and objectively measure the performance of different models. Therefore, the U-Net, UNet+++, PSPNet, DeepLabv3+ and DMS-UNet models were comprehensively evaluated on this task. The comparative test results are shown in the following table:

[0151]

[0152] According to the table above, among the classic semantic segmentation networks, the encoder-decoder of U-Net has the best segmentation effect for the color change task of shiitake mushroom sticks. Compared with the aggregation of context information in a single feature layer or the expansion of the receptive field by dilated convolution, the encoder-decoder structure can more effectively retain and utilize feature information and improve the performance of the segmentation model. The improved DMS-UNet model based on UNet showed the best performance in all four evaluation indicators. Its average precision reached 94.18% and the average pixel accuracy reached 97.99%, highlighting the model's outstanding ability in capturing details and accurately identifying color change areas. In contrast, PSPNet performed the worst, only achieving an average pixel accuracy of 93.32% and an average intersection-over-union ratio of 70.22%. Comprehensive visual observation and evaluation indicators can more comprehensively evaluate the performance of different models in the task. The conclusion clearly shows that the proposed DMS-UNet model is significantly effective in the color change degree recognition task of shiitake mushroom sticks compared with the other four segmentation models.

[0153] In summary, the proposed DMS-UNet model shows higher efficiency and accuracy in the task of identifying the color change degree of shiitake mushroom sticks, which provides a favorable basis for practical applications and proves that the MS-UNet model has achieved ideal results in processing the task of color change segmentation of shiitake mushroom sticks.

[0154] The technical solution is a system for identifying the color change degree of shiitake mushroom sticks based on the above method, and the system comprises:

[0155] - Front-end system: developed based on the Vue framework, used for user interface interaction;

[0156] - Backend system: developed based on SpringBoot framework, used for data processing and model calling;

[0157] - Database: used to store the historical records and results of the color change degree identification of shiitake mushroom sticks;

[0158] - Mobile application: Developed on the Android platform, the model is deployed using the NCNN framework to identify the degree of color change at the production site.

[0159] The mobile application uses a smartphone to capture images of shiitake mushroom sticks, uses a built-in DMS-UNet model to identify the degree of color change, calculates the percentage of color change area, and uploads the results to a database.

[0160] The identification process is as follows:

[0161] like Fig.11 As shown in the page for identifying the degree of color change of shiitake mushroom sticks, two image frames are displayed at the top: the left one is the original image frame to be uploaded, and the right one is the image frame to be identified. Both the original image and the identified image can be clicked to enlarge, so that users can easily view and compare the segmentation effect. When the user clicks the upload image button, the file selector is opened through the trigger function, and the user selects the image file to be uploaded through the file selector. The selected image file will be converted into a URL through data binding, and then updated to the original image for display. Next, use Axios to send a POST request to send the selected image file to the server. During the upload process, monitor the progress of the request, calculate the percentage of the upload progress based on the number of bytes uploaded, and update the display of the progress bar to prompt the user that the upload is in progress.

[0162] After receiving the request, the server first verifies whether the uploaded image type is in RGB format. If not, it is converted to RGB format and saved in the specified directory according to the naming rule of "current date_image name_timestamp" to ensure the uniqueness of the image name. Next, the shiitake mushroom stick color change image segmentation model is called to process the uploaded image, realize pixel-level segmentation and save the segmentation results. According to the segmentation image results, the total number of pixel values ​​in the color change area and the non-color change area is counted, and the proportion of the color change area is calculated, so as to obtain the quantitative result of the color change degree of the shiitake mushroom stick. The calculation formula for the color change degree of the shiitake mushroom stick is as follows.

[0163] ;

[0164] in, Represents the color change degree of shiitake mushroom sticks; Represents the total number of uncolored pixel values; Represents the total number of pixel values ​​that have changed color.

[0165] Finally, through data binding, the processed image data and the color change degree data of the mushroom sticks are returned to the interface. Fig.12 shown.

[0166] After the user performs the identification operation, he can choose whether to store the identification record in the MySQL database below the identification box. The user can choose to specify the number of the culture shed where the mushroom stick is located when saving the identification record. After clicking the Save button, the server will return the results including the address where the uploaded image is stored in the server, the address where the result image is stored in the server, the color change area ratio and the real-time time of the current identification, and the selected shed number and store them in the database. Click on the color change degree identification record management to enter the data display and management page. This page displays the data of all color change mushroom sticks saved by the user. The color change degree identification record management interface is shown in Figure 13. The current page will not only display the data recorded in the database, but also obtain the current mushroom stick culture date from the Shiitake mushroom stick production management system database according to the shed number, and obtain the environmental data within 10 seconds of the identification time from the real-time environmental monitoring data table according to the recorded identification time from the energy and environmental system database, including temperature, humidity, CO2 concentration and light intensity. According to the selection box at the top of the page, you can select the culture shed number and identification date for query. Click the query button and the system will Fig.13 The query results are displayed in the table. If you need to modify the color change information of a mushroom stick, you can click the edit button of the row in the list to enter the details page to modify the information. After the modification, the user will be prompted whether the modification is successful. Click the delete button in the color change information modification interface to delete the row of data and retrieve the data to display on the front-end page.

[0167] This technical solution can also be operated on the mobile terminal, and users can choose to upload images by taking photos or uploading pictures. After selecting real-time photography, call the device camera to take photos and obtain the captured image data. This function needs to ensure the clarity and accuracy of the captured images, and provide a real-time preview function so that users can confirm that the captured images meet the requirements before uploading them to improve the recognition accuracy. After choosing to upload the picture, open the local album, and the user can select the mushroom stick image that they want to identify and calculate, and the system obtains the URI of the user's selected picture. Image display is implemented using the ImageView control, and the user's uploaded images will be displayed in the image box on the interface, which is convenient for users to view and compare.

[0168] After uploading the image and clicking the detection button, the image data is passed to the backend to implement the model call and inference operation. The system first loads the pre-deployed shiitake mushroom stick color change image segmentation model into the memory for offline processing. Through the model's inference process, the color change area and the non-color change area in the shiitake mushroom stick image are identified and distinguished, and the data output by the model is analyzed to count the total number of pixel values ​​in the color change area and the non-color change area, and then the proportion of the color change area is calculated to obtain the quantitative result of the color change degree of the shiitake mushroom stick. After obtaining the quantitative result of the color change degree, the system updates the control on the interface that displays the recognition result and the color change degree quantitative result. The recognition effect of the APP is as follows: Fig.14 shown.

[0169] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for identifying the color change degree of shiitake mushroom sticks based on deep learning, characterized in that: The method comprises the following steps: - Data collection: Use a camera to collect image data of shiitake mushroom sticks during the color change period; - Data enhancement: Brightness enhancement and reduction processing of collected image data to expand the data set and simulate different lighting conditions; - Image annotation: A method combining manual annotation and HSV color space extraction is used to annotate images at the pixel level and generate an annotation dataset; - Model construction: Optimize and improve the U-Net model, including replacing the backbone network, adding multi-scale convolutional attention (MSCA), modifying the skip connection structure, adding channel attention (SE), and optimizing the loss function to build a DMS-UNet model for the segmentation of shiitake mushroom stick color change images; - Model training: Use the constructed labeled dataset to train and test the DMS-UNet model and store the optimal weight file; - Color change degree identification: Use the trained DMS-UNet model to perform pixel-level segmentation on the mushroom stick image, calculate the proportion of pixel values ​​of different categories based on the segmentation results, and then obtain the proportion of color change area and quantify the color change degree.

2. The method for identifying the color change degree of shiitake mushroom sticks based on deep learning according to claim 1, characterized in that: in, The mushroom stick images collected in the data collection step during the spawning period include mushroom stick images with a color change area accounting for 0-25%, mushroom stick images with a color change area accounting for 25-50%, mushroom stick images with a color change area accounting for 50-75%, and mushroom stick images with a color change area accounting for 75-100%.

3. The method for identifying the color change degree of shiitake mushroom sticks based on deep learning according to claim 1, characterized in that: in, The data enhancement step involves increasing and decreasing the brightness of each image by 30% to simulate different lighting conditions.

4. The method for identifying the color change degree of shiitake mushroom sticks based on deep learning according to claim 1, characterized in that: in, The image annotation step combines manual annotation with color extraction, specifically including: - Manual annotation: Perform USM sharpening on the original image to improve image quality and details; use the "magnetic lasso" tool to draw the selection of the mushroom sticks and the inoculation holes of the mushroom sticks, and fill them with different colors; - Color extraction: Convert the image to HSV color space, extract the uncolored area with a color range of (100, 43, 46) to (124,255, 255), and fill it with color. The remaining pixels are classified as the colored area.

5. The method for identifying the color change degree of shiitake mushroom sticks based on deep learning according to claim 1, characterized in that: in, The model building steps specifically include: - Use the first 13 layers of the VGG16 network as the feature extraction network of U-Net to extract features of different levels in the color change image of shiitake mushroom sticks; - The MSCA module is introduced at the bottom layer of the encoder to improve the segmentation performance of the model by aggregating local deep convolutions, multi-branch deep strip convolutions that capture multi-scale contextual information, and 1×1 convolutions that model the relationship between different channels; - Use SE (Squeeze and Excitation) module to enhance feature extraction effect; - Optimization of loss function: The combination of two loss functions is called FD Loss, which specifically includes the combination of Focal Loss and DiceLoss.

6. The method for identifying the color change degree of shiitake mushroom sticks based on deep learning according to claim 5, characterized in that: The calculation formula of Focal Loss function is as follows: ； in, represents the model prediction sample, The probability of belonging to a positive sample, Representation sample The weight of Represents the adjustment factor of sample difficulty. In this paper, γ is set to 2.0; The Dice loss function calculation formula is as follows: ； in, represents the intersection of sets X and Y, (X) represents the number of elements of the true label, (Y) represents the number of elements of the predicted mask, Represents the adjustment parameter to avoid the denominator being zero. Set to 1× ; The FD Loss loss function calculation formula is as follows: 。 7. The method for identifying the color change degree of shiitake mushroom sticks based on deep learning according to claim 1, characterized in that: in, The model training step includes: - The constructed dataset is randomly divided into training set, test set and validation set in a ratio of 7:2:1; - Use Adam optimizer for model training; - Through the cross-validation method, select the optimal training parameters and store the optimal weight file.

8. The method for identifying the color change degree of shiitake mushroom sticks based on deep learning according to claim 1, characterized in that: in, The calculation formula of the color change degree is as follows: ； in, Represents the color change degree of shiitake mushroom sticks; Represents the total number of uncolored pixel values; Represents the total number of pixel values ​​that have changed color.

9. A system for identifying the color change degree of shiitake mushroom sticks according to any one of claims 1 to 8, characterized in that: The system comprises: - Front-end system: developed based on the Vue framework, used for user interface interaction; - Backend system: developed based on SpringBoot framework, used for data processing and model calling; - Database: used to store the historical records and results of the color change degree identification of shiitake mushroom sticks; - Mobile application: Developed on the Android platform, the model is deployed using the NCNN framework to achieve color change degree recognition at the production site.

10. The system for identifying the color change degree of shiitake mushroom sticks according to claim 9, characterized in that: It also includes a web page, which uploads the image of shiitake mushroom sticks through the front-end system page. The back-end system is responsible for receiving the uploaded image data from the web page, and using the DMS-UNet model to perform image segmentation, calculate the color change area ratio, and upload the results to the database.

Images