Edible mushroom disease identification method based on multi-modal time sequence perception and hierarchical closed-loop management and control method

Abstract

The invention discloses an edible mushroom disease identification method based on multi-modal time sequence perception and a hierarchical closed-loop management and control method, belongs to the technical field of agricultural intellectualization and computer vision, and aims to recover edible mushroom image details by using an improved defogging algorithm, introduce a pileus biological brightness model, realize transmissivity lower limit protection of a pileus area, and improve the detection accuracy. Defogging and biological feature retention are both considered; scab growth dynamics and static impurity removal are combined, and precise recognition and background noise suppression of tiny early-stage diseases are achieved; environmental microclimate data and visual features are deeply fused, so that the disease recognition precision in a complex environment is remarkably improved; an automatic closed-loop system from sensing to execution is constructed on the basis, disease diffusion is effectively restrained through graded physical intervention, and the method has remarkable production and application value.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of agricultural intelligence and computer vision technology, and particularly relates to a method for identifying diseases of edible fungi based on multi-modal time-series perception and a hierarchical closed-loop control method. BACKGROUND

[0002] With the rapid popularization of factory cultivation technology of edible fungi, an automatic monitoring system based on machine vision has become a key means to ensure yield and quality. However, the current disease detection method faces many challenges in dealing with the production environment of high humidity, weak light and complex substrate background, especially the "atomization effect" produced by the humidifier in the workshop, and the existing dark channel dehazing method will misjudge the white cap as fog, which has the feature of easy confusion. In addition, the current method mostly relies on single-frame static image for identification, ignoring the dynamic growth characteristics of diseases over time and the decisive influence of environmental microclimate parameters on disease outbreak. Nowadays, visual interference in the growth environment of edible fungi often appears in the form of "same image different disease" or "background confusion", which seriously interferes with accurate detection. Some visually highly similar targets often need to be combined with time-series changes and environmental parameters to accurately distinguish, for example, "substrate particles" and "early brown spots" are difficult to distinguish on a single frame image, but the former is physically stationary and the latter has growth. Moreover, pure visual detection cannot perceive the temperature and humidity conditions that induce diseases, resulting in failure to provide early warning before disease outbreak.

[0003] Disease detection technology has gradually developed from early traditional image processing methods (such as threshold segmentation and edge detection) to deep learning methods. The emergence of convolutional neural networks (CNN) has greatly promoted the progress of the field of agricultural image analysis, and models based on ResNet, YOLO and other architectures have become the mainstream tools for disease identification. However, when facing the high-humidity fog environment unique to edible fungi factories, existing models often fail due to low image contrast and blurred features. The current dehazing algorithm (such as the traditional dark channel prior (DCP)) does not consider the biological characteristics of the white cap, which is easy to misjudge the cap as fog for excessive dehazing, resulting in image distortion.

[0004] Currently, most academic research and practical applications still focus on single-frame static disease detection or time-series diagnosis based on hyperspectral imaging. Specifically, existing dynamic monitoring methods struggle to effectively eliminate static noise, while static image discrimination fails to fully utilize temporal information. Some studies utilize time-series images to track lesion changes to improve recognition reliability, but current technologies primarily focus on continuous trend analysis and have not yet achieved a comprehensive processing mechanism that simultaneously eliminates static background noise based on dynamic lesion behavior discrimination. Furthermore, while existing multimodal agricultural monitoring methods attempt to integrate image and sensor data, they employ only simple feature fusion at the feature fusion level, failing to adequately consider the deep logical connections between environmental parameters and visual features, and failing to effectively utilize environmental data as prior constraints to eliminate visual ambiguity. They often neglect the temporal behavior of lesions and the elimination of static noise, making it difficult to guarantee recognition accuracy in complex environments. Moreover, these methods are often limited to passive alarm output, lacking automated closed-loop control from perception to execution, and are unable to provide physical intervention in the early stages of disease outbreaks.

[0005] Therefore, there is an urgent need for an intelligent management method for edible fungi diseases that can effectively cope with high humidity and low light environments, resist background interference, and integrate environmental parameters. Summary of the Invention

[0006] To address the technical problems existing in the prior art, this invention provides a method for identifying edible fungi diseases based on multimodal temporal perception and a hierarchical closed-loop management method. It utilizes an improved dehazing algorithm to restore details in edible fungi images, introduces a cap bioluminescence model to protect the lower limit of transmittance in the cap region, thus balancing dehazing with the preservation of biological features. For the first time, it combines the dynamics of lesion growth with the removal of static impurities, achieving accurate identification of minor early-stage diseases and suppression of background noise. Furthermore, it deeply integrates environmental microclimate data with visual features, significantly improving the accuracy of disease identification in complex environments. Based on this, it constructs an automated closed-loop system from perception to execution, effectively curbing disease spread through hierarchical physical intervention, demonstrating significant value for production applications.

[0007] The technical solution adopted in this invention is as follows:

[0008] A method for identifying edible fungi diseases based on multimodal temporal sensing includes the following steps:

[0009] S1. Real-time acquisition of images of edible fungi and their surrounding environmental parameters, including temperature, humidity, light intensity, and carbon dioxide concentration;

[0010] S2. Adaptive enhancement preprocessing is performed on the edible fungi images, specifically including:

[0011] S2.1 Based on the atmospheric scattering physical model, a dehazing algorithm based on dark channel prior is used to process edible fungi images. In the processing, the coarse transmittance is first optimized by guided filtering to obtain the fine transmittance. Then, a lower limit threshold of transmittance is introduced into the restoration formula to protect the white cap features and prevent noise amplification caused by excessive dehazing, thereby obtaining a haze-free image.

[0012] S2.2 Convert the haze-free image from RGB space to LAB space, extract the L channel and divide it into several sub-blocks, and obtain the local brightness image by histogram equalization.

[0013] S2.3. Based on adaptive Gamma enhancement technology, nonlinear mapping is performed on the local brightness image to enhance the brightness of the L channel. The enhanced L channel is combined with the original A and B channels and converted back to RGB space to obtain the enhanced image of edible fungi.

[0014] S3. Artificially label disease spots on enhanced images of edible fungi under different environmental parameters to construct a training dataset;

[0015] S4. Using ResNet as the backbone network, a multi-scale feature extraction and fusion model for fungal cells is obtained by introducing Feature Pyramid (FPN) and Spatial Attention (SAM) and fine-tuning based on the training dataset. This model is used to output a weighted feature map of the enhanced image of edible fungi, which includes anchor boxes and visual feature representations of candidate lesions.

[0016] S5. Based on the sliding time window, obtain the weighted feature maps of the same monitoring point at N consecutive time points. Using the multi-frame image registration mechanism, map the weighted feature maps of the first N-1 time points to the coordinate system of the weighted feature map of the last time point to obtain N frames of images with pixel-level overlap of the same monitoring point; where N is a positive integer.

[0017] S6. In N frames of images, extract the area and centroid coordinates of candidate lesions at each time point, and calculate the cumulative change in area from the first time point to the Nth time point. and cumulative change in centroid displacement ,Will and Simultaneously, candidate lesions smaller than the corresponding threshold are removed, resulting in N frames of images after static impurity removal;

[0018] S7. In the N frames of images after static impurity removal, extract the area change rate, color change rate and roundness of the remaining candidate lesions at each time point to construct the growth feature vector at each time point, and thus form a time series growth feature containing N time points.

[0019] S8. Train a Long Short-Term Memory (LSTM) network based on the training dataset. Specifically, use the time-series growth features corresponding to the training dataset as input and manually labeled lesion markers as targets to obtain the trained LSM network. This network serves as a dynamic discrimination model for active diseases, outputting the confidence level that the remaining candidate lesion belongs to an "active disease". ;

[0020] Will The remaining candidate lesions that exceed the preset confidence threshold are identified as edible fungi diseases.

[0021] Furthermore, the Gamma coefficient of the adaptive Gamma enhancement technique described in S2.3 The calculation formula is:

[0022]

[0023] In the formula, These are the two-dimensional coordinates of a pixel; The base Gamma value; This is a local brightness image; For reference brightness; To prevent division by zero of constants.

[0024] Furthermore, the feature pyramid described in S4 is used for top-down cross-scale feature fusion. Specifically, it first extracts four stages of basic feature layers from the input edible fungus enhancement image from the bottom up. ; after that conduct Convolutional dimensionality reduction yields a fused feature layer. ; After 2x upsampling, and Element-wise addition yields the fused feature layer. ; After 2x upsampling, and Element-wise addition yields the fused feature layer. ; After 2x upsampling, and Element-wise addition yields the fused feature layer. This allows for the fusion of high-level semantic features and low-level detailed features.

[0025] Furthermore, the spatial attention mechanism described in S4 is used to suppress background noise such as damp soil and reflective humidifier tubes. Specifically, a spatial attention module is embedded after each fusion feature layer. First, max pooling and average pooling are performed on the fusion feature layer along the channel dimension to generate two descriptors. Then, the two descriptors are concatenated and a spatial attention weight map is generated by a convolutional layer and a sigmoid activation function. The weight map is then multiplied element-wise with the input feature map to obtain a weighted feature map.

[0026] Furthermore, an adaptive anchor frame adjustment strategy is introduced into the S4 multi-scale feature extraction and fusion model for fungal cells. Specifically, K-means clustering analysis is performed on the lesion markers in the training dataset to obtain the lesion geometric morphology clustering results. Based on the geometric morphology of various lesions, corresponding anchor frame scales and aspect ratios are designed for different fusion feature layers, so that the multi-scale feature extraction and fusion model for fungal cells can adaptively adjust the anchor frame scale and aspect ratio according to the morphology and size of edible fungi, thereby improving the detection capability of small lesions and deformed fungal cells.

[0027] Furthermore, in S5, the homography matrix is ​​calculated using ORB (Oriented FAST and Rotated BRIEF) feature point matching and RANSAC (Random Sample Consensus) algorithm to realize spatial perspective transformation of multi-frame images.

[0028] Furthermore, after S8 completes the identification of edible fungi diseases, it also includes a disease type identification process, specifically:

[0029] S9.1. Process real-time environmental parameters using the Min-Max normalization method to construct an environmental feature vector containing temperature, humidity, and carbon dioxide concentration;

[0030] S9.2. For enhanced images of edible fungi that have been identified as diseases of edible fungi under different environmental parameters, the disease type is manually labeled, and a new training dataset is constructed together with the corresponding environmental feature vector.

[0031] S9.3. After the multi-scale feature extraction and fusion model of fungal cells obtained in S4, a fully connected layer is connected, and fine-tuned based on the new training dataset to obtain a multimodal fusion model. Among them, the visual feature representation of edible fungi diseases extracted by the multi-scale feature extraction and fusion model of fungal cells is concatenated with the corresponding environmental feature vector through feature channels to obtain a joint feature vector, which is input to the fully connected layer for disease classification and decision, so as to realize the identification of disease type.

[0032] This invention also proposes a graded closed-loop management method for edible fungi diseases, which uses the disease type and lesion area obtained from the multimodal temporal perception-based edible fungi disease identification method. and confidence level A three-level response mechanism is set up, specifically as follows:

[0033] (1) Level 1 Response (Suspected or Early Stage): Targeting In the critical fuzzy range, or For edible fungi areas smaller than the preset area threshold, the "adaptive high-frequency monitoring and time-series verification" mode is activated. Specifically, the image acquisition frequency of the edible fungi area is increased to obtain high-frequency time-series images. By calculating the dynamic growth rate of lesions, the mode can distinguish between real diseases with biological growth characteristics and static false alarms caused by changes in light or the attachment of impurities, thereby completing secondary confirmation without disturbing management personnel.

[0034] (2) Level II Response (Confirmed or Moderate Stage): Targeting In edible mushroom areas where the disease type exceeds the preset confidence limit and is a specific fungal species that is sensitive to the environment, the "environmental adverse antagonism and macro-control" mode is activated. Specifically, based on the biological characteristics of the disease type, "environmental adverse antagonism" instructions are generated to macro-control environmental parameters, thereby destroying the suitable microenvironment for the corresponding pathogen, inhibiting spore germination and mycelial spread from the source, and striving for a treatment window period.

[0035] (3) Level 3 response (severe or outbreak stage): targeting For edible fungi areas exceeding the preset area warning threshold, or where the disease type is a highly infectious and malignant disease, the "visual servo positioning and precise physical intervention" mode is activated. Specifically, the coordinate transformation algorithm is executed using the intrinsic and extrinsic parameter matrix of a binocular vision or depth camera to map the image pixel coordinates of the edible fungi area to Cartesian coordinates. Visual servo technology is then used to move a robotic arm or sliding rail device equipped with a spray nozzle to the Cartesian coordinate position to perform a fixed-point cleaning operation. This involves micro-spraying of the diseased area to achieve precise disease control.

[0036] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0037] 1. This invention proposes a method for identifying edible fungi diseases based on multimodal temporal perception, which can overcome the environmental limitations of high humidity, low light and complex background in industrialized cultivation of edible fungi, and significantly improve the accuracy and anti-interference ability of disease detection. It not only solves the technical problems of false alarms of static impurities and overexposure of white caps, but also combines the dynamic growth behavior of lesions with the removal of static impurities for the first time, and introduces environmental prior knowledge and temporal dynamics analysis to achieve accurate qualitative and effective identification of early minor diseases.

[0038] 2. Specifically, this invention first utilizes an improved dark channel prior algorithm and an adaptive Gamma correction strategy to solve the data degradation problem caused by high humidity fog occlusion and uneven illumination; then, it uses a deep feature pyramid and spatial attention mechanism to accurately extract multi-scale disease features from complex matrix backgrounds; next, it uses temporal difference and long short-term memory networks to construct a disease growth dynamics discrimination model, effectively eliminating static impurities; finally, it uses environmental sensor data to construct a multimodal fusion model, using microclimate parameters such as temperature and humidity as prior constraints to assist visual classification, thereby achieving high-precision identification of edible fungi diseases.

[0039] 3. Based on the results of edible fungi disease identification, this invention also constructs an automated closed-loop system from perception to execution, realizing automated closed-loop control from high-frequency verification and environmental adverse antagonism to visual servo-guided precise removal. This system can not only dynamically monitor and identify diseases in the early stage, but also actively intervene in the environment and carry out targeted physical treatment in the early stage of disease outbreak, significantly improving the efficiency of prevention and control. It reflects the original design of integrated detection and treatment, and has important production application value. Attached Figure Description

[0040] Figure 1 This is a flowchart of the anti-interference image acquisition and adaptive enhancement preprocessing in Example 1;

[0041] Figure 2 This is a flowchart of the multi-scale feature extraction and fusion process for bacterial cells in Example 1;

[0042] Figure 3 This is a flowchart of the disease authenticity determination based on time-series growth dynamics in Example 1;

[0043] Figure 4 This is a schematic diagram of multimodal feature enhancement and accurate classification that integrates environmental microclimate parameters in Example 1;

[0044] Figure 5 This is a flowchart of graded decision support and automated environmental control based on disease severity in Example 1. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0046] Example 1

[0047] This embodiment proposes a method for identifying edible fungi diseases based on multimodal temporal sensing, including the following steps:

[0048] S1. Anti-interference image acquisition and adaptive enhancement preprocessing, the process is as follows: Figure 1 As shown, it specifically includes:

[0049] S1.1 Construct a multi-source data acquisition terminal that includes a high-resolution CMOS image sensor and an environmental sensor array, aiming to solve the data degradation problem caused by high humidity water mist shading, uneven lighting on shelves and sensor noise in the industrial cultivation of edible fungi.

[0050] The high-resolution CMOS image sensor in the multi-source data acquisition terminal acquires images of edible fungi in real time, while the environmental sensor array collects environmental parameters of the edible fungi in real time, including temperature, humidity, light intensity, and carbon dioxide concentration.

[0051] S1.2. In response to the problem of extremely low image contrast caused by the "fogging effect" generated by the humidifier in the workshop, this embodiment adopts an adaptive defogging algorithm based on dark channel prior improvement. Combined with the biological characteristics of edible fungi (especially white caps), it corrects the defect of the traditional algorithm that misjudges white objects as high-gloss fog.

[0052] Specifically, based on the atmospheric scattering physical model, a dehazing algorithm based on dark channel priors is used to process edible fungi images. During the processing, a radius r=40 and a regularization parameter are first used. The guided filtering optimizes the coarse transmittance to obtain the fine transmittance. Then, a lower limit threshold of transmittance is introduced into the restoration formula to protect the white cap feature and prevent the transmittance estimation of the white cap area from approaching 0 due to the high brightness characteristics, which would generate a lot of noise during restoration, thus obtaining a fog-free image.

[0053] The restoration formula is:

[0054] ;

[0055] In the formula, Represents pixel coordinates; This represents a haze-free image, i.e., surface radiation. Image representing edible fungi; Indicates atmospheric light; This represents the lower limit threshold of transmittance; in this embodiment, the value is 0.1. The transmittance estimated a priori through the dark channel is expressed by the following formula:

[0056] ;

[0057] In the formula, This represents the dehazing retention factor. In this embodiment, the value is 0.95, which aims to retain a very small amount of fog as a depth cue, preventing the image from losing depth of field or appearing unnatural due to excessive dehazing. This represents the filtering window; in this embodiment, the size is set to... The pixel size is sufficient to effectively count dark channel features while avoiding halo effects at the edge of the cap due to an excessively large window. This represents atmospheric light in channels r, g, or b; This represents an image of edible fungi in channel r, g, or b.

[0058] S1.3 To address the issue of gills and matrix confusion caused by shelf shadows, a contrast-limited adaptive histogram equalization strategy is adopted. Specifically, the haze-free image is converted from RGB space to LAB space, the L channel is extracted and converted into LAB space. Sub-blocks, after histogram equalization, yield local brightness images; where, the clipping threshold... Set to 3.0 to limit the magnification of the background matrix grayscale and prevent excessive enhancement of background noise.

[0059] S1.4. Based on adaptive Gamma enhancement technology, nonlinear mapping is performed on the local brightness image. Specifically, the design... For a local brightness image after histogram equalization, the Gamma enhancement formula is: In the formula, for Two-dimensional coordinates of the middle pixel, To enhance the brightness of the L channel, The illumination-adaptive Gamma coefficient is defined as:

[0060] ;

[0061] in, The base Gamma value is taken as 0.6~0.8 in this embodiment. For reference brightness; To prevent the use of small constants for division by zero, this embodiment takes... ;

[0062] This adaptive Gamma enhancement technology can automatically adjust the enhancement intensity according to the local brightness, which can both stretch the details in the dark areas of the gills and compress the bright areas to prevent the white cap from being overexposed.

[0063] After that, Combined with the original AB channels, the image is converted back to RGB space to obtain an enhanced image of edible fungi.

[0064] S2. Artificially label disease spots on enhanced images of edible fungi under different environmental parameters to construct a training dataset.

[0065] S3. Extraction and fusion of multi-scale features of bacterial cells, the workflow is as follows: Figure 2 As shown, it specifically includes:

[0066] To address the challenges of large-scale abnormal morphological variations (including macroscopic deformities and microscopic diseases) during the growth of edible fungi, and the difficulty of confusing the texture of cultivation substrates (sawdust, cottonseed hulls) with tiny lesions, this embodiment uses a deep residual network ResNet (specifically ResNet-50) as the backbone network. By introducing feature pyramids and spatial attention mechanisms, and fine-tuning based on the training dataset, a multi-scale feature extraction and fusion model for fungal cells is obtained. This model is used to output a weighted feature map of the enhanced image of edible fungi, which includes anchor boxes and visual feature representations of candidate lesions, in order to achieve accurate capture of multi-scale targets and suppression of background noise.

[0067] The feature pyramid is used for top-down cross-scale feature fusion to resolve the contradiction between the loss of details in deep networks and the lack of semantics in shallow networks. Specifically, it first extracts four stages of basic feature layers from the input edible fungus enhancement image from the bottom up. Among them, the lower-level basic feature layer and It preserves high-resolution geometric details such as plaque edges and gill textures, and is mainly used for the detection of small lesions. By integrating local structural and textural information, it can effectively characterize the morphology of medium-scale lesions and local bacterial cell distortions. It contains highly abstract semantic category features for overall malformation detection; then... conduct Convolutional dimensionality reduction yields a fused feature layer. ; Double upsampling is performed using nearest neighbor interpolation, compared with the channel-adjusted... Element-wise addition is performed to obtain the fused feature layer. ; Double upsampling is performed using nearest neighbor interpolation, compared with the channel-adjusted... Element-wise addition is performed to obtain the fused feature layer. ; Double upsampling is performed using nearest neighbor interpolation, compared with the channel-adjusted... Element-wise addition is performed to obtain the fused feature layer. This allows for the fusion of high-level semantic features and low-level detail features, significantly improving the detection rate of tiny mold spots and effectively reducing the probability of misclassifying matrix particles as lesions.

[0068] The spatial attention mechanism is used to suppress background noise such as damp soil and reflective humidifier tubes. Specifically, a spatial attention module is embedded after each fused feature layer. This module enhances the feature responses of the cap and stipe regions by generating a weight mask of the same size as the feature map. First, max pooling (to highlight highly significant lesion features) and average pooling (to aggregate background information) are performed along the channel dimension of the fused feature layer to generate two two-dimensional descriptors. Then, the two descriptors are concatenated, and a spatial attention weight map is generated by passing it through a convolutional layer and a sigmoid activation function. This weight map is then multiplied element-wise with the input feature map to obtain a weighted feature map. In this process, the convolutional kernel size is set to [size missing]. Choosing this large-size convolutional kernel instead of a general one This is to increase the receptive field, ensure that the attention mechanism can cover a larger cap area and establish long-distance dependencies between pixels, thereby more accurately separating irregular cell outlines from the complex matrix background.

[0069] To address the biological morphology of edible fungi, this embodiment introduces an adaptive anchor frame adjustment strategy into the multi-scale feature extraction and fusion model of fungal cells. Specifically, K-means clustering analysis is performed on lesion markers in the training dataset to obtain clustering results of lesion geometric morphology. Based on the geometric morphology of various lesions, corresponding anchor frame scales and aspect ratios are designed for different fusion feature layers, enabling the multi-scale feature extraction and fusion model of fungal cells to adaptively adjust the anchor frame scale and aspect ratio according to the morphology and size of edible fungi, thereby improving the detection capability of small lesions and deformed fungal cells.

[0070] The anchor frame dimensions are set as follows:

[0071] In this embodiment, due to the ultra-low basic feature layer Constructed fusion feature layer The resolution is too high; there is no need to set an anchor frame.

[0072] In the low-basic feature layer Constructed high-resolution fusion feature layer Above, a small-scale anchor frame is set, with an area of ​​approximately 16×16 pixels, to detect early mold spots with a diameter of 1~3 mm;

[0073] In the basic feature layer Constructed medium-resolution fusion feature layer Above, a medium-scale anchor frame is set, with an area of ​​approximately 32×32 pixels, to detect medium-scale lesions with a diameter of 3~8 mm;

[0074] In the high basic feature layer Constructed low-resolution fusion feature layer Above, set a large-scale anchor frame with an area of ​​approximately 64×64 pixels to select the entire mature mushroom or areas with obvious morphological distortion.

[0075] The aspect ratios are set as follows: there are three types: 1:1, 1:2 and 2:1. 1:1 corresponds to normal round caps, while 1:2 and 2:1 correspond to slender, deformed mushrooms (such as long-stemmed mushrooms) or clustered forms.

[0076] With the above settings, the bacterial multi-scale feature extraction and fusion model can output a weighted feature map with "multi-scale perception" and "anti-background interference" capabilities, providing high-purity input for subsequent classification and regression.

[0077] S4. Disease authenticity determination based on time-series growth dynamics, the process is as follows: Figure 3 As shown, it specifically includes:

[0078] S4.1. Based on a sliding time window, obtain the same monitoring point at N=3 consecutive time points (i.e., , and The weighted feature map of the image was obtained with a sampling interval of 0.5 h. To eliminate pixel displacement caused by mechanical vibration or slight deformation of the shelving during shooting, a multi-frame image registration mechanism was used. The homography matrix was calculated using ORB feature point matching and the RANSAC algorithm to achieve spatial perspective transformation of the multi-frame images, thereby converting the time frame into a single frame. , Weighted feature map mapped to time step In the weighted feature map coordinate system, N frames of images with pixel-level overlap at the same monitoring point are obtained;

[0079] S4.2. Utilizing the essential difference between the biological activity (growing and spreading over time) of edible fungal diseases and the physically static state of background impurities (soil, substrate particles), this embodiment removes static impurities. Specifically, in N frames of images, the area and centroid coordinates of candidate lesions at each time point are extracted, and the area of ​​the candidate lesion is calculated. to Cumulative change in area over a time period and cumulative change in centroid displacement ,Will and At the same time less than the corresponding threshold (i.e. Less than the area change threshold, and Candidate lesions smaller than the centroid displacement threshold are identified as non-biologically active "soil" or "mechanical damage," marked as background noise, and directly removed to reduce the false alarm rate, resulting in N frames of images after static impurity removal. In this embodiment, the area change threshold is set to 5×5 pixels, and the centroid displacement threshold is set to 2 pixels.

[0080] S4.3. In the N frames of images after static impurity removal, extract the area change rate of the remaining candidate lesions at each time point. Color change rate and roundness To construct growth feature vectors at each time point ,in Pick , or This leads to the formation of a time series growth feature containing N time points. .

[0081] S4.4. Train the Long Short-Term Memory network based on the training dataset, specifically by growing the features corresponding to the time series data of the training dataset. Using manually labeled lesions as input and manually marked lesions as targets, a trained Long Short-Term Memory (LSTM) network is obtained, which serves as a dynamic discrimination model for active diseases. This model outputs the confidence level that the remaining candidate lesions belong to "active diseases". In the formula, express The corresponding hidden state, the formula for calculating the hidden state is: , Pick , or ; and These are all parameters of the fully connected layer;

[0082] Will The remaining candidate lesions that exceed the preset confidence threshold are identified as edible fungi diseases; in this embodiment, the preset confidence threshold is set to 0.85.

[0083] Through S4, this embodiment achieves a leap from "static morphology recognition" to "dynamic behavior recognition", accurately filtering out complex matrix interference and ensuring high specificity of disease detection.

[0084] S5. Multimodal feature enhancement and accurate classification integrating environmental microclimate parameters, the process is as follows: Figure 4 As shown, this study aims to address the problem of "same image, different disease" or "same disease, different image" when relying solely on visual images for disease classification. Given the extremely high sensitivity of edible fungi growth to microclimate conditions such as temperature and humidity, environmental parameters are often the decisive triggers for disease outbreaks. Therefore, non-visual environmental perception data is introduced, using environmental features as key prior constraints to assist visual features in more accurate logical reasoning. Specifically, this includes:

[0085] S5.1 To eliminate the distribution difference between visual features (usually dimensionless activation values) and physical environmental parameters (with specific dimensions), the real-time environmental parameters are processed using the Min-Max normalization method to construct an environmental feature vector containing temperature, humidity, and carbon dioxide concentration. ;

[0086] S5.2 For enhanced images of edible fungi identified as diseases under different environmental parameters, manually label the disease type and its corresponding environmental feature vector. Together, they constructed a new training dataset;

[0087] S5.3. After the multi-scale feature extraction and fusion model of fungal cells obtained in S3, a fully connected layer is connected, and fine-tuned based on the new training dataset to obtain a multimodal fusion model; among which, the visual feature representation of edible fungal diseases extracted by the multi-scale feature extraction and fusion model of fungal cells is... With the corresponding environmental feature vector By concatenating the feature channels, a joint feature vector is obtained. The data is input to a fully connected layer for disease classification and decision-making, thereby identifying the type of disease. It not only encodes the phenotypic information such as the texture and shape of the spots, but also embeds the microclimate state in which the spots are located, providing a full-dimensional input space for the classifier.

[0088] Taking the easily confused "normal white mycelium" and "early white fuzz of green mold" as examples, although they are visually very similar, green mold tends to break out in high temperature and high humidity environments. If the environmental parameters show "high temperature and high humidity," the multimodal fusion model will significantly increase the activation weight of the "green mold" category and suppress the probability of "normal mycelium"; conversely, if the environment is within the standard range suitable for mycelial growth, the multimodal fusion model tends to classify the white fuzz as a normal growth phenomenon.

[0089] Through this multimodal fusion, this embodiment endows the recognition system with the "environmental perception and reasoning" capabilities similar to those of agricultural experts. It effectively utilizes prior environmental knowledge to eliminate visual ambiguity and significantly improves the system's classification robustness and accuracy in complex and ever-changing mushroom house environments.

[0090] The disease type and lesion area obtained from the multimodal time-series perception-based edible fungus disease identification method are as follows: and confidence level This embodiment also proposes a graded closed-loop management method for edible fungi diseases, constructing an automated closed-loop system from "disease perception" to "actual intervention," as follows: Figure 5As shown, by setting up a three-level response mechanism, environmental control equipment and actuators are directly linked, and the optimal handling strategy is automatically matched according to the threat level, so as to achieve intelligent proactive defense and precise governance.

[0091] The three-level response mechanism is as follows:

[0092] (1) Level 1 Response (Suspected or Early Stage): Targeting In the critical fuzzy interval (e.g.) ),or Area smaller than a preset area threshold (e.g., 0.5 cm) 2 In the edible fungus area, the "Adaptive High-Frequency Monitoring and Time-Series Verification" mode is activated. Specifically, not only is the edible fungus area highlighted in yellow on the digital twin management interface, but an adaptive frequency conversion sampling mechanism is also triggered to increase the image acquisition frequency of the edible fungus area, such as automatically increasing it from the usual 30 minutes / time to 5 minutes / time, to obtain high-frequency time-series images. The dynamic growth rate of lesions is then calculated. This allows for the differentiation between real diseases with biological growth characteristics and static false alarms caused by changes in light or the attachment of impurities, thus enabling secondary confirmation without disturbing management personnel.

[0093] (2) Level II Response (Confirmed or Moderate Stage): Targeting In edible mushroom areas where the confidence level exceeds the preset limit (e.g., 90%) and the disease type is a specific fungal species sensitive to the environment (e.g., hygrophilous molds, thermophilic bacteria), the "environmental adverse antagonism and macro-control" mode is activated. Specifically, based on the biological characteristics of the disease type, an "environmental adverse antagonism" instruction is generated and sent directly to the programmable logic controller to macro-control environmental parameters, thereby destroying the suitable microenvironment for the corresponding pathogen, inhibiting spore germination and mycelial spread from the source, and striving for a window of opportunity for treatment. Taking the outbreak of green mold as an example, given that the pathogen reproduces exponentially in high temperature and high humidity environments, the system will immediately implement a "humidification and antibacterial" strategy: the instruction reduces the output power of the ultrasonic humidifier and simultaneously turns on the powerful exhaust fan to forcibly and rapidly reduce the relative humidity in the workshop to a safe threshold (e.g., below 85%).

[0094] (3) Level 3 response (severe or outbreak stage): targeting For edible mushroom areas exceeding the preset area warning threshold, or where the disease type is highly infectious and malignant such as *Neprion*, simple environmental adjustments are insufficient to control the outbreak. In such cases, a "visual servo positioning and precise physical intervention" mode is activated. Specifically, using the intrinsic and extrinsic parameter matrices of binocular vision or depth cameras, a coordinate transformation algorithm is executed to map the image pixel coordinates of the edible mushroom area to Cartesian coordinates. Visual servo technology then moves a robotic arm or sliding rail device equipped with a spray nozzle to this Cartesian coordinate position to perform targeted cleaning. This involves micro-spraying (such as with biological agents, fungicides, or lime water) on the affected area, achieving precise disease control. This "surgical" precision strike avoids the pesticide residue damage caused by traditional large-area spraying throughout the greenhouse, while also effectively preventing secondary aerosol transmission of pathogenic spores within the workshop via airflow.

[0095] Through the above-mentioned hierarchical decision-making strategy, this embodiment realizes the fully automated closed-loop management of the edible fungi production process, which includes "early detection (first-level time-series verification), early inhibition (second-level environmental antagonism), and early treatment (third-level precise elimination)".

[0096] It should be noted that this is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for edible mushroom disease identification based on multi-modal temporal perception, characterized in that, The method comprises the following steps: S1, real-time acquisition of edible mushroom image and environment parameters, the environment parameters include temperature, humidity, illumination and carbon dioxide concentration; S2, adaptive enhancement preprocessing of edible mushroom image, specifically including: S2.1, based on the atmospheric scattering physical model, using the dark channel prior based defogging algorithm to process the edible mushroom image, and in the processing process, first using guided filter to optimize the coarse transmittance, obtaining the fine transmittance, then introducing the transmittance lower threshold in the restoration formula, and then obtaining the haze-free image; S2.2, convert the haze-free image from RGB space to LAB space, extract L channel and divide into several sub-blocks, and get local brightness image after histogram equalization processing; S2.3, based on adaptive Gamma enhancement technology, nonlinear mapping is carried out on the local brightness image, the enhanced L channel is combined with the original A and B channels, and it is converted back to RGB space to obtain the edible mushroom enhanced image; S3, the disease spot mark of edible mushroom enhanced image under different environment parameters is manually labeled, and the training data set is obtained; S4, taking the deep residual network ResNet as the backbone network, introducing the feature pyramid and spatial attention mechanism, and based on the training data set fine tuning, obtaining the fungus multi-scale feature extraction and fusion model, which is used to output the weighted feature map of the edible mushroom enhanced image, including the anchor box of the candidate disease spot and the visual feature representation; S5, based on the sliding time window, the weighted feature map of the same monitoring point at the continuous N time points is obtained, the weighted feature map of the previous N-1 time points is mapped into the coordinate system of the weighted feature map of the last time point by using the multi-frame image registration mechanism, and N frame images with pixel level overlap of the same monitoring point are obtained; wherein N is a positive integer; S6、In the N frames of images, the area and the centroid coordinates of the candidate lesion at each time point are extracted, and the accumulated change amount of the area from the first time point to the Nth time point is calculated and the accumulated change amount of the centroid displacement , are calculated and The candidate lesion whose accumulated change amount is less than the corresponding threshold value is removed, and N frames of images after removal of static impurities are obtained. S7, in the N frame images after static impurity elimination, the area change rate, color change rate and circularity of the remaining candidate disease spots at each time point are extracted to construct the growth feature vector at each time point, and then the time sequence growth feature containing N time points is formed; S8. Training the long short-term memory network based on the training data set, specifically taking the time series growth features corresponding to the training data set as input, and taking the artificially labeled disease spot marks as target, to obtain the trained long short-term memory network as the active disease dynamic discrimination model, which is used to output the confidence degree of the remaining candidate disease spot belonging to "active disease" ; the remaining candidate disease spots exceeding the preset confidence threshold are identified as edible mushroom diseases.

2. The mushroom disease recognition method based on multi-modal timing perception according to claim 1, characterized in that, Gamma coefficients of the adaptive Gamma enhancement technique described in S2.3 The calculation formula is: ； wherein is a two-dimensional coordinate of a pixel; is a base Gamma value; is a local luminance image; is a reference luminance; is a constant to prevent division by zero.

3. The method according to claim 2, wherein, The feature pyramid in S4 is used for top-down cross-scale feature fusion, specifically, the input edible mushroom enhanced image is first extracted from bottom to top to obtain four stages of basic feature layers ; then convolution dimension reduction is performed to obtain a fusion feature layer ; ; After 2 times of upsampling, element-by-element addition is performed with to obtain a fusion feature layer ; After 2 times of upsampling, element-by-element addition is performed with to obtain a fusion feature layer ; After 2 times of upsampling, element-by-element addition is performed with to obtain a fusion feature layer ; and fusion of high-level semantic features and low-level detail features is realized.

4. The mushroom disease recognition method based on multi-modal timing perception according to claim 2, characterized in that, The spatial attention mechanism in S4 is used to suppress background noise, which is specifically embedded in the spatial attention module after each fusion feature layer. First, the fusion feature layer is respectively subjected to maximum pooling and average pooling along the channel dimension to generate two descriptors. Then, the two descriptors are spliced, and the spatial attention weight map is generated through the convolution layer and the Sigmoid activation function. The weighted feature map is obtained by element-wise multiplication with the input feature map.

5. The method according to claim 2, wherein, In the fungus multi-scale feature extraction and fusion model of S4, an anchor box adaptive adjustment strategy is introduced. Specifically, K-means clustering analysis is performed on the disease spot labels in the training data set to obtain the geometric morphological clustering results of the disease spots. According to the geometric morphologies of different disease spots, the corresponding anchor box scales and aspect ratios are designed for different fusion feature layers, so that the fungus multi-scale feature extraction and fusion model can adaptively adjust the anchor box scales and aspect ratios according to the morphology and size of the edible mushrooms, thereby improving the detection capability of small disease spots and deformed fungus bodies.

6. The mushroom disease recognition method based on multi-modal timing perception according to claim 2, characterized in that, In S5, the ORB feature point matching and RANSAC algorithm are used to calculate the homography matrix to realize the spatial perspective transformation of multiple frames of images.

7. The method according to claim 2, wherein, After the mushroom disease recognition is completed in S8, a disease type recognition process is further included, specifically: S9.1, processing real-time environmental parameters by Min-Max normalization method to construct an environmental feature vector containing temperature, humidity and carbon dioxide concentration; S9.2, manually labeling the disease type of the mushroom enhanced image recognized as a mushroom disease under different environmental parameters, and the corresponding environmental feature vector, to jointly construct a new training data set; S9.3, connecting a fully connected layer after the mushroom multi-scale feature extraction and fusion model obtained in S4, and fine-tuning based on the new training data set to obtain a multi-modal fusion model; wherein the visual feature representation of the mushroom disease extracted by the mushroom multi-scale feature extraction and fusion model is spliced with the corresponding environmental feature vector to obtain a joint feature vector, which is input into the fully connected layer for disease classification decision, realizing the recognition of disease type.

8. A method for closed-loop management of edible mushroom disease classification, characterized in that, The disease type, the disease spot area and the confidence degree obtained by the disease identification method based on multi-modal timing perception according to any one of claims 1-7 A three-level response mechanism is set, specifically: (1) First response: for For the critical fuzzy interval or the edible mushroom area with the lesion area less than the preset area threshold, the "adaptive high-frequency monitoring and time sequence verification" mode is started. Specifically, the image acquisition frequency of the edible mushroom area is increased to obtain high-frequency time sequence images. By calculating the dynamic growth rate of the lesion, the real disease with biological growth characteristics and the static false alarm caused by light changes and impurity attachment are distinguished, so as to complete the secondary confirmation without disturbing the management personnel. (2) Secondary response: for When the confidence exceeds the upper limit of the preset confidence and the disease type is a specific strain of edible fungi that is sensitive to the environment, the "environmental reverse antagonism and macro-control" mode is started. Specifically, according to the biological characteristics of the disease type, the "environmental reverse antagonism" instruction is generated to macro-control the environmental parameters, thereby destroying the suitable microenvironment of the corresponding pathogen, inhibiting spore germination and mycelium spread from the source, and striving for the management window period. (3) Three-level response: for the mushroom area with disease spot area exceeding the preset area warning threshold or high infectious and malignant disease type, the "visual servo positioning and precise physical intervention" mode is started. Specifically, the coordinate transformation algorithm is used to map the image pixel coordinates of the mushroom area to the rectangular coordinates by using the internal and external parameter matrices of the binocular vision or depth camera, and the mechanical arm or sliding rail device carrying the spray head is moved to the rectangular coordinate position by visual servo technology, and the point cleaning operation is performed, and the disease area is sprayed in a small amount, realizing the precise attack on the disease.

Images