A method for detecting surface flaws and grading quality of edible fungi based on semantic guidance

Abstract

The application discloses a kind of based on semantic guidance's edible mushroom surface flaw detection and quality grading method, comprising the following steps: by edible mushroom semantic encoder, semantic information is encoded into semantic condition vector;By semantic modulation double-path image encoder, receive the image to be detected and extract features, generate global semantic features and local structure features, and by semantic condition vector, local structure features are modulated, obtain semantic local structure features;By semantic enhancement bimodality attention fusion module, global semantic features and semantic local structure features are fused, and generate fusion features;Defect features are input into semantic guidance quality grading branch, and combined with semantic condition vector, dynamically set grading threshold, generate the quality grading score and quality grade of edible mushroom.The application can bind business semantics and visual features deeply, realize dynamic, scene-based flaw discrimination and quality grading.

Description

Technical Field

[0001] This invention relates to the field of agricultural visual inspection technology, and in particular to a semantically guided method for detecting surface defects and grading the quality of edible fungi. Background Technology

[0002] Surface defect detection in edible fungi is a core aspect of quality control in factory production. The key objective is to accurately identify defects on the fruiting body surface, such as mold infection, bacterial brown spots, mechanical damage, and morphological deformities. In recent years, the application of deep learning technology in agricultural visual inspection has deepened, and convolutional neural networks and visual Transformer models have made some progress in defect detection for single-variety edible fungi at fixed growth stages.

[0003] However, the unique nature of edible fungi detection presents two major challenges for existing methods: First, the dynamic nature of the discrimination criteria. Different fungal species exhibit significant differences in normal texture, and the morphology and color of the same species change regularly during the juvenile and mature stages. If the model lacks business semantic guidance, it is prone to misclassifying normal growth variations as abnormalities. Second, the subtlety of defect features. Defects such as early mold infection and slight browning show minimal differences in texture and color from normal fungi and are easily interfered with by culture medium residues and background debris. Models relying solely on visual features struggle to achieve accurate identification. Existing defect detection methods can be categorized into two types: zero-sample detection in industrial scenarios and supervised detection in agricultural scenarios. The former focuses on the scarcity of labeled data in industrial scenarios, achieving zero-sample transfer through visual-text cross-modal fusion, but it does not consider the dynamic discrimination criteria and quality grading requirements of agricultural organisms. The latter, while attempting to apply to edible fungi detection, often ignores key semantic information such as species and growth stage, resulting in insufficient model robustness and a failure to link defect detection with quality grading. While large-scale pre-trained visual models possess powerful general feature extraction capabilities, they lack specific adaptation to the edible fungi business scenario, making it difficult to directly meet the needs for accurate detection and defect determination. Summary of the Invention

[0004] The purpose of this invention is to provide a semantically guided method for detecting and grading the surface defects of edible fungi. This invention can deeply bind business semantics with visual features to achieve dynamic and scenario-based defect identification and quality grading.

[0005] The technical solution of this invention: A method for detecting and grading the surface defects of edible fungi based on semantic guidance, comprising the following steps: Step 1: Collect the image of the edible fungus to be detected and the semantic information corresponding to the image; input the image to be detected and the semantic information into a pre-trained detection grading network; the detection grading network includes an edible fungus semantic encoder, a semantic modulation dual-path image encoder, a semantic enhancement dual-modal attention fusion module, a semantic enhancement defect perception pooling module, and a semantic guidance quality grading branch; Step 2: Encode semantic information into semantic condition vectors using an edible fungus semantic encoder; receive the image to be detected and extract features using a semantic modulation dual-path image encoder to generate global semantic features and local structural features; and modulate the local structural features using the semantic condition vector to obtain semantic local structural features. Step 3: The global semantic features and the semantic local structural features are fused through the semantic enhancement bimodal attention fusion module to generate fused features; Step 4: Input the fused features into the semantic enhancement defect perception pooling module, use the anomaly mask generated by the fused features and optimized by the semantic condition vector as the prior to perform feature aggregation, output the aggregated defect features, and generate image-level defect anomaly scores at the same time. Step 5: Input the defect features into the semantic-guided quality grading branch, and dynamically set the grading threshold in combination with the semantic condition vector to generate the quality grading score and quality grade of edible fungi.

[0006] In the semantically guided method for detecting surface defects and grading the quality of edible fungi described above, step 1 includes the species type and growth days of the edible fungi.

[0007] In the aforementioned semantically guided method for detecting and grading the surface defects of edible fungi, step 1 involves training the detection and grading network using a multi-task loss function. This multi-task loss function includes segmentation loss, classification loss, and quality grading loss, expressed as: ; In the formula, , which is the first loss balancing weight; , which is the second loss balance weight; This is the third loss balance weight; To divide the loss, For classifying losses, Loss due to quality grading; This is the multi-task loss function.

[0008] In the aforementioned semantically guided method for detecting and grading the surface defects of edible fungi, step 2, which involves encoding semantic information into a semantic condition vector using an edible fungi semantic encoder, includes: In the edible fungi semantic encoder, the unique thermal encoding of the fungal species type and the normalized value of the growth days are combined into a semantic input vector, and the semantic input vector is encoded into a semantic condition vector through a two-layer perceptron.

[0009] In the aforementioned semantically guided method for detecting and grading the surface defects of edible fungi, step 2, which modulates local structural features through semantic condition vectors to obtain semantic local structural features, includes: The semantic conditional vector is mapped to feature modulation parameters, and the local structural features are then subjected to affine transformation using these parameters to generate semantic local structural features, expressed as follows: ; In the formula, These are semantic local structural features; This represents element-wise multiplication; It is the characteristic modulation scaling factor in the characteristic modulation parameters; It is the characteristic modulation offset factor in the characteristic modulation parameters; These are local structural features.

[0010] In the aforementioned semantically guided method for detecting and grading the surface defects of edible fungi, step 3 involves fusing global semantic features with semantically local structural features through a semantically enhanced bimodal attention fusion module to generate fused features, including: In the semantically enhanced bimodal attention fusion module, global semantic guidance weights are calculated based on global semantic features, and local self-similarity weights are calculated based on semantic local structural features; The semantic condition vector is mapped to semantic adaptation weights, and based on these weights, a fusion attention weight is constructed by dynamically adjusting the fusion ratio of local self-similarity weights and global semantic guidance weights. The expression is as follows: ; In the formula, It integrates attention weights; It is semantic adaptation weight; It is a local self-similarity weight; It is a global semantic guidance weight; The fused attention weights and semantic local structural features are multiplied together to generate fused features.

[0011] In the aforementioned semantically guided method for detecting and grading the surface defects of edible fungi, step 4 involves inputting the fused features into a semantically enhanced defect-aware pooling module. The module uses the anomaly mask generated from the fused features and optimized by the semantic conditional vector as a priori for feature aggregation, outputting the aggregated defect features, and simultaneously generating image-level defect anomaly scores, including: In the semantically enhanced defect-aware pooling module, the fused features are transposed and convolved, and then mapped to a predicted anomaly mask through the Sigmoid function. The semantic condition vector is transformed into semantic mask weights, and the predicted anomaly mask and semantic mask weights are weighted and fused into an optimized mask, expressed as follows: ; In the formula, For semantic mask weights; To optimize the mask; To predict anomaly masks; The fused features are parsed into basic defect features and contextual refinement features through parallel convolutional layers. Based on the optimization mask, the basic defect features and contextual refinement features are gated and fused and then input into a nonlinear activation function for nonlinear mapping to output defect features. By using fully connected layers and a sigmoid function, defect features are mapped to image-level defect anomaly scores.

[0012] The aforementioned semantically guided method for detecting and grading the surface defects of edible fungi, wherein the method involves gating and fusing basic defect features and context-refined features using an optimized mask, then inputting the fused features into a nonlinear activation function for nonlinear mapping, and outputting defect features, including: The optimized mask is downsampled to the same size as the fused features and normalized to generate a defect-aware attention weight map. The basic defect features and contextual refinement features are gated and fused using the defect-aware attention weight map as the gating coefficient, and the GELU function is used for nonlinear mapping to generate defect features.

[0013] In the aforementioned semantically guided method for detecting and grading the surface defects of edible fungi, step 5, which involves inputting defect features into the semantically guided quality grading branch and dynamically setting grading thresholds based on semantic condition vectors to generate quality grading scores and quality grades for edible fungi, includes: In the semantically guided quality grading branch, a quality grading feature vector is extracted from the defect features. The quality grading feature vector and the semantic condition vector are concatenated and mapped to a quality grading score. The grading threshold is dynamically set in combination with the semantic condition vector to grade the quality grading score and obtain the quality level.

[0014] A semantically guided system for detecting and grading the surface defects of edible fungi, comprising: The data acquisition module is used to acquire images of edible fungi to be detected and the semantic information corresponding to the images to be detected, and input the images to be detected and the semantic information into a pre-trained detection classification network. The feature extraction module is used to encode semantic information into semantic condition vectors through the edible fungi semantic encoder, receive the image to be detected and extract features through the semantic modulation dual-path image encoder, generate global semantic features and local structural features, and modulate the local structural features through the semantic condition vector to obtain semantic local structural features. The feature fusion module is used to fuse global semantic features with semantic local structural features through the semantically enhanced bimodal attention fusion module to generate fused features; The feature aggregation module is used to input the fused features into the semantic enhancement defect perception pooling module. It uses the anomaly mask generated by the fused features and optimized by the semantic condition vector as a priori to perform feature aggregation, output the aggregated defect features, and generate image-level defect anomaly scores. The quality grading module is used to input defect features into semantic guidance branches for quality grading, dynamically set grading thresholds in combination with semantic condition vectors, and generate quality grading scores and quality grades for edible fungi.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention transforms non-image semantics such as strain type and growth days into feature modulation parameters, and dynamically adjusts the visual feature distribution through affine transformation, so that the model can adapt to the discrimination criteria of different varieties and growth stages.

[0016] 2. This invention introduces semantic weight adjustment on the basis of traditional dual-modal fusion to enhance the feature response that adapts to the current scene and make up for the shortcomings of a single model in macroscopic morphology and microscopic texture representation.

[0017] 3. This invention uses the semantically guided anomaly mask as a priori, and combines it with the spatial distribution characteristics of defects in edible fungi to adaptively aggregate features, thereby improving the sensitivity of identifying minute defects.

[0018] 4. Based on the defect detection results and business semantics, this invention realizes automated defect degree scoring and quality grade determination of edible fungi, completing the closed-loop of the entire chain from defect detection to quality grading. Attached Figure Description

[0019] Figure 1 This is an overall flowchart of the method of the present invention; Figure 2 This is a diagram of the overall architecture of the detection hierarchical network; Figure 3 This is a flowchart of the semantically enhanced bimodal attention fusion module; Figure 4 This is the flowchart of the semantic enhancement defect awareness pooling module; Figure 5 This is the flowchart of the semantic adaptive modulation module. Detailed Implementation

[0020] The present invention will be further described below with reference to the accompanying drawings and embodiments, but this should not be construed as limiting the present invention.

[0021] Example: A semantically guided method for detecting surface defects and grading the quality of edible fungi, such as... Figure 1 As shown, it includes the following steps: Step 1: Collect the image of the edible fungus to be detected and the semantic information corresponding to the image; input the image to be detected and the semantic information into a pre-trained detection grading network; the detection grading network includes an edible fungus semantic encoder, a semantic modulation dual-path image encoder, a semantic enhancement dual-modal attention fusion module, a semantic enhancement defect perception pooling module, and a semantic guidance quality grading branch; Figure 2 The overall architecture of the detection and grading network is shown. The network receives edible fungi images and edible fungi metadata (i.e. semantic information, such as fungi species type and growth days) as input. It extracts global semantic features and local structural features through dual paths, modulates local structural features using semantic information, performs feature fusion through a semantically enhanced dual-modal attention fusion module, generates defect features through a semantically enhanced defect perception pooling module, and finally outputs defect detection results and quality grades.

[0022] Step 2: Encode semantic information into semantic condition vectors using an edible fungus semantic encoder; receive the image to be detected and extract features using a semantic modulation dual-path image encoder to generate global semantic features and local structural features; and modulate the local structural features using the semantic condition vector to obtain semantic local structural features. In this step, the semantic information includes the edible fungus species type and the number of days the edible fungus has grown. The edible fungus species type is one-hot encoded (e.g., enoki mushroom = [1,0,0], king oyster mushroom = [0,1,0]), and the number of days the edible fungus has grown is normalized to [0,1]. The one-hot encoded edible fungus species type and the normalized value of the number of days the edible fungus has grown are combined into a semantic input vector, expressed as: ; In the formula, It is a semantic input vector; It is a unique thermal code for edible fungi strains; This is the normalized value of the number of days for edible fungi to grow; The semantic input vector is processed through two layers of MLP (Multilayer Perceptron). Encoded as a semantic condition vector The expression is: ; In the formula, It is a semantic condition vector; It is a linear rectified function, a commonly used activation function in artificial neural networks; This represents the first layer of the MLP; This indicates the second-layer MLP; The image to be detected is received by a semantic modulation dual-path image encoder, and global semantic features are extracted through the (Vision Transformer) ViT path, and multi-scale local structural features are extracted through the (DINOv2 is a self-supervised visual model based on the Vision Transformer architecture) DINOv2 path. Feature modulation parameters, including feature modulation scaling factors, are generated using semantic conditional vectors. and characteristic modulation offset factor The expression is: ; In the formula, It is the characteristic modulation scaling factor; It is the characteristic modulation offset factor; This represents a multilayer perceptron used to generate conditional modulation parameters; Affine transformation is performed on the multi-scale local structural features extracted by DINOv2 using feature modulation parameters to achieve semantic modulation and obtain semantic structural features, expressed as follows: ; In the formula, These are semantic local structural features; This represents element-wise multiplication; Indicates local structural features; The overall process of this step can be summarized into a module called the semantic adaptive modulation module, such as... Figure 5 As shown, Figure 5 The workflow of the semantic adaptive modulation module is demonstrated. The module first encodes the input bacterial species type and growth days into a semantic condition vector through a multilayer perceptron. Then, the vector is used to generate feature modulation parameters through the perceptron. Finally, these parameters are used to perform affine transformation on the local structural features extracted by DINOv2 to achieve semantic-guided feature modulation.

[0023] Step 3: The global semantic features and the semantic local structural features are fused through the semantic enhancement bimodal attention fusion module to generate fused features; In this step, within the semantically enhanced bimodal attention fusion module, three types of attention weights are calculated: local self-similarity weights, etc. Global semantic guidance weight

[0024] and semantic adaptation weights The expression for the local self-similarity weight is: ; In the formula, It is a local self-similarity weight; It is a normalized exponential function; The square root of the feature dimension (scaling factor). This represents the query matrix in the self-attention mechanism. This represents the key matrix in the self-attention mechanism; Indicates matrix transpose; The expression for the global semantic guidance weight is: ; In the formula, It is a global semantic guidance weight; It is a global semantic feature; The expression for semantic adaptation weight is: ; In the formula, It is semantic adaptation weight; It is a multilayer perceptron used to generate semantic adaptation weights; The sigmoid function is a commonly used activation function.

[0025] The formulas for calculating the fusion attention weights and fusion features are as follows: ; ; In the formula, It integrates attention weights; Represents the value matrix in the self-attention mechanism; It is a feature of fusion.

[0026] Figure 3 The internal implementation process of the semantically enhanced bimodal attention fusion module is shown. This module receives global semantic features and semantically modulated local structural features, and calculates global semantic guidance weights and local self-similarity weights, respectively. At the same time, semantic condition vectors are used to calculate semantic adaptation weights to dynamically fuse the first two weights and generate the final fused attention weights to construct fused features.

[0027] Step 4: Input the fused features into the semantic enhancement defect perception pooling module, use the anomaly mask generated by the fused features and optimized by the semantic condition vector as the prior to perform feature aggregation, output the aggregated defect features, and generate image-level defect anomaly scores at the same time. In this step, the fused features obtained from cross-modal fusion are processed in the semantically enhanced defect-aware pooling module. Anomaly masks are generated by upsampling to the input image size via transposed convolution and then activating with a Sigmoid function. The expression is: ; The anomaly mask is a probability mask with the same size as the input image. Each pixel has a value in the range of [0,1], which represents the probability that the position belongs to the defect area. It directly corresponds to the annotation format of the real anomaly mask. Indicates transposed convolution; Using semantic condition vectors Through learnable MLP layers Generate semantic mask weights The expression is: ; In the formula, For semantic mask weights, the sensitivity is adjusted according to the fungal species and growth stage (e.g., the young mushroom stage). Higher values ​​enhance the response to minor defects; during the mature stage, the weight is appropriately reduced to avoid misclassifying normal scales and textures as defects. For learnable MLP layers; For predicting anomaly masks Adaptive smoothing and enhancement are performed to obtain an optimized mask. The expression is: ; Optimize the mask Downsampled to the fused feature size, and normalized to generate a defect-aware attention weight map. The expression is: ; In the formula, It is a defect perception attention weight map; It is an optimized mask after downsampling; This indicates that normalization is being performed; Fusion features Input two parallel 1×1 convolutional layers to generate basic defect features. Contextual refinement features The expression is: ; ; In the formula, Indicates the characteristics of basic defects; This indicates context-refined features; This represents a 1×1 convolutional layer; use Basic defect characteristics Contextual refinement features Gated fusion is performed to enhance the features of the defect area, suppress background interference, and generate defect features. The expression is: ; In the formula, The Gaussian error linear unit is a commonly used activation function in neural networks. The defect characteristics can be decomposed into two feature vectors: the quality grading feature vector and the quality grading feature vector. Feature vectors of defect types To determine the type and extent of defects in edible fungi. It also serves as a core input feature for quality grading.

[0028] Depend on Image-level defect anomaly scores (defect probabilities) are generated by applying a fully connected layer and Sigmoid activation.

[0029] Figure 4 The workflow of the semantically enhanced defect-aware pooling module is demonstrated. This module generates an initial anomaly mask by transposing convolutions and then optimizes it into a final mask guided by semantic information. On the other hand, it extracts basic defect features and contextual refinement features through parallel 1x1 convolutional layers. Finally, it uses the optimized mask to generate attention weights, performs gated fusion of the two features, and outputs a quality grade feature vector and a defect type feature vector.

[0030] Step 5: Input the defect features into the semantic-guided quality grading branch, and dynamically set the grading threshold in combination with the semantic condition vector to generate the quality grading score and quality grade of edible fungi.

[0031] In this step, within the semantically guided quality grading branch, the quality grading feature vector is... With semantic condition vector By splicing, the quality characteristics of the fusion are obtained. By using a fully connected layer and Sigmoid activation, The quality grading score is mapped to a continuous range of 0 to 100, with higher scores indicating better quality edible fungi. Grading thresholds are dynamically set based on the semantics of fungal strains and growth stages, dividing the quality grading score into four levels: excellent, good, acceptable, and unacceptable, thus obtaining the quality grade.

[0032] To evaluate the performance, experiments were conducted based on a detection-leveling network. The experiments were implemented using Python 3.9 and PyTorch 2.3.0 frameworks, with an NVIDIA A100 GPU (40G) and an Intel Xeon Gold 6330 CPU. The input image size was uniformly 384×384, and data augmentation strategies such as random flipping, brightness / contrast adjustment, and Gaussian blur were employed. The backbone network of the model used pre-trained ViT-B / 16 and DINOv2-ViT-small, trained using the AdamW optimizer (initial learning rate 1e-4, weight decay 1e-5), with a batch size of 16 and 30 training epochs. An early stopping strategy was adopted (training was stopped if there was no improvement in validation set performance for 5 consecutive epochs).

[0033] The dataset used in the experiment was constructed as follows: a self-built "Edible Fungus Full Growth Cycle Surface Defect Detection Dataset" was created, covering three mainstream fungal species (enoki mushroom, king oyster mushroom, and button mushroom) used in factory production, covering the entire growth cycle from 5 to 30 days (each 5 days as a stage), containing a total of 12,000 images (7,200 normal samples and 4,800 defective samples). The defective samples included four typical defects: Trichoderma contamination, bacterial brown spots, cap deformities, and mechanical damage. All images were labeled with semantic information such as fungal species type and growth days. Defective samples were additionally labeled with pixel-level anomaly masks, and all samples were simultaneously labeled with artificial defect types and quality grades. The dataset was divided into training, validation, and test sets in an 8:1:1 ratio to ensure consistent distribution of fungal species, growth stages, defect types, and quality grades across each subset.

[0034] The comparison method used in the experiment was as follows: 1. Baseline: ViT combined with DINOv2 dual-path fusion model (without semantic guidance); 2. Ours (w / o ES-CAM): Remove ES-CAM (Semantic Adaptive Modulation Module), and retain only SDMA (Semantic Enhancement Bimodal Attention Fusion Module) and S-DAP (Semantic Enhancement Defect-Aware Pooling Module). 3. Ours(w / o Stype): Retains the semantics of growth days, removes the semantics of strain type; 4. Ours(w / o Sdays): Retains the semantics of strain type, removes the semantics of growth days; 5. Ours (Full): A complete testing and grading network (including ES-CAM, SDMA, S-DAP, and quality grading branches).

[0035] To address the pixel imbalance problem (low percentage of defective pixels) in edible fungus defect detection, and to optimize both pixel-level segmentation and image-level classification tasks, a multi-task loss function is designed: ; In the formula, , which is the first loss balancing weight; , which is the second loss balance weight; This is the third loss balance weight; To divide the loss, For classifying losses, This is a loss due to quality grading.

[0036] Segmentation loss It can be represented as, ; In the formula, For segmentation loss; This is a true pixel-level anomaly mask; This refers to Focal Loss, which reduces the weight of normal pixels by focusing on hard-to-classify samples; Dice Loss, which improves the segmentation integrity of minor defects by optimizing the crossover-union ratio.

[0037] Classification loss It can be represented as, ; In the formula, The labels are based on the actual image level (0 = normal, 1 = defective). The defect probability output by the model (by (Generated via a fully connected layer and Sigmoid activation) For classification loss; Quality grading loss It can be represented as, ; In the formula, The actual quality scores are manually labeled. Predict quality scores for the model; To reflect the true quality level, To predict the quality level for the model; Loss due to quality grading; This is the mean square error loss; Cross-entropy loss; The experiment adopted mainstream indicators in the field of agricultural defect detection, including: image-level AUROC (area under the receiver operating characteristic curve) and pixel-level AUPRO (area under the curve of overlap per region). At the same time, the false alarm rate of cross-variety detection and the accuracy of early defect (minor mold spots and browning within 10 days of growth) identification were statistically analyzed. The quality grading evaluation indicators used included: quality grade accuracy and quality grade score mean square error (MSE).

[0038] The quantitative results are compared below: The quantitative performance comparisons of the various methods are shown in Table 1.

[0039]

[0040] Table 1 As can be seen from Table 1, 1. The baseline model has an image-level AUROC of 94.1% and a pixel-level AUPRO of 93.7%, but the cross-variety false alarm rate is 8.5%, the early defect identification accuracy is only 82.3%, and the quality grading accuracy is only 89.6%, proving that simple visual fusion is difficult to cope with the dynamic discrimination standards and quality grading requirements of edible fungi scenarios. 2. Ours (w / o ES-CAM) model performs slightly better than the baseline, with image-level AUROC improved to 94.5%, indicating that the optimization of SDMA and S-DAP has a certain enhancement effect on visual features, but the lack of semantic guidance still cannot solve the core pain point; 3. After removing the semantic information of fungal strain or growth days separately, the model performance decreased significantly, proving that both types of semantic information are indispensable for the detection and quality grading of edible fungi. 4. The full detection and classification network Ours (Full) achieved optimal performance, with image-level AUROC reaching 96.8% and pixel-level AUPRO reaching 96.2%. The cross-variety false alarm rate was reduced to 5.3%, the early defect identification accuracy was improved to 88.0%, the quality classification accuracy reached 95.7%, and the quality grade score MSE=2.13, which fully verified the synergistic effect of semantic adaptive modulation, cross-modal fusion, and quality classification branch.

[0041] The qualitative results are analyzed as follows: 1. For the sample of "King Oyster Mushroom with Culture Medium Residue in the Young Mushroom Stage", the Baseline model misclassified the culture medium residue as an anomaly (an anomaly mask covered the residue area), while the Ours (Full) model identified the residue at this stage as a normal phenomenon through the semantics of the growth days (day 8), and only generated a high-response mask for the slight mold spots on the edge of the cap. 2. For the sample of "mixture of filamentous structure and mold spots in mature enoki mushrooms", the Baseline model has difficulty distinguishing between normal filamentous structure and mold hyphae, and the mask response is chaotic. However, the Ours (Full) model strengthens the suppression of normal features of filamentous structure through species semantics (enoki mushrooms) and accurately locates the mold infection area. 3. For the "early browning defect of Agaricus bisporus" sample, the Baseline model did not generate a significant response due to the weak defect features. The Ours (Full) model improved its sensitivity to subtle texture changes by using semantic analysis of the growth stage (day 12), clearly outlining the browning area. To verify the effectiveness of each core module, ablation experiments were conducted, and the results are shown in Table 2.

[0042] Table 2 As can be seen from Table 2, 1. After removing the ES-CAM module, the model's image-level AUROC decreased by 2.3 percentage points, pixel-level AUPRO decreased by 2.2 percentage points, and quality classification accuracy decreased by 5.5 percentage points, proving that semantic adaptive modulation is the core of improving detection and quality classification performance; 2. After removing the SDMA module (and replacing it with simple feature concatenation), the model performance dropped significantly, with the image-level AUROC decreasing to 95.1%, indicating that semantically enhanced dual-modal fusion can effectively strengthen feature discriminative power; 3. After removing the S-DAP module (and replacing it with global average pooling), the accuracy of early defect identification decreased by 3.7 percentage points, and the accuracy of quality grading decreased simultaneously, proving that defect-aware pooling is crucial for the detection of minor defects and quality grading. 4. After removing the quality grading branch, only defect detection can be completed, and a closed-loop quality control system cannot be achieved.

[0043] To evaluate the robustness of the model in complex scenes, the effects of different interference factors (illuminance changes, background clutter, slight occlusion) on detection and quality grading performance were tested. The results are shown in Table 3.

[0044] Table 3 As can be seen from Table 3, 1. In scenarios with varying lighting conditions, Ours (Full)'s image-level AUROC decreased by only 1.2 percentage points, and its quality grading accuracy decreased by only 0.8%, significantly outperforming the Baseline. 2. When background clutter is present, Ours (Full) has a false alarm rate of 6.1%, and its quality grading stability is much higher than that of Baseline; 3. In slightly occluded scenes, Ours (Full) still maintains an image-level AUROC of 95.3% and a quality rating accuracy of 94.5%, proving that semantic guidance can effectively improve the model's resistance to interference factors.

[0045] In summary, this invention breaks through the limitations of traditional visual inspection models that "have fixed discrimination criteria and lack quality grading capabilities." Through the deep integration of semantics and vision, the model is equipped with the ability to adaptively detect defects and grade quality for different edible fungi varieties and growth stages, providing a new approach to agricultural biological inspection that integrates "business semantics, visual features, and quality grading."

[0046] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A semantically guided method for detecting surface defects and grading the quality of edible fungi, characterized in that, Includes the following steps: Step 1: Collect the image of the edible fungus to be detected and the semantic information corresponding to the image; input the image to be detected and the semantic information into a pre-trained detection grading network; the detection grading network includes an edible fungus semantic encoder, a semantic modulation dual-path image encoder, a semantic enhancement dual-modal attention fusion module, a semantic enhancement defect perception pooling module, and a semantic guidance quality grading branch; Step 2: Encode semantic information into semantic condition vectors using an edible fungus semantic encoder; receive the image to be detected and extract features using a semantic modulation dual-path image encoder to generate global semantic features and local structural features; and modulate the local structural features using the semantic condition vector to obtain semantic local structural features. Step 3: The global semantic features and the semantic local structural features are fused through the semantic enhancement bimodal attention fusion module to generate fused features; Step 4: Input the fused features into the semantic enhancement defect perception pooling module, use the anomaly mask generated by the fused features and optimized by the semantic condition vector as the prior to perform feature aggregation, output the aggregated defect features, and generate image-level defect anomaly scores at the same time. Step 5: Input the defect features into the semantic-guided quality grading branch, and dynamically set the grading threshold in combination with the semantic condition vector to generate the quality grading score and quality grade of edible fungi.

2. The method for detecting and grading the surface defects of edible fungi based on semantic guidance according to claim 1, characterized in that, In step 1, the semantic information includes the species type and growth days of the edible fungus.

3. The method for detecting and grading the surface defects of edible fungi based on semantic guidance according to claim 1, characterized in that, In step 1, the detection and grading network is trained using a multi-task loss function, which includes segmentation loss, classification loss, and quality grading loss, and its expression is: ； In the formula, , which is the first loss balancing weight; , which is the second loss balance weight; This is the third loss balance weight; To divide the loss, For classifying losses, Loss due to quality grading; This is the multi-task loss function.

4. The method for detecting and grading the surface defects of edible fungi based on semantic guidance according to claim 2, characterized in that, In step 2, encoding semantic information into a semantic condition vector using an edible fungus semantic encoder includes: In the edible fungi semantic encoder, the unique thermal encoding of the fungal species type and the normalized value of the growth days are combined into a semantic input vector, and the semantic input vector is encoded into a semantic condition vector through a two-layer perceptron.

5. The method for detecting and grading the surface defects of edible fungi based on semantic guidance according to claim 1, characterized in that, In step 2, the modulation of local structural features using semantic conditional vectors to obtain semantic local structural features includes: The semantic conditional vector is mapped to feature modulation parameters, and the local structural features are then subjected to affine transformation using these parameters to generate semantic local structural features, expressed as follows: ； In the formula, These are semantic local structural features; This represents element-wise multiplication; It is the characteristic modulation scaling factor in the characteristic modulation parameters; It is the characteristic modulation offset factor in the characteristic modulation parameters; These are local structural features.

6. The method for detecting and grading the surface defects of edible fungi based on semantic guidance according to claim 1, characterized in that, In step 3, the semantically enhanced bimodal attention fusion module fuses global semantic features with semantic local structural features to generate fused features, including: In the semantically enhanced bimodal attention fusion module, global semantic guidance weights are calculated based on global semantic features, and local self-similarity weights are calculated based on semantic local structural features; The semantic condition vector is mapped to semantic adaptation weights, and based on these weights, a fusion attention weight is constructed by dynamically adjusting the fusion ratio of local self-similarity weights and global semantic guidance weights. The expression is as follows: ； In the formula, It integrates attention weights; It is semantic adaptation weight; It is a local self-similarity weight; It is a global semantic guidance weight; The fused attention weights and semantic local structural features are multiplied together to generate fused features.

7. The method for detecting and grading the surface defects of edible fungi based on semantic guidance according to claim 1, characterized in that, In step 4, the fused features are input into the semantically enhanced defect-aware pooling module. The module uses the anomaly mask generated from the fused features and optimized by the semantic conditional vector as a priori for feature aggregation, outputting the aggregated defect features and simultaneously generating an image-level defect anomaly score, including: In the semantically enhanced defect-aware pooling module, the fused features are transposed and convolved, and then mapped to a predicted anomaly mask through the Sigmoid function. The semantic condition vector is transformed into semantic mask weights, and the predicted anomaly mask and semantic mask weights are weighted and fused into an optimized mask, expressed as follows: ； In the formula, For semantic mask weights; To optimize the mask; To predict anomaly masks; The fused features are parsed into basic defect features and contextual refinement features through parallel convolutional layers. Based on the optimization mask, the basic defect features and contextual refinement features are gated and fused and then input into a nonlinear activation function for nonlinear mapping to output defect features. By using fully connected layers and a sigmoid function, defect features are mapped to image-level defect anomaly scores.

8. The method for detecting and grading the surface defects of edible fungi based on semantic guidance according to claim 7, characterized in that, The method, based on an optimized mask, involves gating and fusing basic defect features and context-refined features, then inputting the fused features into a nonlinear activation function for nonlinear mapping, outputting defect features, including: The optimized mask is downsampled to the same size as the fused features and normalized to generate a defect-aware attention weight map. The basic defect features and contextual refinement features are gated and fused using the defect-aware attention weight map as the gating coefficient, and the GELU function is used for nonlinear mapping to generate defect features.

9. The method for detecting and grading the surface defects of edible fungi based on semantic guidance according to claim 8, characterized in that, In step 5, the step of inputting defect features into a semantically guided quality grading branch, dynamically setting grading thresholds in conjunction with semantic condition vectors, and generating quality grading scores and quality grades for edible fungi includes: In the semantically guided quality grading branch, a quality grading feature vector is extracted from the defect features. The quality grading feature vector and the semantic condition vector are concatenated and mapped to a quality grading score. The grading threshold is dynamically set in combination with the semantic condition vector to grade the quality grading score and obtain the quality level.

10. A semantically guided system for detecting and grading the surface defects of edible fungi, based on the semantically guided method for detecting and grading the surface defects of edible fungi according to any one of claims 1 to 9, characterized in that, include: The data acquisition module is used to acquire images of edible fungi to be detected and the semantic information corresponding to the images to be detected, and input the images to be detected and the semantic information into a pre-trained detection classification network. The feature extraction module is used to encode semantic information into semantic condition vectors through the edible fungi semantic encoder, receive the image to be detected and extract features through the semantic modulation dual-path image encoder, generate global semantic features and local structural features, and modulate the local structural features through the semantic condition vector to obtain semantic local structural features. The feature fusion module is used to fuse global semantic features with semantic local structural features through the semantically enhanced bimodal attention fusion module to generate fused features; The feature aggregation module is used to input the fused features into the semantic enhancement defect perception pooling module. It uses the anomaly mask generated by the fused features and optimized by the semantic condition vector as a priori to perform feature aggregation, output the aggregated defect features, and generate image-level defect anomaly scores. The quality grading module is used to input defect features into semantic guidance branches for quality grading, dynamically set grading thresholds in combination with semantic condition vectors, and generate quality grading scores and quality grades for edible fungi.

Images