Tobacco leaf grading and defect detection method based on inter-layer adapter and diffusion model

Abstract

The application discloses a tobacco grading and defect detection method based on an inter-layer adapter and a diffusion model, and a software and hardware cooperative pipeline system containing visual acquisition, pretreatment, double model reasoning and result output is built. A light inter-layer adapter ILA embedded in a SwinTransformer and MobileNetV2 double branch model FDAILA is proposed at the grading end, attention collapse can be inhibited by only increasing 0.04% of the parameter amount, and the grading accuracy is 96.33%; an ADA-Net unsupervised diffusion network is constructed at the defect detection end, the optimal noise step is dynamically predicted through a multi-scale MLP, three-dimensional differences of pixels, features and structures are fused to realize pixel-level defect segmentation, and the AUROC is 96.1%. The method does not need defect pixel labeling, the single inference time is less than 1s, the automatic grading, defect quantification and availability accounting of tobacco can be simultaneously completed, the labor cost and raw material waste are greatly reduced, and the method is suitable for industrialization landing of the tobacco purchasing pipeline.

Description

Technical Field

[0001] The present invention relates to the technical field of vision-based intelligent detection of tobacco leaves, and in particular to a tobacco leaf grading and defect detection method based on inter-layer adapters and diffusion models. Background Art

[0002] Tobacco is an important strategic cash crop in China, and tobacco leaf quality control is a core link in the high-quality development of the tobacco industry chain. In the tobacco leaf purchasing process, tobacco leaf grading is the core basis for implementing national grading standards and realizing pricing based on quality. Defect detection can accurately quantify the proportion of tobacco leaf defects, calculate available value, and reduce raw material waste.

[0003] At present, in the core link of tobacco leaf purchasing, that is, tobacco leaf grading and defect detection, it still mainly relies on manual experience. The manual mode has three inherent bottlenecks: first, it is highly subjective and easily affected by factors such as environment, experience, and working hours; second, it is inefficient, professional graders have high mobility and long training cycles, making it difficult to meet the high-intensity work requirements during the centralized purchasing period; third, the ability to quantify defects is insufficient, and it can only simply determine qualified / unqualified, resulting in a large number of tobacco leaves with only local defects being discarded as a whole, causing serious raw material waste.

[0004] In terms of intelligent detection technology, existing research has obvious limitations: (1) Pure CNN architecture grading algorithms: such as ResNet, MobileNet, etc. Although they can better extract local detail features of tobacco leaves, they are limited by the local receptive field of convolution, and have insufficient modeling ability for global semantic features such as tobacco leaf shape and overall color distribution. They have weak discrimination ability for adjacent-grade tobacco leaves with minimal visual differences, and are prone to misjudgment between adjacent grades.

[0005] (2) Pure Transformer architecture grading algorithms: such as SwinTransformer, ViT, etc. Although they can realize global long-distance dependency modeling, they are very prone to attention collapse as the number of network layers deepens. The self-attention weight distribution tends to be uniform, and they cannot form stable focusing on key fine-grained regions such as tobacco leaf veins and color transition. Moreover, they have high computational complexity and slow inference speed, which is difficult to meet the real-time requirements of industry.

[0006] (3) Supervised defect detection algorithms: require a large number of defect samples with pixel-level annotation, while tobacco leaf defect samples are scarce, the annotation cost is extremely high, and annotation consistency is difficult to guarantee, which greatly limits the practical application of such methods.

[0007] (4) Unsupervised defect detection algorithms: Feature embedding-based methods (such as PatchCore) are prone to misjudging normal natural variations in tobacco leaves as abnormalities; Reconstruction-based diffusion model methods (such as DiffusionAD) generally adopt a fixed noise step design, which has poor adaptability to subtle color defects such as cyan and mixed colors and large-scale structural defects such as damage, and cannot simultaneously achieve full destruction of defect areas and high-fidelity reconstruction of normal areas. Summary of the Invention

[0008] The present invention proposes a method for tobacco leaf grading and defect detection based on interlayer adapter and diffusion model, which can solve at least one of the technical problems in the background art, realize the automation, standardization and intelligent processing of the entire process of tobacco leaf acquisition, significantly improve the grading accuracy and defect detection precision while ensuring industrial real-time performance, and reduce economic losses and raw material waste.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: A method for tobacco leaf grading and defect detection based on interlayer adapter and diffusion model, comprising the following steps performed by computer equipment: S1. Image Acquisition: A visual acquisition subsystem integrating an industrial camera, a strip light source, and a through-beam photoelectric sensor to automatically acquire raw images of single tobacco leaves from the production line under stable, shaded lighting conditions. S2. Differentiated preprocessing: For graded tasks, image cropping and normalization are performed to remove the background, while for defect detection tasks, the full resolution is preserved and only the background is removed. S3. Tobacco Leaf Grading Inference: The preprocessed image is fed into the dual-branch FDAILA grading model with an embedded interlayer adapter ILA, and the prediction results of 6 grades of tobacco leaves are output. The ILA module is embedded in each stage of the SwinTransformer and injects the bottleneck structure of channel compression-depth separable convolution-channel recovery into the local inductive bias of the CNN to alleviate deep attention collapse. S4. Branch Judgment: If the output is a qualified tobacco leaf grade, directly output the grading result; if it is determined to be an unqualified tobacco leaf, execute the defect detection step. S5. Unsupervised Defect Detection: Input the image of unqualified tobacco leaves into the ADA-Net diffusion detection model. After multi-scale feature extraction, MLP network adaptive prediction of the optimal noise step number, adaptive noise injection, one-step U-Net denoising and reconstruction, the multi-dimensional differences of pixel, feature, and structural similarity are fused to generate an anomaly score map, complete pixel-level defect segmentation and calculate the proportion of tobacco leaf defects and availability. S6. Results Visualization and Export: Synchronously output graded and defect quantification data and generate inspection reports.

[0010] In another aspect, the present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method described above.

[0011] In another aspect, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method described above.

[0012] As can be seen from the above technical solutions, the tobacco leaf grading and defect detection method based on interlayer adapter and diffusion model of the present invention solves two core technical problems in the field of vision-based intelligent tobacco leaf detection: one is the attention collapse problem that is common in deep Transformer in fine-grained classification tasks, and the other is the fundamental contradiction that the diffusion model cannot simultaneously adapt to multiple types of defects in unsupervised defect detection with a fixed number of noise steps.

[0013] This invention proposes two original technical solutions: (1) Lightweight Interlayer Adapter (ILA): With only a 0.04% increase in parameters, it injects CNN local inductive bias into the Transformer, which fundamentally prevents attention collapse and enables the hierarchical accuracy of the dual-branch hybrid model to exceed 96%; (2) Adaptive noise step prediction mechanism: Through the MLP network driven by multi-scale features, the optimal noise intensity dynamic matching of different defect types is realized, so that the detection accuracy of the diffusion model for color defects and structural defects can simultaneously meet the industrial-grade requirements.

[0014] This invention constructs an end-to-end hardware and software collaborative system that can complete training without any defect pixel annotation. While ensuring industrial real-time performance (single-sheet inference time <1s), it achieves a classification accuracy of 96.33% and a defect detection pixel-level AUROC of 96.1%, which is significantly better than all existing mainstream algorithms. It provides a complete technical solution for the automation of the entire tobacco leaf purchasing process.

[0015] In summary, this invention addresses the core pain points of existing technologies and achieves the following beneficial effects: (1) Significantly improved grading accuracy: The proposed FDILA grading model effectively alleviates the attention collapse problem of the Transformer branch by embedding a lightweight ILA module. The grading accuracy on the self-built single tobacco leaf grading dataset reaches 96.33%, which is 11.06% higher than the baseline FDANet model. It significantly reduces misjudgment of adjacent grades and significantly reduces economic losses caused by grading errors.

[0016] (2) Excellent defect detection performance: The proposed ADA-Net defect detection model solves the problem of poor adaptability of fixed noise steps to multiple types of defects through an adaptive noise step mechanism. On the self-built tobacco defect detection dataset, the pixel-level AUROC reaches 96.1% and AUPRO reaches 92.3%, and the detection performance is significantly better than the mainstream unsupervised algorithms in the industry.

[0017] (3) Get rid of the dependence on defect labeling: The ADA-Net model adopts an unsupervised learning paradigm, which only requires normal tobacco leaf samples to complete the training. It does not require a large number of manually labeled defect samples, which greatly reduces the threshold for implementation of the algorithm and the data cost.

[0018] (4) Achieve full-process automation: An end-to-end hardware and software collaborative system integrating image acquisition, preprocessing, grading, defect detection, result visualization and export has been built, which has greatly improved acquisition efficiency and reduced labor costs.

[0019] (5) Significantly improve raw material utilization: Through pixel-level defect segmentation and utilization calculation, the usable part of unqualified tobacco leaves can be accurately identified, avoiding the problem of discarding unqualified tobacco leaves as a whole under the traditional manual mode, and reducing raw material waste from the source. Attached Figure Description

[0020] Figure 1 This is a diagram of the ADA-Net architecture of the present invention; Figure 2 This is the FDANet hierarchical obfuscation matrix of the present invention; Figure 3 This is the MFASTG hierarchical confusion matrix of the present invention; Figure 4 This is the FDAILA hierarchical confusion matrix of the present invention; Figure 5 This is a feature heatmap of the present invention on a single tobacco leaf dataset; Figure 6 Comparison of noise addition and reconstruction effects on tobacco leaf samples of the present invention (I); Figure 7 Comparison of noise addition and reconstruction effects on tobacco leaf samples of the present invention (II); Figure 8 This is a schematic diagram of the automatic grading system of the present invention; Figure 9 This is a schematic diagram of the automatic defect detection of the present invention; Figure 10 This is a flowchart of the automatic grading and defect detection method of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0022] like Figure 1 As shown in the figure, the tobacco leaf grading and defect detection method based on interlayer adapter and diffusion model described in this embodiment performs the following steps using a computer device: 1. System Overall Architecture The intelligent tobacco leaf grading and defect detection system of the present invention comprises four core parts: an industrial vision acquisition subsystem, a data preprocessing subsystem, a core algorithm inference subsystem, and a result visualization and export subsystem. Each subsystem achieves data interaction and collaborative work through standardized interfaces.

[0023] 1.1 Industrial Vision Acquisition Subsystem The industrial vision acquisition subsystem is deeply integrated with the tobacco leaf purchasing production line, adopting a dual-station deployment scheme, including: (1) Hardware equipment: Hikvision MV-CS050-10GCCMOS industrial area array camera (resolution 2448×2048), Hikvision FA 8mm lens, two FG-BRTXGZ-150027 strip LED light sources, through-beam photoelectric sensor, black cloth for shading, and matching power supply and switch.

[0024] (2) Deployment scheme: The camera and light source are installed vertically from the top, with an installation height of 1050±250mm, covering a field of view of about 1 meter; a black cloth is added around the perimeter to create a stable light source environment and eliminate ambient light interference.

[0025] (3) Work process: When the conveyor belt transports a single tobacco leaf to the inspection station, the photoelectric sensor triggers the industrial camera to complete the automatic image acquisition. The acquired image is named according to the "acquisition timestamp_tobacco leaf serial number" standard and is automatically transmitted to the data preprocessing subsystem.

[0026] 1.2 Data Preprocessing Subsystem The data preprocessing subsystem employs differentiated preprocessing strategies for the two main tasks of classification and defect detection: (1) Hierarchical task preprocessing: The image size is uniformly cropped to 1800×1800, HSV color space background removal, minimum bounding box cropping, square expansion, and size normalization are performed in sequence to remove background interference and maximize the proportion of tobacco leaf targets in the image.

[0027] (2) Defect detection task preprocessing: Only HSV color space background removal operation is performed to preserve the full resolution of the original image and ensure the accuracy of pixel-level annotation and detection.

[0028] 1.3 Core Algorithm Inference Subsystem The core algorithm inference subsystem is the core of this invention. It integrates a single tobacco leaf grading model based on interlayer adapter (FDAILA) and an unsupervised tobacco leaf defect detection model with adaptive noise steps (ADA-Net), and adopts a serial processing flow of "grading first and then defect detection".

[0029] The preprocessed images are first fed into the FDILA grading model to predict the five main acquisition grades (B2F, B3F, C2F, C3F, C4F) and the non-compliant category (BAD).

[0030] If the image is determined to be of a qualified level, the grading result is output directly; if it is determined to be of an unqualified level, the image is automatically sent to the ADA-Net defect detection model to complete pixel-level defect segmentation and availability calculation.

[0031] The FDAILA model employs a dual-branch parallel hybrid architecture, consisting of an input preprocessing module, a SwinTransformer global feature extraction branch with embedded inter-layer adapter (ILA), a MobileNetV2 local feature extraction branch, a dual-branch multi-scale feature adaptive fusion module, and a hierarchical output module.

[0032] (1) Interlayer Adapter (ILA) module: This is one of the core innovations of this invention. It adopts a bottleneck structure of "channel down-projection - depthwise separable spatial convolution - channel up-projection" and is embedded at the end of Stage0, Stage1, and Stage2 of the SwinTransformer branch. This module injects CNN local inductive bias into the Transformer with a very small parameter increment (about 0.04% of the total parameters). It expands token differences and filters redundant information by compressing / restoring the structure through nonlinear transformation, effectively alleviating the attention collapse problem of deep networks and enhancing the ability to capture subtle features of tobacco leaves with high similarity levels.

[0033] (2) Dual-branch multi-scale feature adaptive fusion module: For dual-branch features of the same size on four scales, channel alignment, adaptive calculation of channel weights and weighted fusion output are performed in sequence to realize the organic fusion of global semantic features and local detailed features.

[0034] (3) Hierarchical output module: The fused features are compressed into a one-dimensional feature vector by global average pooling, and then fed into a fully connected network and a Softmax activation function to output the predicted probabilities of 6 categories.

[0035] The ADA-Net model is based on DiffusionAD and consists of a multi-scale feature extractor, an adaptive noise step prediction network, an adaptive noise injection module, a one-step denoising and reconstruction network, and a multi-dimensional difference evaluation module.

[0036] (1) Adaptive noise step prediction network: This is another core innovation of the present invention. It adopts a three-layer MLP structure, receives multi-scale feature vectors extracted by ResNet-50 and FPN, and directly predicts the optimal noise step number through one forward propagation. This network can dynamically match the optimal noise injection intensity for color defects such as bluish-green and mixed colors in tobacco leaves and structural defects such as damage, thus solving the problem of poor adaptability of fixed noise step number to multiple types of defects.

[0037] (2) Adaptive noise injection module: Adaptive noise injection is performed on the input image according to the predicted noise step number. Strong noise is applied to the color abnormality area to completely destroy the abnormal features, and moderate noise is used to the structural abnormality area to retain sufficient contrast information.

[0038] (3) Multi-dimensional difference assessment module: integrates pixel-level difference, feature-level difference and structural similarity difference to generate a comprehensive anomaly score map, realizes pixel-level accurate segmentation of defect areas, and calculates the proportion of defect area and the utilization rate of tobacco leaves.

[0039] 1.4 FDAILA Mathematical Expression and Working Mechanism ILA can effectively mitigate attention collapse in VisionTransformer, mainly due to the following three mechanisms: (1) Nonlinear transformation amplifies token differences: In deep Transformers, the superposition of linear transformations can easily lead to the convergence of similarity between tokens, and the attention weights are close to uniform. ILA significantly amplifies the feature differences between different tokens through multi-level nonlinearity (GELU + convolution), so that the attention can learn a more discriminative weight distribution and avoid its degradation into "average distribution".

[0040] (2) Feature redundancy removal and diversity enhancement during compression-recovery: The bottleneck structure filters redundant information during the compression stage and introduces new combinations and transformations during the recovery stage, thereby reducing the accumulation of deep redundancy and opening up new expression directions in the feature space, enhancing diversity and improving sensitivity to fine-grained local patterns.

[0041] (3) Spatial convolution introduces local inductive bias: Self-attention lacks strong local constraints, which easily leads to feature learning averaging. Convolution explicitly models local correlations in the spatial neighborhood, making the ILA output pay more attention to structural local regions (texture, edges, etc.), thereby guiding subsequent attention to concentrate weights in these key regions and alleviating the global homogenization trend. The mathematical expression and working mechanism of the ILA module will be explained in detail below.

[0042] Let the input features be ,in The batch size, This represents the number of tokens (patches). For channel dimensions. The forward computation of ILA can be fully represented by the following steps: (1) Feature projection and nonlinear transformation The input features are first compressed in dimensionality by a lower projection layer, and then nonlinearly transformed by layer normalization and the GELU activation function, as shown in the following formula:

[0043] In the formula Here is the weight matrix for the lower projection layer. For layer normalization operation, These are the compressed low-dimensional features.

[0044] (2) Feature reshaping and spatial convolution enhancement The low-dimensional features in sequence form are reshaped into two-dimensional feature maps to accommodate spatial convolution operations, and then local spatial feature enhancement is achieved through convolutional layers, as shown in the following formula:

[0045] In the formula and For the height and width of the feature map, This is a 3×3 spatial convolution operation. This is the feature map after convolution enhancement.

[0046] (3) Feature projection and residual output The convolutional feature map is flattened back into a sequence, then restored to its original feature dimensions via an upper projection layer. Finally, the module output is obtained through residual connections and random deactivation, as shown in the following formula:

[0047] In the formula Here is the weight matrix of the upper projection layer. This is the final output of the module, and its dimensions are completely consistent with the input features, so it can be directly connected to subsequent SwinTransformerBlocks.

[0048] 1.5 ADA-Net Model Design Core To address the core limitations of existing algorithms, this chapter proposes an Adaptive Noise Reduction Step Anomaly Detection Network (ADA-Net). The model design is entirely focused on the practical pain points of tobacco leaf procurement, with three core objectives: First, it adopts an unsupervised learning paradigm, using only normal tobacco leaf samples for training, completely eliminating the reliance on high-cost defect labeling; second, through an adaptive noise reduction step mechanism, it solves the problem of poor adaptability of fixed step counts to various types of tobacco leaf defects, achieving accurate detection of all types of defects; and third, through pixel-level precise segmentation, it achieves quantitative calculation of defect area, supporting the defect removal of substandard tobacco leaves and the recycling of usable portions, addressing the core pain point of raw material waste in the industry.

[0049] 1.5.1 Overall Model Architecture and Core Design Logic ADA-Net is based on the industrial-grade unsupervised anomaly detection model DiffusionAD. It addresses the core pain points of tobacco leaf defect detection by innovating the design of three core modules. The overall architecture consists of five core parts: a multi-scale feature extractor, an adaptive noise step prediction network, an adaptive noise injection module, a one-step denoising and reconstruction network, and a multi-dimensional difference assessment and pixel-level segmentation module. This forms an end-to-end processing flow of "feature extraction → step prediction → adaptive noise addition → one-step denoising and reconstruction → pixel-level defect segmentation", which is fully adapted to the operational needs of tobacco leaf purchasing production lines.

[0050] The core design logic of the model corresponds one-to-one with industry pain points, and each innovative module has a clear problem-solving objective: (1) Adopting an unsupervised learning paradigm, the core training stage of the model only requires normal tobacco leaf samples, which fundamentally solves the industry pain points of scarce tobacco leaf defect samples and high pixel-level annotation costs, and significantly reduces the threshold for algorithm implementation.

[0051] (2) An adaptive noise step prediction network and an adaptive noise injection module were designed to dynamically match the optimal noise injection intensity for different types and scales of defects such as green, mixed colors and broken tobacco leaves. This completely solves the problem of poor adaptability of fixed noise step count to multiple types of tobacco leaf defects. At the same time, it achieves the dual goals of "fully destroying the defect area and reconstructing the normal area with high fidelity", laying the foundation for subsequent accurate segmentation.

[0052] (3) Construct a multi-dimensional difference assessment and segmentation module, integrate pixel-level, feature-level and structural similarity differences, realize pixel-level accurate positioning and segmentation of defect areas, accurately output the binary mask of defect areas, provide millimeter-level data support for defect area calculation and availability assessment, and solve the problem of raw material waste caused by the inability of traditional mode to accurately quantify the proportion of defects.

[0053] 1.5.2 Overall Model Architecture The overall architecture of the ADA-Net model is as follows: Figure 1 As shown, this architecture combines the efficiency of DiffusionAD's one-step denoising with the flexibility of GLAD's adaptive noise step count. It consists of five main components: a feature extractor, a noise step count prediction network, an adaptive noise injection module, a one-step denoising and reconstruction network, and an anomaly scoring module. These components form an end-to-end workflow: extracting image features → predicting the optimal noise step count → adaptively injecting noise → one-step denoising and reconstruction → comprehensive difference evaluation, effectively adapting to the detection needs of different types of tobacco leaf defects.

[0054] (1) Feature extractor The feature extractor uses ResNet-50 as the backbone network and fuses multi-scale features through FPN. It receives the input image. Output feature vector This provides information for noise step prediction. The detailed structure of the feature extractor is as follows: (a) Input layer: receive size is (b) RGB image backbone network: ResNet-50, extracting multi-scale features (c) Feature Pyramid Network (FPN): Fuses multi-scale features to generate (d) Adaptive pooling layer: converts the feature map to a fixed size. (e) Fully connected layer: generates a 512-dimensional feature vector.

[0055] This multi-scale feature extraction design enables the model to simultaneously capture fine-grained local texture information and global semantic information of tobacco leaves, providing a comprehensive image representation for subsequent noise step prediction.

[0056] (2) Noise step prediction network The noise step count prediction network is a three-layer MLP structure that receives feature vectors. Predict the optimal number of noise steps Unlike the iterative search in GLAD, the prediction network directly determines the optimal number of noise steps through a single forward propagation, significantly improving computational efficiency. The detailed structure of this network is as follows: (a) Input layer: receives a 512-dimensional feature vector. (b) Hidden layer 1: 512 neurons, using ReLU activation and BatchNorm (c) Hidden layer 2: 256 neurons, using ReLU activation and BatchNorm (d) Output layer: 1 neuron, using Sigmoid activation (e) Scaling layer: scales the output value to scope in, This is a predefined maximum number of noise steps. This design, while maintaining the model's expressive power, ensures the stability and accuracy of predictions through regularization techniques such as BatchNorm and Dropout.

[0057] (3) Adaptive noise injection module The adaptive noise injection module is a key component that combines the adaptive principles of GLAD with the DiffusionAD framework. This module determines the noise injection level based on the predicted noise steps. For the input image Perform adaptive noise injection to generate a noisy image. Unlike DiffusionAD with a fixed number of steps, this method can dynamically adjust the noise intensity for different types of tobacco leaf defects.

[0058] This adaptive noise injection strategy can apply strong noise to color anomaly regions to completely destroy the anomaly features, while using moderate noise to retain sufficient contrast information in structural anomaly regions, which significantly improves the model's adaptability to different types of defects.

[0059] (4) One-step denoising and network reconstruction The one-step denoising and reconstruction network adopts a U-Net structure, retaining the efficient reconstruction capability of DiffusionAD. It receives noisy images. and noise steps Directly predict noise-free images The detailed structure of the network is as follows: (a) Input layer: receives noisy images. and time embedding (b) Encoder: 4 downsampling blocks, each containing two residual convolutional layers and one downsampling layer. (c) Bottleneck layer: two residual convolutional layers. (d) Decoder: 4 upsampling blocks, each containing two residual convolutional layers and one upsampling layer. (e) Output layer: convolutional layer that generates an output of the same size as the input image. (5) Anomaly scoring module The anomaly scoring module calculates the original image. With reconstructed images The multidimensional differences between them generate anomaly score maps. This module combines pixel-level, feature-level, and structural-level difference metrics to comprehensively capture the characteristics of different types of defects on the surface of tobacco leaves.

[0060] Abnormal score graph The higher the value, the more likely the corresponding pixel is an anomaly. This comprehensive scoring method can effectively distinguish between normal variations in tobacco leaves and true defects, reducing false positive and false negative test results.

[0061] (6) Model training strategy The training of the ADA-Net model is divided into three stages. This progressive training strategy ensures that each component of the model can fully learn its respective target task: Phase 1: The reconstruction network is trained using only normal samples, with the goal of enabling the model to accurately reconstruct normal images. The loss function is the reconstruction loss.

[0062] Phase 2: Fix the reconstruction network parameters and train the noisy step prediction network using samples with outlier labels. The loss function includes reconstruction loss and detection loss.

[0063] In the formula It is a binary mask for the abnormal region. and These are weight hyperparameters. The key goal at this stage is to teach the noise step prediction network to select the optimal number of noise steps for different types of defects.

[0064] Phase 3: Predict network parameters with a fixed number of noise steps, and train the anomaly scoring module using samples with anomaly labels. The loss function is the binary cross-entropy loss.

[0065] In the formula It's the number of pixels. It is a pixel The actual label (0 indicates normal, 1 indicates abnormal). It is a pixel The predicted outlier score.

[0066] Through these three stages of training, the ADA-Net model can learn the distribution characteristics of normal tobacco leaves, predict the optimal number of noise steps, and accurately detect various defects on the surface of tobacco leaves. This progressive training ensures that each component of the model can fully perform its function, forming a collaborative end-to-end detection system.

[0067] 1.5.3 Adaptive Noise Step Optimization Module To address the limitations of fixed noise step count in DiffusionAD, this invention proposes an adaptive noise step count optimization algorithm. First, the Adaptive Denoising Step (ADS) technique in the GLAD model is analyzed. This technique dynamically determines the optimal number of denoising steps by evaluating the difference between the input image and the normal distribution. The main idea of ​​GLAD's ADS is that different anomaly types and severity require different numbers of denoising steps—more steps are needed for complex anomalies (such as structural anomalies, i.e., tobacco leaf damage), while fewer steps are needed for simple anomalies (such as color anomalies, i.e., abnormal tobacco leaf color).

[0068] GLAD achieves adaptive denoising steps through a search process: first, the test sample is given the maximum number of noise steps, and then a suitable number of steps is found by comparing thresholds during the denoising process. While effective, this method is computationally expensive and requires multiple denoising iterations. To improve efficiency, a noise step prediction network is proposed that can directly learn and predict the optimal number of noise steps based on the input image features in one step, and this network is integrated into the one-step denoising framework of DiffusionAD.

[0069] The adaptive noise step optimization algorithm designed in this invention comprises three key components: a feature extractor, a noise step prediction network, and an adaptive noise injection module. These components work together to form an end-to-end detection process of "feature extraction → step prediction → adaptive noise addition → one-step noise reduction," effectively improving the model's adaptability to different types of tobacco leaf defects.

[0070] (1) Feature extractor. The feature extractor is responsible for extracting representative features from the input image. These features should be able to reflect the content, structure and potential abnormal information of the image. A pre-trained ResNet-50 is used as the backbone network of the feature extractor, and multi-scale features are fused through the Feature Pyramid Network (FPN)

[62] to capture the hierarchical information of the image. Specifically, the feature extraction process can be expressed as:

[0071] In the formula These represent the feature maps output by ResNet-50 in four stages, with the size decreasing and the number of channels increasing sequentially.

[0072] To fully utilize multi-scale feature information, these features are fused using FPN:

[0073] In the formula It is a multi-scale feature after fusion. This represents the convolution operation. This indicates an upsampling operation.

[0074] Finally, adaptive average pooling and fully connected layers are used to transform multi-scale features into fixed-dimensional feature vectors:

[0075] In the formula Indicates adaptive average pooling. Indicates feature splicing, Indicates a fully connected layer. It is the final feature vector with a dimension of 512.

[0076] (2) Noise Step Prediction Network. The noise step prediction network is the core component of the adaptive noise step optimization algorithm. It receives the feature vector output by the feature extractor and predicts the noise step number best suited for the current image through a multilayer perceptron. Unlike the iterative search process in GLAD, this method directly predicts the optimal noise step number through a single forward propagation, significantly improving computational efficiency. The specific structure is as follows:

[0077] In the formula Indicates a fully connected layer. Indicates batch normalization, It is an activation function. It is the Sigmoid function. It is a predefined maximum number of noise steps. This is the predicted noise step count. The Sigmoid function restricts the output to the [0,1] interval, multiplied by... The final noise step count is obtained.

[0078] To make the noise step prediction more stable and reliable, regularization techniques and an attention mechanism were introduced. Regularization employs Dropout and weight decay to prevent overfitting; the attention mechanism helps the model focus on key regions in the image, improving prediction accuracy. The specific implementation is as follows:

[0079] In the formula It is a spatial attention module. Indicates element-wise multiplication. It is a feature vector that has been weighted by attention.

[0080] (3) Design of the noise step count loss function. In order to maximize the anomaly detection performance by predicting the noise step count, a special loss function was designed to guide the training of the noise step count prediction network. This loss function is based on the design idea of ​​GLAD, but has been simplified and optimized, and consists of two parts: reconstruction loss and detection loss.

[0081] Reconstruction loss measures the similarity between the reconstructed image and the original image in normal regions:

[0082] In the formula It is the original image. It is to reconstruct the image. It is a binary mask for the abnormal region. This represents element-wise multiplication.

[0083] Detection loss measures the difference between the original image and the reconstructed image in anomalous regions:

[0084] The total loss function is a weighted sum of two parts:

[0085] In the formula and It is a weight hyperparameter used to balance the contributions of the two loss terms.

[0086] Through this loss function, the noise step prediction network learns how many noise steps to select for different types of anomalies to make the reconstructed image similar to the original image in normal regions while showing significant differences in anomaly regions. This adaptive learning strategy enables the model to select the optimal number of noise steps for different types of tobacco leaf defects.

[0087] 1.5.4 Adaptive Noise Injection Adaptive noise injection is a key step in integrating the adaptive denoising step count idea of ​​GLAD into the DiffusionAD framework. In the original GLAD model, the adaptive process mainly occurs in the denoising stage, determining the appropriate stopping step count by comparing intermediate results during denoising. In our method, this adaptive mechanism is moved forward to the noise injection stage, using the predicted noise step count to directly guide the noise injection process, and then leveraging DiffusionAD's one-step denoising technique to achieve efficient reconstruction.

[0088] Given an input image and the predicted number of noise steps The adaptive noise injection process can be represented as:

[0089] In the formula , It is a predefined noise scheduling, It is standard Gaussian noise.

[0090] Noise Dispatch This is a crucial hyperparameter in the diffusion model, determining the rate at which noise is added. In this experiment, linear noise scheduling is employed:

[0091] In the formula , , It is the maximum number of noise steps.

[0092] Adaptive noise injection exhibits different behaviors depending on the type of tobacco leaf defect: For color anomalies (cyan and mixed colors): Since color anomalies are mainly manifested as local tonal anomalies, the model tends to select a larger noise step number so that the color features of these areas are completely covered by Gaussian noise, thus being "corrected" to normal colors during the reconstruction process.

[0093] For structural anomalies (damaged parts): Structural anomalies typically manifest as physical damage to the tobacco leaf surface, such as cracks and perforations. For these types of anomalies, an excessive number of noise steps may lead to the loss of important structural information, affecting subsequent anomaly detection. Therefore, the model tends to select a smaller number of noise steps to retain sufficient structural information for comparison with the reconstruction results.

[0094] This adaptive strategy enables the model to select the optimal number of noise steps for different types of tobacco leaf defects, thereby improving the accuracy and robustness of detection.

[0095] Furthermore, drawing on the local feature fusion concept from GLAD, the adaptive noise injection process was further optimized. By introducing conditional information, the noise injection process can better utilize image features:

[0096] In the formula Based on feature vectors The modulation function is used to adjust the noise distribution according to image features. Specifically, It is a learned function, which can be represented as:

[0097] In the formula It is the Sigmoid function, with an output range of (0,2), used to scale noise. In this way, the noise injection process considers not only the number of noise steps but also image features, further improving the model's adaptability.

[0098] Furthermore, similar to the Spatial Adaptive Feature Fusion (SAFF) introduced in GLAD, a spatial attention mechanism is also introduced in the noise injection stage, enabling the model to adopt different noise strategies for normal and abnormal regions:

[0099] In the formula It is a spatial attention map with values ​​ranging from [0,1]. For potentially anomalous regions, the attention value is higher, resulting in increased noise intensity; for normal regions, the attention value is lower, maintaining a moderate noise intensity.

[0100] By incorporating the adaptive denoising concept of GLAD into the noise injection stage of DiffusionAD, the method achieves an efficient reconstruction process of "adaptive noise addition + one-step denoising". This retains the computational efficiency advantage of DiffusionAD while introducing the adaptive capability of GLAD, which can better adapt to different types of tobacco leaf defects.

[0101] 1.5.5 Reconstruction-based Defect Detection After completing adaptive noise injection and one-step denoising reconstruction, the reconstructed image was obtained. The difference between the reconstructed image and the original image can be used to detect defects on the surface of tobacco leaves. Traditional methods typically use pixel-level differences directly as anomaly scores, but this approach is sensitive to noise and cannot fully utilize high-level semantic information. To address this, a comprehensive difference evaluation module was designed that considers multiple difference measures, including pixel-level differences, feature-level differences, and structural similarity differences.

[0102] (1) Pixel-level differences. Pixel-level differences are calculated directly by measuring the Euclidean distance between the original image and the reconstructed image in pixel space:

[0103] This difference measure is simple and intuitive, and can capture changes in color, brightness, etc. of an image, but it is sensitive to noise and may ignore semantic differences.

[0104] (2) Feature-level difference. Feature-level difference compares the distance between the original image and the reconstructed image in the feature space. Image features are extracted using a pre-trained feature extractor (such as VGG or ResNet), and then the distance between the feature vectors is calculated:

[0105] In the formula Indicates the first Layer feature extractor, These are weighting coefficients used to balance the contributions of features from different layers.

[0106] Feature-level differences can capture semantic differences in images, which is particularly effective for detecting structural anomalies. Multiple levels of features from ResNet-50 were selected for comparison, including low-level features (such as edges and textures) and high-level features (such as semantic information).

[0107] (3) Structural similarity difference. The structural similarity difference is calculated based on the Structural Similarity Index (SSIM), which takes into account the brightness, contrast, and structural information of the image:

[0108] In the formula Calculate the structural similarity index between two images, ranging from [0,1], where a larger value indicates greater similarity. Therefore, A larger value indicates a greater difference in the image.

[0109] The formula for calculating SSIM is:

[0110] In the formula and It is an image and The mean, and It is their variance. It is covariance. and It is a constant added to avoid the denominator being zero.

[0111] Structural similarity differences can better reflect the human visual system's perception of image quality and are effective in detecting color and structural anomalies.

[0112] (4) Comprehensive Anomaly Score. The comprehensive anomaly score is obtained by weighted fusion of the above three types of differences:

[0113] In the formula , and These are weighting coefficients, determined through validation set optimization. In the experiments, , , This set of weights achieved the best results in the task of detecting defects in tobacco leaves.

[0114] To further improve the accuracy of anomaly scoring, a self-attention mechanism was introduced, enabling the model to adaptively focus on key features of different types of anomalies.

[0115] In the formula It is a self-attention module. It is attention weight. This represents three measures of difference.

[0116] This comprehensive difference assessment method can more accurately detect different types of defects on the surface of tobacco leaves, whether they are color abnormalities or structural abnormalities.

[0117] 2 Simulation Results 2.1 FDILA Simulation Verification To verify the advancement of the FDILA method of this invention, this experiment selected classic CNN classification algorithms, Transformer classification algorithms, and tobacco leaf grading algorithms FDANet and MFASTG for comparison. All algorithms were trained on the self-built single-leaf 6-classification dataset of this invention, with the original parameter settings from their respective literatures to ensure experimental fairness. The grading results are shown in Table 1. The experimental results readily lead to the following core conclusions: Table 1 shows the grading results of each method on the single tobacco leaf dataset.

[0118] (1) The CNN-Transformer hybrid architecture performs significantly better than the single-structure algorithm. Tobacco leaf grading requires capturing both global semantic features and local detail features. The dual-branch hybrid architecture can achieve complementarity between the two types of features and is more suitable for fine-grained grading requirements. Among them, the average accuracy of the classic CNN structures ResNet50 and MobileNetV2 is 78.25% and 77.63%, respectively, and the macro-F1 is 78.10% and 77.40%, respectively. The average accuracy of the pure Transformer structure SwinTransformer is only 76.33%, and the macro-F1 is 76.20%. It is not good at extracting local details, and deep networks are prone to attention collapse. Its ability to capture fine-grained features is weaker than that of the CNN structure.

[0119] (2) The overall classification performance of FDILA has achieved a breakthrough improvement. The baseline model FDANet (dual-branch structure) achieved an average accuracy of 85.27% and a macro-F1 score of 85.26%. Although the MFASTG model improved the average accuracy to 91.45% and the macro-F1 score to 91.38%, it suffered severe performance degradation in the core level C2F, with an accuracy of only 76.59%, which was lower than the baseline model. In contrast, the average accuracy and macro-F1 score of the present invention FDILA are improved by 11.06% and 11.06% respectively compared with the baseline FDANet, and by 4.88% and 4.94% respectively compared with MFASTG, which significantly reduces misclassification, especially cross-level misclassification, and significantly reduces the economic losses caused by classification errors.

[0120] (3) FDILA demonstrates comprehensive advantages in all classification levels. The model achieves the best accuracy among the comparison algorithms in the five levels of B2F, B3F, BAD, C2F, and C3F. Among them, the accuracy of the core high similarity levels C2F and C3F reaches 94.15% and 95.54% respectively, which is 11.22% and 12.08% higher than FDANet, and 17.56% and 5.94% higher than MFASTG, respectively. This fully verifies the enhanced capture capability of the ILA module for fine-grained features and solves the problem of performance degradation of the core level of MFASTG. At the same time, FDILA achieves 100% classification accuracy for the non-conforming BAD class, which fully meets the needs of accurate removal of non-conforming tobacco leaves at the acquisition site.

[0121] This invention further analyzes the parameter size (Weight) and average inference time (Time) of each algorithm model to evaluate the feasibility of deploying the proposed FDILA in actual tobacco leaf purchasing scenarios. The comparison results are shown in Table 2. The lightweight model MobileNetV2 has a parameter size of only 8.5MB and an average inference time of only 8.4ms, but its performance is poor and cannot meet the accuracy requirements of the purchasing site. Compared with the baseline FDANet, the proposed FDILA only increases the number of parameters by 0.5MB and the inference time by only 0.7ms, with a parameter increase of less than 0.4%, yet achieves an accuracy improvement of over 11%. This achieves "extremely small resource overhead for extremely large performance improvement," fully meeting the memory usage and real-time computing speed requirements of industrial embedded devices, and possesses strong engineering application value.

[0122] Table 2. Parameter values ​​and inference time for each algorithm model.

[0123] Next, this invention uses a confusion matrix to conduct an in-depth analysis of the model's misjudgments. Figure 2 , Figure 3 , Figure 4 The confusion matrices of baseline FDANet, MFASTG, and the present invention FDAILA on the single tobacco leaf test set are shown respectively.

[0124] Combination Figure 2 As can be seen from the confusion matrix of the baseline FDANet, the misclassification distribution of the original model is relatively scattered. Not only are there a large number of mutual misclassifications between adjacent levels, but there are also many misclassifications across parts and levels. For example, B2F is misclassified as C2F and C3F, and C2F is misclassified as B2F and B3F. This is because the attention collapse of the Transformer branch of FDANet makes it difficult for the deep network to focus on fine-grained discriminative features, resulting in insufficient ability to distinguish between different levels.

[0125] Combination Figure 3 The confusion matrix of the MFASTG model shows that it significantly improves the classification accuracy of B2F, B3F, and C4F levels through multi-layer feature fusion, but it suffers severe performance degradation at the C2F level, with an accuracy of only 76.59%. A large number of C2F samples are misclassified as B2F, B3F, and C3F. This is because the feature fusion strategy of the model fails to capture the subtle features of the middle tobacco leaves, resulting in a decrease in the ability to distinguish high similarity levels.

[0126] Combination Figure 4 The confusion matrix of the FDILA module proposed in this invention shows that the model's misclassification is greatly improved. The diagonal accuracy of all grades is at an extremely high level, with misclassifications concentrated only between adjacent grades and the misclassification rate is extremely low. No serious misclassifications across grades or parts occur. Specifically, misclassifications of C2F are mainly concentrated in the adjacent C3F grade, accounting for only 5.85%, with no misclassifications across grades such as B2F and B3F. Misclassifications of C3F are also concentrated only in the adjacent C2F and C4F grades, accounting for less than 5% in total. The misclassification rates of B2F, B3F, and BAD grades are all below 2%. This result fully demonstrates that the ILA module effectively alleviates the attention collapse problem of the Transformer branch, allowing the model to accurately capture the discriminative features of different grades of tobacco leaves, greatly reducing misclassifications, especially eliminating cross-grade misclassifications that can cause serious economic losses.

[0127] To visually verify the enhancement effect of the ILA module on fine-grained feature focusing capability, this invention employs the Grad-weighted Class Activation Mapping (Grad-CAM) method to visually compare the feature focus regions of the baseline FDANet model and the proposed FDILA model. The results are as follows: Figure 5 As shown.

[0128] The visualization results clearly show that the attention distribution of the baseline FDANet model has obvious defects: its attention weight distribution is relatively scattered, with a large number of invalid responses in the background area, and a serious lack of focus on the key discriminative regions of the tobacco leaf surface (leaf vein texture, color transition area, leaf surface structural details, etc.). In some cases, the attention deviates from the main body of the tobacco leaf, which is a direct manifestation of the decrease in feature discriminativeness caused by the collapse of attention in deep networks.

[0129] The FDAILA model, after introducing the ILA module, exhibits three significant optimization effects in its attention distribution, which corroborates the theoretical analysis above: (1) Precise focus on the main body area: The model attention is completely locked on the main body area of ​​the tobacco leaf, which effectively suppresses the invalid response of the background area and greatly reduces the interference of irrelevant information on the classification decision; (2) Fine-grained feature depth perception: Attention is stably focused on the core discrimination area of ​​the tobacco leaf surface, including fine-grained feature areas that are strongly related to tobacco grade determination, such as leaf vein texture, leaf color distribution, and leaf shape structure, and the ability to capture subtle visual differences is significantly enhanced. (3) Improved adaptability of highly similar samples: For level samples such as B2F and C2F, C2F and C3F with highly similar visual features, the model can still adaptively lock the corresponding discriminative features and accurately capture the subtle differences between levels, which verifies the role of the ILA module in improving the ability to distinguish highly similar categories.

[0130] The above visualization results intuitively verify that the ILA module can effectively alleviate the attention collapse problem of the Transformer branch, significantly enhance the model's ability to perceive and focus on fine-grained features, and lay a core feature foundation for the subsequent improvement of classification performance.

[0131] 2.2 ADA-Net Simulation Verification Table 3 shows the overall performance comparison of different algorithms on a self-built tobacco leaf defect dataset, comprehensively presenting the overall performance of each algorithm from three dimensions: detection accuracy, localization capability, and inference efficiency. The table shows that the feature embedding-based method, due to the absence of a reverse reconstruction process, has extremely fast inference speed, perfectly meeting the real-time requirements of industry. However, it also exposes its limitations in detecting defects in natural products like tobacco leaves; it cannot achieve high-precision defect detection, a loss that may be unacceptable. The reconstruction-based method, due to the reverse reconstruction process, sacrifices some inference speed, but gains sufficient detection accuracy in return. In actual field operations, considering the defect removal time for workers, a defect detection time of less than 2 seconds per tobacco leaf is acceptable.

[0132] Table 3 compares the performance of different methods in the task of detecting surface defects in tobacco leaves.

[0133] The following key conclusions can be drawn from the experimental results: (1) Reconstruction methods based on diffusion models have significantly better overall performance than traditional methods based on feature embedding. Taking the baseline model DiffusionAD as an example, its AUROC index reaches 95.4%, which is 5.3% higher than the best-performing feature embedding algorithm PatchCore, and its AUPRO index is also 2.1% higher. The core reason is that tobacco leaves are non-rigid natural products, and their normal samples have large intra-class differences in morphology, color, and texture. Methods based on feature embedding are prone to misjudging normal natural variations of tobacco leaves as abnormalities, resulting in a large number of false positive results. In contrast, diffusion models, by learning the complete generation distribution of normal tobacco leaves, can better distinguish between "normal natural variations" and "real defects." At the same time, with its powerful pixel-level reconstruction capability, its ability to capture subtle color defects such as cyan and mixed colors is far superior to feature matching methods, making it more suitable for the task characteristics of tobacco leaf defect detection.

[0134] (2) The ADA-Net method proposed in this invention surpasses all comparison algorithms in core accuracy metrics, achieving a further breakthrough in tobacco defect detection performance. Compared to the baseline model DiffusionAD, ADA-Net improves pixel-level AUROC by 0.7% and AUPRO by 1.0%, with both core metrics reaching the best level on the dataset. This result fully verifies the effectiveness of the core innovation of this invention: addressing the core pain point of DiffusionAD's fixed noise step count, the designed adaptive noise step count prediction network and dynamic noise injection mechanism can match the optimal noise intensity for different defect types and degrees, ensuring sufficient destruction and accurate reconstruction of abnormal areas while avoiding the loss of details in normal areas. At the same time, combined with the multi-dimensional difference evaluation module, it achieves comprehensive capture of defect features, ultimately improving both pixel-level classification accuracy and region-level localization capability.

[0135] (3) From the perspective of inference efficiency, feature embedding-based methods generally have faster inference speed, but their detection accuracy is difficult to meet the core requirements of defect detection rate in industrial scenarios. Although the traditional multi-step diffusion model DDAD has better accuracy than feature embedding methods, its inference speed is extremely slow and cannot meet the real-time detection requirements of the pipeline. The baseline DiffusionAD compresses the single-image inference time to 200ms through a one-step denoising strategy, which is an order of magnitude speed improvement compared to the multi-step diffusion model. The ADA-Net method of this invention increases the single-image inference time to 500ms (FPS=2) due to the introduction of a noise step prediction network and a multi-dimensional difference evaluation module. However, combined with the actual operating speed of the tobacco purchasing pipeline, the dwell time of a single tobacco leaf at the detection station is usually greater than 2s. The 500ms inference time can fully meet the real-time requirements of the industrial site and achieve a reasonable balance between detection accuracy and inference efficiency.

[0136] To intuitively verify the superior reconstruction fidelity of the adaptive noise step count mechanism of this invention for normal tobacco leaf samples, this section selects defect-free normal tobacco leaf test samples and conducts comparative experiments using baseline DiffusionAD and the ADA-Net method of this invention. The noisy images and denoised reconstruction results under different noise step counts are visualized, further demonstrating the core value of the adaptive noise step count mechanism from the perspective of the feature restoration capability of normal samples. The reconstruction process and visualization results of the two methods are as follows: Figure 6 , Figure 7 As shown.

[0137] The visualization comparison results clearly show the significant difference in reconstruction capabilities between the baseline model and the method of this invention. The core conclusions are as follows: (1) The fixed noise step strategy of the baseline DiffusionAD has inherent defects such as excessive destruction of normal features and insufficient reconstruction fidelity. DiffusionAD adopts a fixed dual-step noise injection scheme, with shallow noise addition fixed at t=200 and deep noise addition fixed at t=400. As can be seen from the results, the fixed-step noise addition process causes irreversible excessive destruction of the texture and color features of normal tobacco leaves: after shallow noise addition and denoising, the veins and natural texture of tobacco leaves are obviously blurred; after deep noise addition, Gaussian noise completely covers the original structure and color information of tobacco leaves. Even after one step of denoising and reconstruction, the final result still has serious graininess, texture distortion and color deviation, and cannot restore the fine structure and natural gradient features of tobacco leaves. The severe reconstruction distortion of normal areas will directly lead to a large number of false abnormal responses in the subsequent difference assessment, which is one of the core reasons for the high false positive rate of the baseline model.

[0138] (2) The adaptive noise step count mechanism of the ADA-Net in this invention achieves high-fidelity reconstruction of normal regions, fundamentally reducing the risk of false positives. For defect-free normal tobacco leaf samples, the noise step count prediction network of this invention adaptively matches a more reasonable shallow noise step count (t=150), which, while meeting the denoising reconstruction requirements of the diffusion model, preserves the core features of normal tobacco leaves to the maximum extent. Three major advantages can be summarized from the reconstruction results: (a) Significantly improved detail retention: After shallow noise addition and denoising, the subtle features of tobacco leaves, such as leaf vein texture, edge structure, and natural leaf wrinkles, are completely preserved. The reconstructed image is smooth and grain-free, and its visual consistency with the original input is far superior to the baseline model. (b) Superior noise reduction and reconstruction capabilities: Even at a high noise level of t=400 in the deep layer, the model can still accurately restore the image content that conforms to the normal tobacco leaf distribution, without obvious noise residue, structural distortion or color distortion; (c) Significantly reduced reconstruction error: Quantization results show that the average pixel difference Δmean between the final reconstruction result of ADA-Net and the original input is only about 0.0027, and the maximum pixel difference Δmax is controlled within 0.3. The reconstruction accuracy of the normal area has been improved by orders of magnitude.

[0139] The reconstruction comparison results of the above-mentioned normal samples, together with the quantitative experiments on defect detection and the visualization analysis of defect samples mentioned above, form a complete mutual verification, which fully proves that the adaptive noise step mechanism of the present invention can dynamically adapt to the features of the input image. It not only solves the problem of missed detection due to insufficient noise injection into the defect area by the baseline model, but also solves the problem of false positives caused by excessive noise addition to the normal area with a fixed number of steps. It achieves the dual goal of "sufficient destruction of the defect area and high-fidelity reconstruction of the normal area", providing core support for high-precision and low-false-detection of tobacco defect detection.

[0140] In summary, such as Figure 8 As shown, this is a production line for grading individual tobacco leaves. After sorting, the individual tobacco leaves enter a conveyor belt, where a camera-based vision algorithm completes the grading process. Defective tobacco leaves then enter a conveyor belt for defect detection. A vision algorithm detects the defects, and recycling personnel remove the defective parts based on the marked areas. Figure 9 As shown, substandard tobacco leaves will enter the conveyor belt for tobacco leaf defect detection. The defective parts of the tobacco leaves will be detected by visual algorithms, and the recycling personnel will then remove the defective parts according to the marked parts.

[0141] like Figure 10 As shown, this is the process of purchasing individual tobacco leaves through a system. If the tobacco leaf is qualified, it can be purchased according to its grade. If it is unqualified, it needs to enter the defect detection line to remove the defective parts before purchasing. Specifically, it includes the following steps: S1. Image Acquisition: The industrial vision acquisition subsystem automatically acquires images of individual tobacco leaves on the conveyor belt, triggered by photoelectric sensors. S2. Image Preprocessing: The data preprocessing subsystem performs differentiated preprocessing operations on the acquired images according to the task type. S3. Individual Tobacco Leaf Grading: The preprocessed image is sent to the FDILA grading model, which outputs the grade prediction result. S4. Result Judgment: If the grading result is qualified, proceed to step S6; if it is unqualified, proceed to step S5. S5. Defect Detection and Utilization Calculation: Images of unqualified tobacco leaves are sent to the ADA-Net defect detection model to complete pixel-level defect segmentation and calculate the defect area ratio and the tobacco leaf utilization rate. S6. Result Output: The result visualization and export subsystem displays the detection results and generates a detection report.

[0142] Related alternatives: Alternatives to the hierarchical model: The SwinTransformer branch can be replaced with other Transformer architectures such as ViT and DeiT, or the MobileNetV2 branch can be replaced with other CNN architectures such as ResNet and EfficientNet, depending on the actual accuracy and speed requirements.

[0143] Defect detection model alternative: The one-step denoising reconstruction network can be replaced with a multi-step denoising network, at the cost of sacrificing some inference speed, to further improve reconstruction quality and detection accuracy.

[0144] Hardware alternatives: Depending on the actual budget and resolution requirements, other brands or models of industrial cameras and light sources can be selected, or line scan cameras can be used instead of area scan cameras to adapt to higher-speed assembly line operations.

[0145] In another aspect, the present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method described above.

[0146] In another aspect, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method described above.

[0147] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the tobacco leaf grading and defect detection methods based on interlayer adapter and diffusion models in the above embodiments.

[0148] It is understood that the systems, devices, and storage media provided in the embodiments of the present invention correspond to the methods provided in the embodiments of the present invention, and the explanations, examples, and beneficial effects of the relevant content can be referred to the corresponding parts of the above methods.

[0149] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0150] It should be noted that in this invention, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0151] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0152] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for tobacco leaf grading and defect detection based on interlayer adapter and diffusion model, characterized in that, Perform the following steps using a computer device. S1. Image Acquisition: A visual acquisition subsystem integrating an industrial camera, a strip light source, and a through-beam photoelectric sensor to automatically acquire raw images of single tobacco leaves from the production line under stable, shaded lighting conditions. S2. Differentiated preprocessing: For graded tasks, image cropping and normalization are performed to remove the background, while for defect detection tasks, the full resolution is preserved and only the background is removed. S3. Tobacco Leaf Grading Inference: The preprocessed image is fed into the dual-branch FDAILA grading model with an embedded interlayer adapter ILA, and the prediction results of 6 grades of tobacco leaves are output. The ILA module is embedded in each stage of the SwinTransformer and injects the bottleneck structure of channel compression-depth separable convolution-channel recovery into the local inductive bias of the CNN to alleviate deep attention collapse. S4. Branch Judgment: If the output is a qualified tobacco leaf grade, directly output the grading result; if it is determined to be an unqualified tobacco leaf, execute the defect detection step. S5. Unsupervised Defect Detection: Input the image of unqualified tobacco leaves into the ADA-Net diffusion detection model. After multi-scale feature extraction, MLP network adaptive prediction of the optimal noise step number, adaptive noise injection, one-step U-Net denoising and reconstruction, the multi-dimensional differences of pixel, feature, and structural similarity are fused to generate an anomaly score map, complete pixel-level defect segmentation and calculate the proportion of tobacco leaf defects and availability. S6. Results Visualization and Export: Synchronously output graded and defect quantification data and generate inspection reports.

2. The method for tobacco leaf grading and defect detection based on interlayer adapter and diffusion model according to claim 1, characterized in that: S2 includes, The data preprocessing subsystem employs differentiated preprocessing strategies for the two main tasks of classification and defect detection: Hierarchical task preprocessing: sequentially perform image size uniform cropping to 1800×1800, HSV color space background removal, minimum bounding box cropping, square expansion, and size normalization operations to remove background interference and maximize the proportion of tobacco leaf targets in the image; Defect detection task preprocessing: Only HSV color space background removal is performed to preserve the full resolution of the original image and ensure pixel-level annotation and detection accuracy.

3. The method for tobacco leaf grading and defect detection based on interlayer adapter and diffusion model according to claim 1, characterized in that: S3 specifically includes, The preprocessed images are first fed into the FDILA grading model to predict the five main acquisition grades, namely B2F, B3F, C2F, C3F, C4F, and the non-compliant category BAD. If the image is determined to be of the qualified level, the grading result is output directly; if it is determined to be of the unqualified level, the image is automatically sent to the ADA-Net defect detection model to complete pixel-level defect segmentation and availability calculation. The FDAILA model adopts a dual-branch parallel hybrid architecture, consisting of an input preprocessing module, a SwinTransformer global feature extraction branch with embedded inter-layer adapter ILA, a MobileNetV2 local feature extraction branch, a dual-branch multi-scale feature adaptive fusion module, and a hierarchical output module. The interlayer adapter ILA module adopts a bottleneck structure of "channel down-projection - depthwise separable spatial convolution - channel up-projection" and is embedded at the end of Stage 0, Stage 1, and Stage 2 of the SwinTransformer branch. This module injects CNN local inductive bias into the Transformer with a very small increment of parameters. It expands token differences and filters redundant information by compressing / restoring the structure through nonlinear transformation, effectively alleviating the attention collapse problem of deep networks and enhancing the ability to capture subtle features of tobacco leaves with high similarity levels. Dual-branch multi-scale feature adaptive fusion module: For dual-branch features of the same size at four scales, channel alignment, adaptive calculation of channel weights, and weighted fusion output are performed sequentially to achieve organic fusion of global semantic features and local detailed features; Hierarchical output module: The fused features are compressed into a one-dimensional feature vector by global average pooling, and then fed into a fully connected network and a Softmax activation function to output the predicted probabilities of 6 categories; The ADA-Net model is based on DiffusionAD and consists of a multi-scale feature extractor, an adaptive noise step prediction network, an adaptive noise injection module, a one-step denoising and reconstruction network, and a multi-dimensional difference evaluation module. Adaptive noise step prediction network: It adopts a three-layer MLP structure, receives multi-scale feature vectors extracted by ResNet-50 and FPN, and directly predicts the optimal noise step through one forward propagation; Adaptive noise injection module: Adaptive noise injection is performed on the input image according to the predicted noise step number. Strong noise is applied to color abnormality areas to completely destroy the abnormal features, while moderate noise is applied to structural abnormality areas to retain sufficient contrast information. Multi-dimensional difference assessment module: It integrates pixel-level differences, feature-level differences and structural similarity differences to generate a comprehensive anomaly score map, achieving pixel-level accurate segmentation of defective areas and calculating the proportion of defective area and the usability of tobacco leaves.

4. The method for tobacco leaf grading and defect detection based on interlayer adapter and diffusion model according to claim 3, characterized in that: The inter-layer adapter ILA module includes the following three mechanisms: Nonlinear transformations amplify token differences: In deep Transformers, the superposition of linear transformations can easily lead to convergence in the similarity between tokens, and the attention weights become nearly uniform. The inter-layer adapter ILA module significantly amplifies the feature differences between different tokens through multi-level non-linearity, namely GELU+ convolution, enabling the attention to learn a more discriminative weight distribution and avoid its degradation into "average distribution". Compression-Recovery Feature Redundancy Removal and Diversity Enhancement: The bottleneck structure filters redundant information in the compression stage and introduces new combinations and transformations in the recovery stage, thereby reducing the accumulation of deep redundancy and opening up new expression directions in the feature space, enhancing diversity and improving sensitivity to fine-grained local patterns. Spatial convolution introduces local inductive bias: self-attention lacks strong local constraints, which can easily lead to feature learning averaging. Convolution explicitly models local correlations in the spatial neighborhood, making the output of the inter-layer adapter ILA module pay more attention to structural local regions, thereby guiding subsequent attention to concentrate weights in these key regions and alleviating the global homogenization trend.

5. The method for tobacco leaf grading and defect detection based on interlayer adapter and diffusion model according to claim 4, characterized in that: The mathematical expression and working mechanism of the inter-layer adapter ILA module are as follows: Let the input features be ,in The batch size, For the number of tokens, For the channel dimension; the forward computation of the inter-layer adapter ILA module is represented by the following steps: (1) Feature projection and nonlinear transformation The input features are first compressed in dimensionality by a lower projection layer, and then nonlinearly transformed by layer normalization and the GELU activation function, as shown in the following formula: In the formula Here is the weight matrix for the lower projection layer. For layer normalization operation, These are the compressed low-dimensional features; (2) Feature reshaping and spatial convolution enhancement The low-dimensional features in sequence form are reshaped into two-dimensional feature maps to accommodate spatial convolution operations, and then local spatial feature enhancement is achieved through convolutional layers, as shown in the following formula: In the formula and For the height and width of the feature map, This is a 3×3 spatial convolution operation. The feature map is enhanced by convolution; (3) Feature projection and residual output The convolutional feature map is flattened back into a sequence, then restored to its original feature dimensions via an upper projection layer. Finally, the module output is obtained through residual connections and random deactivation, as shown in the following formula: In the formula Here is the weight matrix of the upper projection layer. This is the final output of the module, and its dimensions are completely consistent with the input features, so it can be directly connected to subsequent SwinTransformerBlocks.

6. The method for tobacco leaf grading and defect detection based on interlayer adapter and diffusion model according to claim 3, characterized in that: The ADA-Net model includes a feature extractor, a noise step prediction network, an adaptive noise injection module, a one-step denoising and reconstruction network, and an anomaly scoring module. These components form an end-to-end process: extracting image features → predicting the optimal noise step count → adaptively injecting noise → one-step denoising and reconstruction → comprehensive difference evaluation. The feature extractor uses ResNet-50 as the backbone network and fuses multi-scale features through FPN; it receives the input image. Output feature vector This provides information for noise step count prediction; The detailed structure of the feature extractor is as follows: (a) Input layer: Receive size is (a) RGB image; (b) Backbone network: ResNet-50, extracting multi-scale features (c) Feature Pyramid Network (FPN): fuses multi-scale features to generate (d) Adaptive pooling layer: converts the feature map to a fixed size; (e) Fully connected layer: generates a 512-dimensional feature vector. ; The noise step count prediction network is a three-layer MLP structure that receives feature vectors. Predict the optimal number of noise steps The detailed structure of the network is as follows: (a) Input layer: receives 512-dimensional feature vectors. (b) Hidden layer 1: 512 neurons, using ReLU activation and BatchNorm; (c) Hidden layer 2: 256 neurons, using ReLU activation and BatchNorm; (d) Output layer: 1 neuron, using Sigmoid activation; (e) Scaling layer: scales the output value to... scope; The adaptive noise injection module determines the number of noise steps based on the predicted noise level. For the input image Perform adaptive noise injection to generate a noisy image. Unlike DiffusionAD with a fixed number of steps, this method can dynamically adjust the noise intensity for different types of tobacco leaf defects. The one-step denoising and reconstruction network adopts a U-Net structure, retaining the efficient reconstruction capability of DiffusionAD; it receives noisy images. and noise steps Directly predict noise-free images The detailed structure of the network is as follows: (a) Input layer: receives noisy images. and time embedding (b) Encoder: 4 downsampling blocks, each block containing two residual convolutional layers and one downsampling layer; (c) Bottleneck layer: two residual convolutional layers; (d) Decoder: 4 upsampling blocks, each block containing two residual convolutional layers and one upsampling layer; (e) Output layer: convolutional layer that generates an output of the same size as the input image. The anomaly scoring module calculates the original image. With reconstructed images The multidimensional differences between them generate anomaly score maps. This module combines pixel-level, feature-level, and structural-level difference measurements to comprehensively capture the characteristics of different types of defects on the surface of tobacco leaves. Abnormal score graph The larger the value, the higher the probability that the corresponding pixel is abnormal.

7. The method for tobacco leaf grading and defect detection based on interlayer adapter and diffusion model according to claim 3, characterized in that: The training of the ADA-Net model is divided into three stages. This progressive training strategy ensures that each component of the model can fully learn its respective target task: Phase 1: The reconstruction network is trained using only normal samples, with the goal of enabling the model to accurately reconstruct normal images; the loss function is the reconstruction loss. Phase 2: Fix the reconstruction network parameters and train the noisy step prediction network using samples with outlier labels; the loss function includes reconstruction loss and detection loss: In the formula It is a binary mask for the abnormal region. and These are weight hyperparameters; the key goal at this stage is to enable the noise step prediction network to learn to select the optimal number of noise steps for different types of defects. Phase 3: Predict network parameters with a fixed number of noise steps, and train the anomaly scoring module using samples with anomaly labels; the loss function is binary cross-entropy loss. In the formula It's the number of pixels. It is a pixel The true label is 0 for normal and 1 for abnormal. It is a pixel The predicted outlier score.

8. The method for tobacco leaf grading and defect detection based on interlayer adapter and diffusion model according to claim 1, characterized in that: The S5 MLP network adaptively predicts the optimal noise step count, which also includes an adaptive noise step count optimization algorithm. It contains three key components: a feature extractor, a noise step count prediction network, and an adaptive noise injection module. These components work together to form an end-to-end detection process of "feature extraction → step count prediction → adaptive noise addition → one-step noise reduction". The feature extractor is responsible for extracting representative features from the input image. These features should reflect the image's content, structure, and potential anomalies. A pre-trained ResNet-50 is used as the backbone network of the feature extractor, and multi-scale features are fused through a feature pyramid network to capture the hierarchical information of the image. Specifically, the feature extraction process is represented as follows: In the formula These represent the feature maps output by ResNet-50 in four stages, with the size decreasing and the number of channels increasing sequentially. To fully utilize multi-scale feature information, these features are fused using FPN: In the formula It is a multi-scale feature after fusion. This represents the convolution operation. Indicates an upsampling operation; Finally, adaptive average pooling and fully connected layers are used to transform multi-scale features into fixed-dimensional feature vectors: In the formula Indicates adaptive average pooling. Indicates feature splicing, Indicates a fully connected layer. This is the final feature vector, with a dimension of 512; The noise step count prediction network is the core component of the adaptive noise step count optimization algorithm. It receives the feature vector output by the feature extractor and predicts the most suitable noise step count for the current image using a multilayer perceptron. Its specific structure is as follows: In the formula Indicates a fully connected layer. Indicates batch normalization, It is an activation function. It is the Sigmoid function. It is a predefined maximum number of noise steps. It is the number of noise steps predicted; the Sigmoid function restricts the output to the interval [0,1], multiplied by The final noise step count is obtained; To make noise step prediction more stable and reliable, regularization techniques and an attention mechanism were introduced. Regularization employs Dropout and weight decay to prevent overfitting; the attention mechanism helps the model focus on key regions in the image, improving prediction accuracy. The specific implementation is as follows: In the formula It is a spatial attention module. Indicates element-wise multiplication. It is a feature vector that has been weighted by attention; (3) In order to maximize the anomaly detection performance by predicting the noise step count, a special loss function was designed to guide the training of the noise step count prediction network; it consists of two parts: reconstruction loss and detection loss; Reconstruction loss measures the similarity between the reconstructed image and the original image in normal regions: In the formula It is the original image. It is to reconstruct the image. It is a binary mask for the abnormal region. Indicates element-wise multiplication; Detection loss measures the difference between the original image and the reconstructed image in anomalous regions: The total loss function is a weighted sum of two parts: In the formula and It is a weight hyperparameter used to balance the contributions of the two loss terms.

9. The method for tobacco leaf grading and defect detection based on interlayer adapter and diffusion model according to claim 8, characterized in that: Adaptive noise injection is a key step in integrating the adaptive denoising step concept of GLAD into the DiffusionAD framework. In the original GLAD model, the adaptive process mainly occurs in the denoising stage, and the appropriate stopping step is determined by comparing the intermediate results in the denoising process. Including a given input image and the predicted number of noise steps The adaptive noise injection process is represented as: In the formula , It is a predefined noise scheduling, It is standard Gaussian noise; Noise Dispatch It is an important hyperparameter in the diffusion model, which determines the rate at which noise is added; Linear noise scheduling is employed. In the formula , , It is the maximum number of noise steps; Adaptive noise injection exhibits different behaviors depending on the type of tobacco leaf defect: For color anomalies: Since color anomalies are mainly manifested as local tonal anomalies, the model tends to select a larger noise step number so that the color features of these areas are completely covered by Gaussian noise, and thus "corrected" to normal colors during the reconstruction process. For structural anomalies: Structural anomalies are usually manifested as physical damage on the surface of tobacco leaves. For these anomalies, an excessive number of noise steps may lead to the loss of important structural information, affecting subsequent anomaly detection. Therefore, the model tends to select a smaller number of noise steps to retain enough structural information for comparison with the reconstruction results. Furthermore, drawing on the local feature fusion concept from GLAD, the adaptive noise injection process was further optimized by introducing conditional information, enabling the noise injection process to better utilize image features. In the formula Based on feature vectors The modulation function is used to adjust the noise distribution according to image features; specifically... It is a learned function, represented as: In the formula It is the Sigmoid function, with an output range of (0,2), used to scale noise; Furthermore, a spatial attention mechanism is introduced during the noise injection stage, enabling the model to employ different noise strategies for normal and abnormal regions: In the formula It is a spatial attention map with values ​​ranging from [0,1]. For potentially anomalous regions, the attention value is higher, resulting in increased noise intensity; for normal regions, the attention value is lower, maintaining a moderate noise intensity.

10. The method for tobacco leaf grading and defect detection based on interlayer adapter and diffusion model according to claim 9, characterized in that: After completing adaptive noise injection and one-step denoising reconstruction, the reconstructed image was obtained. The difference between the reconstructed image and the original image is used to detect defects on the surface of tobacco leaves. To this end, a comprehensive difference evaluation module was designed, which considers multiple difference measures, including pixel-level difference, feature-level difference, and structural similarity difference. Pixel-level differences are directly calculated by measuring the Euclidean distance between the original and reconstructed images in pixel space: Feature-level difference comparison compares the distance between the original image and the reconstructed image in the feature space; image features are extracted using a pre-trained feature extractor, and then the distance between the feature vectors is calculated: In the formula Indicates the first Layer feature extractor, These are weighting coefficients used to balance the contributions of features from different layers; Structural similarity differences are calculated based on a structural similarity index, which takes into account the brightness, contrast, and structural information of the image. In the formula Calculate the structural similarity index between two images, ranging from [0,1]. The larger the value, the more similar the images are. therefore, A larger value indicates greater image differences; The formula for calculating SSIM is: In the formula and It is an image and The mean, and It is their variance. It is covariance. and It is a constant added to avoid the denominator being zero; The comprehensive anomaly score is obtained by weighted fusion of the above three differences: In the formula , and These are weighting coefficients, determined through validation set optimization. To further improve the accuracy of anomaly scoring, a self-attention mechanism was introduced, enabling the model to adaptively focus on key features of different types of anomalies. In the formula It is a self-attention module. It is attention weight. This represents three measures of difference.

Images