Tobacco leaf mildew detection method based on VMama visual state space model

Abstract

The invention discloses a tobacco leaf mildew detection method based on a VMama visual state space model, and the method comprises the steps: firstly collecting a tobacco leaf image through an industrial camera, and carrying out the preprocessing operation of size unification, data enhancement, normalization and the like; then, a deep learning model with VMama as a trunk network is constructed, and the model utilizes a core two-dimensional selective scanning module to efficiently capture the global feature and long-distance dependency relationship of the tobacco leaf image; thirdly, performing mixed precision training and optimization on the model by using the preprocessed data set; and finally, the trained model is used for real-time detection, rapid reasoning is carried out on a tobacco leaf image to be detected, and a judgment result about whether the tobacco leaf image is mildewed or not is output. According to the method, the advantages of high precision and high efficiency of the VMama model are fully utilized, the problems that a traditional method is poor in robustness and an existing deep learning model is high in calculation cost are effectively solved, tobacco leaf mildewing can be rapidly, accurately and automatically detected, and the method has extremely high industrial application value.

Description

Technical Field

[0001] This invention relates to the field of detection technology, specifically to a method for detecting mold growth in tobacco leaves based on the VMamba visual state space model. Background Technology

[0002] The quality of tobacco leaves is one of the decisive factors in cigarette production, directly affecting the taste and market value of the final product. During the acquisition, aging, storage, and processing of tobacco leaves, improper temperature and humidity control can easily lead to mold growth. Moldy tobacco leaves not only lose their economic value, but the mycotoxins they produce can also contaminate healthy tobacco leaves, causing serious losses to production and potentially endangering consumer health.

[0003] Currently, methods for detecting mold in tobacco leaves mainly rely on manual sorting and traditional machine vision technology. Manual sorting depends entirely on the worker's vision and sense of smell, which suffers from high subjectivity, low efficiency, and fatigue leading to missed or false detections, making it difficult to adapt to modern, large-scale tobacco processing. Traditional machine vision methods typically rely on threshold segmentation or classification based on fixed features such as color, texture, and spectrum. While this improves efficiency to some extent, it is highly sensitive to factors such as changes in lighting, the diversity of mold morphology, and background complexity, resulting in poor robustness and generalization ability of the models.

[0004] With the development of deep learning technology, detection methods represented by convolutional neural networks (CNNs) have been applied to the quality inspection of agricultural products. CNNs gradually expand their receptive field by stacking convolutional layers, but their core operations (such as 3x3 or 5x5 convolutional kernels) are essentially local receptive fields. This means that the model needs very deep network layers to indirectly capture the global information of the image. In this process, the precise spatial localization of subtle features such as mold spots may be lost, and it is difficult to efficiently establish feature associations for distant regions of the image (such as the ends of tobacco leaves).

[0005] To overcome the locality limitation of CNNs, models such as the Visual Transformer (ViT) have been introduced. ViT effectively captures global dependencies in images through its self-attention mechanism. However, its core self-attention mechanism suffers from a computational complexity (O(N²)) that quadratically increases with the number of input image patches. When processing high-resolution tobacco leaf images for fine-grained detection, the number of image patches increases dramatically, leading to enormous computational and memory consumption. This makes the training cost of ViT models high, and its inference speed insufficient to meet the real-time detection requirements of industrial production lines.

[0006] Therefore, there is an urgent need in this field for a novel model that combines the linearity of CNNs with the global modeling capabilities of Transformers. This model needs to effectively model long-range dependencies in images while maintaining low computational complexity, thereby achieving high-precision, high-efficiency, and real-time detection of tobacco mold, overcoming the shortcomings of existing technologies, and meeting the real-time quality inspection needs of automated production lines. Summary of the Invention

[0007] The purpose of this invention is to improve upon the shortcomings of existing technologies and provide a method for detecting mold growth in tobacco leaves based on the VMamba visual state space model. This method aims to address the problems of low accuracy and poor robustness in traditional detection methods, as well as the difficulty in balancing global feature capture and computational efficiency in existing deep learning models.

[0008] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0009] A method for detecting mold growth in tobacco leaves based on the VMamba visual state space model includes the following steps:

[0010] S1. Image data acquisition and preprocessing: Acquire and process tobacco leaf images to generate standardized numerical tensors;

[0011] S2. Construct a deep learning model with the VMamba visual state space model as the backbone network. The model includes an image patch embedding module, a VMamba core feature extraction module, and a classification decision module.

[0012] S3. Use the standardized numerical tensor to train and optimize the deep learning model to obtain a trained mold detection model.

[0013] S4. Input the image of the tobacco leaf to be tested into the trained mold detection model to output the classification prediction result.

[0014] Furthermore, in step S1, the image data acquisition and preprocessing, which involves acquiring and processing tobacco leaf images to generate standardized numerical tensors, specifically includes:

[0015] S1.1 On the industrial production line conveyor belt, high-resolution industrial cameras are used to collect tobacco leaf image data to construct an original dataset containing normal tobacco leaves and moldy tobacco leaves.

[0016] S1.2 Perform preprocessing operations on the acquired images, including: uniformly adjusting the image size to meet the model input requirements; performing data augmentation by expanding the dataset through random rotation, flipping, cropping, brightness and contrast adjustment, etc., to improve the model's generalization ability.

[0017] S1.3 Normalize the image to convert the image data into a standardized numerical tensor to accelerate model convergence and adapt to the input requirements of the pre-trained model.

[0018] Furthermore, in step S2, constructing a deep learning model with the VMamba visual state space model as the backbone network specifically includes:

[0019] S2.1 Construct an image patch embedding module, which is responsible for segmenting a two-dimensional image into image patches and converting them into a one-dimensional feature sequence through linear embedding;

[0020] S2.2 Construct the VMamba core feature extraction module, which consists of multiple stacked VSS Blocks, and each VSS Block contains an SS2D module for deep global context modeling of feature sequences.

[0021] S2.3 Construct a classification decision module, which converts the extracted features into the final classification probability output through global average pooling and fully connected layers.

[0022] Furthermore, in step S2.2, the SS2D module expands the two-dimensional features into a one-dimensional sequence along multiple directions and processes the sequence using a state-space model to capture global context information.

[0023] Furthermore, in step S3, the deep learning model is trained and optimized using the standardized numerical tensor, which specifically includes:

[0024] S3.1 Training Configuration and Initialization: This includes loading the pre-trained model, defining the binary cross-entropy loss function, selecting the AdamW optimizer, and splitting the dataset.

[0025] S3.2, Dynamic learning rate adjustment strategy: A cosine annealing learning rate scheduler is used to achieve an efficient and stable training process;

[0026] S3.3, Iterative supervised training: The model is iteratively learned through a loop of forward propagation, loss calculation, gradient backpropagation, and parameter update;

[0027] S3.4 Training Acceleration and Efficiency Optimization: An automatic mixed precision training strategy is adopted to improve training speed and reduce resource consumption;

[0028] S3.5 Model Validation and Optimization: During the training process, the model is periodically evaluated on the validation set, and the model with the best performance is saved as the final model, which is the trained mold detection model.

[0029] Furthermore, in step S4, the step of inputting the image of the tobacco leaf to be tested into the trained mold detection model to output a classification prediction result specifically includes:

[0030] S4.1 Model Deployment and Integration: Optimize the trained model and deploy it to the inference device, integrating it with the industrial inspection system's hardware and software;

[0031] S4.2 Real-time image processing and model input: Perform deterministic preprocessing on single tobacco leaf images acquired online, consistent with the training phase;

[0032] S4.3, Forward Inference and Probability Output: The model performs fast forward calculation on the input image tensor and outputs the probability value that it is moldy tobacco leaf.

[0033] S4.4 Decision-making and result execution: Compare the probability output by the model with the preset threshold, make the final judgment, and link the physical execution mechanism to remove the moldy tobacco leaves.

[0034] Furthermore, in step S4.1, the trained model is compiled and optimized using NVIDIA TensorRT and then deployed to an edge computing device.

[0035] Compared with the prior art, the beneficial effects of the present invention are:

[0036] 1. This invention leverages the powerful global and long-distance dependency modeling capabilities of the VMamba model. This method can accurately identify mold spots of various shapes, sizes, and locations, effectively distinguishing mold spots from the spots and wrinkles on the tobacco leaves themselves, significantly reducing the false negative and false positive rates, thus giving it high accuracy and strong robustness.

[0037] 2. In this invention, the core SS2D module of the VMamba model has linear computational complexity. Compared with the visual Transformer model, it has lower computational cost and faster inference speed when processing high-resolution images, making it more efficient and able to meet the real-time requirements of industrial production lines.

[0038] 3. Through rich data augmentation strategies and advanced model architecture, the model trained by this method has excellent generalization ability and can adapt to the task of detecting mold in different batches and varieties of tobacco leaves.

[0039] 4. This method automates the entire process from image acquisition to result output, and can be seamlessly integrated into existing tobacco leaf sorting lines to replace manual quality inspection, significantly improving production efficiency and reducing labor costs for enterprises. Attached Figure Description

[0040] Figure 1This is a flowchart of the tobacco leaf mold detection method described in this invention.

[0041] Figure 2 This is a schematic diagram of the internal flow of the VMamba core VSS module in this invention.

[0042] Figure 3 This is a conceptual schematic diagram of the two-dimensional selective scanning (SS2D) mechanism in this invention.

[0043] Figure 4 The graph shows the model evaluation index results in Example 2. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0045] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0046] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0047] Please see Figure 1 This invention provides a method for detecting mold growth in tobacco leaves based on the VMamba visual state space model, comprising the following steps:

[0048] S1. Image data acquisition and preprocessing: Acquire and process tobacco leaf images to generate standardized numerical tensors;

[0049] Specifically, the steps include:

[0050] S1.1 Data Acquisition: A high-resolution industrial camera KS2A543-3.0 is fixedly installed above the tobacco leaf sorting line. Under fixed lighting conditions (LED light source color temperature 5500K, illuminance 1200 Lux), images of normal and moldy tobacco leaves are acquired to construct an original dataset containing normal and moldy tobacco leaves. During acquisition, it is necessary to ensure that the image background is consistent (sorting machine conveyor belt) to avoid environmental interference.

[0051] S1.2 Perform preprocessing operations on the acquired images, including: uniformly adjusting the image size to meet the model input requirements; performing data augmentation by expanding the dataset through random rotation, flipping, cropping, brightness and contrast adjustment, etc., to improve the model's generalization ability.

[0052] Preferably, in this step, all acquired images are uniformly scaled to 224×224 pixels to match the model input size. Online data augmentation is performed on the training set images, including: random horizontal and vertical flipping with a 50% probability; random rotation within the range of [-15°, 15°]; and small-range random perturbations to the image brightness, contrast, and saturation.

[0053] S1.3 Normalize the image to convert the image data into a standardized numerical tensor to accelerate model convergence and adapt to the input requirements of the pre-trained model.

[0054] In this embodiment, the normalization step employs a standardization method. The pixel values ​​of the data-enhanced image are first converted to floating-point numbers in the range [0.0, 1.0]. Then, each color channel is standardized using the official mean [0.485, 0.456, 0.406] and standard deviation [0.229, 0.224, 0.225] of the ImageNet dataset. This step ensures that the tobacco leaf images input to the model maintain the same data distribution as during the VMamba model pre-training, which is crucial for successful transfer learning. Finally, each image is converted into a 3×224×224 floating-point tensor.

[0055] S2. Construct a deep learning model with the VMamba visual state space model as the backbone network. The model includes an image patch embedding module, a VMamba core feature extraction module, and a classification decision module.

[0056] Specifically, the steps include:

[0057] S2.1 Construct an image patch embedding module responsible for segmenting a 2D image into image patches and converting them into a 1D feature sequence through linear embedding. In this step, the input 3 x 224 x 224 image is processed by a 2D convolutional layer with a kernel size of 16x16 and a stride of 16. This converts the image into a sequence of 196 D-dimensional (e.g., D=96) feature vectors.

[0058] S2.2 Construct the VMamba core feature extraction module, which consists of multiple stacked Visual State Space (VSS) Blocks. Each VSS Block contains a two-dimensional selective scanning (SS2D) module. The VSS module flow and SS2D module mechanism are as follows: Figure 2-3 As shown, the SS2D module inside each Block adopts a four-directional scanning mechanism of horizontal, vertical and reverse to expand the two-dimensional features into a one-dimensional sequence along multiple directions, and uses a state space model to process the sequence to ensure that the tobacco leaf image features are modeled globally without dead angles, so as to perform in-depth global context modeling of the feature sequence.

[0059] S2.3 Construct a classification decision module. Through global average pooling and fully connected layers, the extracted features are converted into the final classification probability output. More specifically, after the last VSS Block, a global average pooling layer is connected to aggregate the feature maps into a D-dimensional vector. Subsequently, this vector is fed into a fully connected layer with 2 output nodes, and finally outputs the probabilities of "normal" and "moldy" through the Softmax function.

[0060] S3. Use the standardized numerical tensor to train and optimize the deep learning model to obtain a trained mold detection model.

[0061] Specifically, the steps include:

[0062] S3.1 Training Configuration and Initialization: This includes loading the pre-trained model, defining the binary cross-entropy loss function, selecting the AdamW optimizer, and splitting the dataset. The initial learning rate is 0.0005, the weight decay is 0.05, and the dataset is divided into training, validation, and test sets in a ratio of 8:1:1.

[0063] S3.2, Dynamic learning rate adjustment strategy: A cosine annealing learning rate scheduler is adopted. Preferably, the learning rate is smoothly reduced from the initial value to near zero within 100 training rounds to achieve an efficient and stable training process.

[0064] S3.3 Iterative supervised training: The model is iteratively learned through a loop of forward propagation, loss calculation, gradient backpropagation, and parameter update.

[0065] In practical use, the batch size is 64, and 100 rounds of iterative training are performed on the training set. In each iteration step, the model completes one forward propagation, loss calculation, backpropagation and parameter update.

[0066] S3.4 Training Acceleration and Efficiency Optimization: The Automatic Mixed Precision (AMP) training strategy built into the PyTorch framework is adopted to train in FP16 / FP32 mixed mode to improve training speed and reduce resource consumption.

[0067] S3.5 Model Validation and Optimization: After each round of training, the F1-Score of the model is calculated on the validation set. During the training process, the model is periodically evaluated on the validation set. The model weight corresponding to the round with the highest F1-Score is recorded and saved as the final model, which is the trained mold detection model.

[0068] S4. Input the image of the tobacco leaf to be tested into the trained mold detection model to output the classification prediction result.

[0069] Specifically, the steps include:

[0070] S4.1 Model Deployment and Integration: The trained model is optimized and deployed to the inference device, integrated with the industrial inspection system's hardware and software. Preferably, NVIDIA TensorRT is used for compilation and optimization during this process to generate a highly optimized inference engine. This engine is deployed on an NVIDIA Jetson AGX Orin edge computing device located next to the production line.

[0071] S4.2 Real-time image processing and model input: When the photoelectric sensor detects tobacco leaves, it triggers the camera to take a picture. The acquired image is immediately scaled to 224×224 pixels. Then, the online acquired single tobacco leaf image is subjected to deterministic preprocessing consistent with the training phase, that is, ImageNet mean and standard deviation are normalized.

[0072] S4.3 Model Forward Inference and Probability Output: The model performs fast forward computation on the input image tensor and outputs the probability value of it being moldy tobacco leaves. During implementation, the preprocessed image tensor is fed into the TensorRT engine for inference. The average time of the entire process (from image acquisition to probability output) is controlled within 30 milliseconds. The model outputs a two-dimensional probability vector, such as [0.5, 0.95], representing the probabilities of normal and moldy tobacco leaves, respectively.

[0073] S4.4 Decision Judgment and Result Execution: The probability output by the model is compared with a preset threshold to make a final judgment, and the physical execution mechanism is activated to remove the moldy tobacco leaves. In this embodiment, the judgment threshold is set to 0.5. The system extracts a mold probability of 0.95. If the value is greater than 0.5, it is judged as "moldy tobacco leaf". This result is recorded, and at the same time, the edge computing device sends a high-level signal to the PLC through the GPIO interface. The PLC controls a high-speed electromagnetic valve to spray air instantaneously, blowing the moldy tobacco leaf away from the main conveyor belt.

[0074] Example 2:

[0075] To evaluate the overall performance of the tobacco leaf mold detection method described in this invention in actual tobacco leaf mold detection tasks, the following experiments were conducted:

[0076] Experimental Setup: A tobacco leaf image dataset collected and constructed in S1 was used. This dataset contains 10,000 high-resolution images, with normal tobacco leaves and moldy tobacco leaves each accounting for approximately 50%. The dataset was strictly divided into a training set (8,000 images), a validation set (1,000 images), and a test set (1,000 images) in an 8:1:1 ratio to ensure data independence during training, validation, and testing for fair performance evaluation. The experiment used accuracy, precision, recall, and F1-score to measure model performance. The results obtained after conducting experiments using this invention are as follows: Figure 4 The model evaluation index results are shown in the figure.

[0077] The results show that the VMamba-based visual state space model can effectively identify moldy tobacco leaves and achieve high detection accuracy.

[0078] Example 3:

[0079] To highlight the method of this invention, two representative deep learning models were selected as baselines for comparative experiments:

[0080] Experimental Setup: ResNet50 and ViT-B / 16 (Vision Transformer) were selected as baselines for comparative experiments. ResNet50 is a classic and widely used convolutional neural network (CNN) model, representing a detection method based on local receptive fields. ViT-B / 16 is a Transformer model based on a self-attention mechanism with a global receptive field, representing another mainstream vision architecture. The experiment also used accuracy, precision, recall, and F1-score as evaluation metrics to measure model performance, resulting in the experimental data shown in Table 1.

[0081] Model Accuracy (%) Accuracy (%) Recall rate (%) F1 score Average inference time per graph (ms) ResNet50 97.2% 96.5% 97.8% 0.971 28ms ViT-B / 16 98.6% 98.1% 99.0% 0.985 55 ms The method of this invention (VMamba-S) 99.6% 99.2% 99.9% 0.995 25 ms

[0082] Table 1. Performance Comparison Experiment Results of ResNet50, ViT-B / 16, and the Model of the Present Invention

[0083] As shown in Table 1, the VMamba-based detection method proposed in this invention performs well in all key performance indicators, especially in recall and F1-Score, which are significantly better than the comparison model, demonstrating its high efficiency and robustness in tobacco mold detection.

[0084] The experimental results demonstrate that the method of this invention achieves near-perfect performance in tobacco leaf mold detection. Compared with ResNet50, this method shows significant improvements in all metrics, thanks to the powerful global context modeling capability of the VMamba model, which effectively overcomes the limitations of the local receptive field in CNNs. Compared with the ViT model, which also has a global receptive field, the method of this invention not only slightly surpasses it in accuracy, but more importantly, its inference speed is more than twice as fast, fully demonstrating its unique advantages of combining high accuracy and high efficiency, perfectly meeting the application requirements of real-time detection in industrial production lines.

[0085] Although the invention has been described herein with reference to several illustrative embodiments, it should be understood that many other modifications and implementations can be devised by those skilled in the art, which will fall within the scope and spirit of the principles disclosed herein. More specifically, various modifications and improvements can be made to the components or layout of the subject matter arrangement within the scope of the disclosure, drawings, and claims. Besides modifications and improvements to the components or layout, other uses will be apparent to those skilled in the art.

Claims

1. A method for detecting tobacco leaf mildew based on a VMamba visual state space model, characterized in that, The method comprises the following steps: S1, image data acquisition and preprocessing, acquiring and processing tobacco leaf images to generate standardized numerical tensors; S2, a deep learning model with a VMamba visual state space model as the backbone network is constructed, which includes an image block embedding module, a VMamba core feature extraction module and a classification decision module; S3, using the standardized numerical tensors to train and optimize the deep learning model, obtaining a trained mold detection model; S4, inputting the image of the tobacco leaf to be tested into the trained mold detection model to output a classification prediction result.

2. The method of claim 1, wherein the method is based on a VMamba visual state space model. In step S1, the image data acquisition and preprocessing, acquiring and processing tobacco leaf images to generate standardized numerical tensors, specifically includes: S1.1, using a high-resolution industrial camera to collect tobacco leaf image data on an industrial pipeline conveyor belt, and constructing an original data set containing normal tobacco leaves and moldy tobacco leaves; S1.2, performing preprocessing operations on the collected images, including: uniformly adjusting the image size to adapt to the model input requirements; performing data enhancement, expanding the data set through random rotation, flipping, cropping, brightness and contrast adjustment, etc., to improve the generalization ability of the model; S1.3, normalizing the image, converting the image data into a standardized numerical tensor to speed up model convergence and adapt to the input requirements of the pre-trained model.

3. The method of claim 1, wherein the method is based on a VMamba visual state space model. In step S2, the deep learning model with a VMamba visual state space model as the backbone network is constructed, specifically including: S2.1, constructing an image block embedding module, which is responsible for dividing a two-dimensional image into image blocks and converting them into one-dimensional feature sequences through linear embedding; S2.2, constructing a VMamba core feature extraction module, which is composed of multiple stacked VSS Blocks, and the VSS Block contains an SS2D module inside, which is used for deep global context modeling of feature sequences; S2.3, constructing a classification decision module, which converts the extracted features into final classification probability output through global average pooling and fully connected layers.

4. The method of claim 3, wherein the method is based on a VMamba visual state space model. In step S2.2, the SS2D module expands two-dimensional features into one-dimensional sequences along multiple directions, and processes the sequences using a state space model to capture global context information.

5. The method of claim 1, wherein the method is based on a VMamba visual state space model. In step S3, the standardized numerical tensors are used to train and optimize the deep learning model, which specifically includes: S3.1, training configuration and initialization: including loading a pre-trained model, defining a binary cross-entropy loss function, selecting an AdamW optimizer and dividing a data set; S3.2, learning rate dynamic adjustment strategy: adopting a cosine annealing learning rate scheduler to achieve an efficient and stable training process; S3.3, iterative supervised training: through the cycle of forward propagation, loss calculation, gradient back propagation and parameter update, the model is iteratively learned; S3.4, training acceleration and efficiency optimization: adopting an automatic mixed precision training strategy to improve training speed and reduce resource consumption; S3.5, model verification and optimization: periodically evaluate the model on the validation set during training, and save the best model as the final model, which is the trained mold detection model.

6. The method of claim 1, wherein the method is based on a VMamba visual state space model. In step S4, the image of the tobacco to be tested is input into the trained mold detection model to output a classification prediction result, which specifically includes: S4.1, model deployment and integration: optimize the trained model and deploy it to the inference device, and integrate it with the software and hardware of the industrial detection system; S4.2, real-time image processing and model input: perform consistent deterministic preprocessing on the single tobacco image collected online as in the training phase; S4.3, model forward inference and probability output: the model performs fast forward calculation on the input image tensor and outputs the probability value that it is moldy tobacco; S4.4, decision making and result execution: compare the probability output by the model with the preset threshold value, make a final decision, and link the physical execution mechanism to remove the moldy tobacco.

7. The method of claim 1, wherein the method is based on a VMamba visual state space model. In step S4.1, the trained model is compiled and optimized using NVIDIA TensorRT, and deployed to the edge computing device.

Images