Tobacco leaf multi-granularity identification method based on semantic segmentation
By constructing a multi-granularity semantic segmentation dataset and model architecture, the problems of semantic ambiguity and weak scene adaptability in tobacco leaf recognition technology are solved, realizing multi-granularity recognition and automated graded diagnosis of tobacco leaves, and improving recognition accuracy and efficiency.
Patent Information
- Application Number
- CN202511807464.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-03
- Publication Date
- 2026-02-27
AI Technical Summary
Existing tobacco leaf recognition technologies suffer from problems such as semantic ambiguity, single granularity, weak scene adaptability, and low data utilization efficiency. They cannot output multi-granularity semantic results simultaneously, and their recognition accuracy is not high in complex environments, making it difficult to meet the needs of automated tobacco leaf grading and mold diagnosis.
A multi-granularity semantic segmentation dataset for tobacco leaves was constructed. A model architecture consisting of a visual encoder, a decoupled text encoder, and a multi-granularity deformable mask decoder was adopted. Combined with a multi-selection learning mechanism and a field scene adaptation enhancement strategy, the data with semantic annotation and without semantic mask were jointly trained through a many-to-many Hungarian matching strategy to achieve multi-granularity semantic recognition.
It achieves multi-granular semantic recognition of different growth stages, morphology and parts of tobacco leaves, improves recognition accuracy and model generalization ability in complex environments, supports automated grading and mold diagnosis, and significantly reduces annotation costs and missed diagnosis rate.
Smart Images
Figure CN121582674A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image recognition technology, and more specifically to a multi-granularity recognition method for tobacco leaves based on semantic segmentation. Background Technology
[0002] As tobacco is an important economic crop, monitoring its growth status, grading, and diagnosing mold are crucial aspects of tobacco agricultural production. Traditional tobacco leaf identification and analysis rely on manual operation: people judge the growth stage of tobacco leaves, classify tobacco leaves (such as according to leaf size and integrity), and identify mold types by visual inspection. This method suffers from problems such as low efficiency, strong subjectivity, large errors, and dependence on professional experience, making it difficult to meet the automation needs of large-scale tobacco planting.
[0003] With the development of computer vision technology, existing technologies attempt to achieve automated tobacco leaf identification through image recognition methods, but they suffer from the following core drawbacks:
[0004] 1. Semantic ambiguity and single granularity: Existing models mostly focus on single-granularity recognition of "intact tobacco leaf" or "moldy area", and cannot output multi-granularity semantic results at the same time. For example, when clicking on the leaf tip area, only "leaf tip" is recognized, and it cannot be associated with outputting coarser-grained semantics such as "leaf leaf" or "intact tobacco leaf", or finer-grained semantics such as "moldy spots on leaf tip", which makes the recognition results unable to meet the multi-dimensional needs of grading, diagnosis and other scenarios;
[0005] 2. Poor scene adaptability: The tobacco growing environment is complex (different varieties, growth stages, light intensity, field shading). Existing models are mostly trained on single scene datasets, which are not robust to changes in light intensity, leaf overlap, and small spots in the early stage of mold. The recognition accuracy is easily affected by environmental interference.
[0006] 3. Semantic and segmentation disconnect: Some models can only output segmentation masks (without semantic labels), requiring manual secondary judgment of the tobacco leaf parts or states corresponding to the masks; other models have semantic labels, but only support fixed granularity (e.g., only recognize "moldy tobacco leaves", unable to distinguish "moldy type" or "affected leaf parts"), and cannot achieve "segmentation equals understanding".
[0007] 4. Low data utilization efficiency: The tobacco field lacks large-scale multi-granularity labeled datasets. Existing models can only utilize a small amount of semantically labeled data and cannot effectively integrate multi-granularity masked data without semantic labels (such as a large number of field tobacco leaf images), which limits the generalization ability of the models.
[0008] While existing semantic segmentation technologies (such as SAM and Mask DINO) have achieved multi-granularity segmentation or semantic recognition in general scenarios, these general models are not optimized for tobacco leaf scenarios: SAM can only output multi-granularity masks but lacks semantic labels, making it unable to distinguish between "tobacco leaf tips" and "weed leaf tips"; Mask DINO, although supporting semantic segmentation, has a single granularity and cannot adapt to the multi-granularity requirements of tobacco leaves, ranging from "tobacco plant" to "mold spots." Therefore, there is an urgent need for a recognition method optimized for tobacco leaf scenarios that combines semantic awareness and granularity abundance. Summary of the Invention
[0009] This invention aims to solve the problems of "semantic ambiguity, single granularity, weak scene adaptability, and low data utilization efficiency" in existing tobacco leaf identification technologies. It achieves multi-granular semantic identification of tobacco leaves at different growth stages, in different forms (intact / damaged), in different parts (leaf tip / leaf edge / leaf vein), and in moldy areas, providing technical support for automated tobacco leaf grading and early diagnosis of mold.
[0010] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:
[0011] A multi-granularity tobacco leaf identification method based on semantic segmentation includes the following steps:
[0012] S1. Construct a multi-granularity semantic segmentation dataset for tobacco leaves: Collect images of tobacco leaves of different varieties, growth stages, and scenes, label them with multi-level granularity semantic labels and non-semantic multi-granularity masks, and perform data augmentation to adapt to field scenes.
[0013] S2. Construct a multi-granularity semantic segmentation model for tobacco leaves: This model includes a visual encoder, a decoupled text encoder, and a multi-granularity deformable mask decoder. Through this model, user input is converted into anchor boxes, generating K multi-granularity content embeddings, and outputting the semantic label and mask for each granularity.
[0014] S3. Model Training: Design a loss function and use a many-to-many Hungarian matching strategy to jointly train two types of data: those with semantic annotations and those without semantic masks.
[0015] S4. Multi-granularity inference application: After receiving the user's selection input instruction, the model outputs multi-granularity semantic segmentation results, which are integrated into the tobacco intelligent monitoring system for grading and mold diagnosis.
[0016] Furthermore, in step S1, the three levels of semantic tags—coarse, medium, and fine—are specifically as follows:
[0017] Coarse-grained: Semantic tags include "whole tobacco leaves" and "coarsely crushed tobacco leaves";
[0018] Medium granularity: Semantic tags include "medium broken tobacco leaves";
[0019] Fine-grained: Semantic tags include "fragmented tobacco leaves" and "mold spots";
[0020] The semantically nullable multi-granularity mask is labeled using the SA-1B dataset, and the labeled semantically nullable multi-granularity mask is not associated with semantic labels.
[0021] Furthermore, in step S1, the data augmentation for adapting to the field scene specifically involves: using enhancement strategies such as illumination perturbation, geometric transformation, and noise addition to improve the model's anti-interference ability in response to interference in the field scene.
[0022] Furthermore, in step S2, the construction of the multi-granularity semantic segmentation model for tobacco leaves specifically includes:
[0023] S2.1 Visual Encoder: The pre-trained Swin Transformer-L is used as the visual backbone to extract multi-scale visual features of tobacco leaf images. Its input is a 1024×1024 tobacco leaf image, and its output is a multi-scale feature map.
[0024] S2.2 Decoupled Text Encoder: Based on the UniCL pre-trained text model, a decoupled semantic encoding module specifically for tobacco leaves is constructed. Its input is the semantic label of tobacco leaves, and its output is two types of embedding vectors: object embedding and component embedding.
[0025] S2.3 Multi-granularity deformable mask decoder: To address the multi-granularity requirements of tobacco leaves, a deformable mask decoder based on multi-selection learning is designed. This decoder converts user interaction input into unified anchor boxes, generating K content embedding vectors for each anchor box, with each embedding vector corresponding to a granularity level.
[0026]
[0027] in, For granular-level embedding, Embed for input type;
[0028] The deformable mask decoder employs the deformable attention mechanism DeformDec, combining visual features, positional embedding, and multi-granularity content embedding to output the prediction result at each granularity, formally represented as:
[0029]
[0030] Where O is the DeformDec output set containing K content embedding vectors, b is the anchor box, F is the visual encoder output feature, and o i c is the prediction tuple for the i-th granularity; iFor semantic label prediction at the i-th granularity; m i Let be the segmentation mask for the i-th granularity.
[0031] Furthermore, in step S3, the design loss function specifically includes mask loss, semantic classification loss, and semantic data loss. The mask loss uses Dice loss to calculate the overlap between the predicted mask and the standard mask; the semantic classification loss uses cross-entropy loss to calculate the object classification loss and the component classification loss respectively; the semantic data loss only calculates the mask loss, using unlabeled multi-granularity mask data to improve segmentation capability.
[0032] Furthermore, in step S3, the joint training includes two types of data: semantically labeled data and data without semantic masks. Specifically, it includes first using data without semantic multi-granularity masks to pre-train the model's segmentation capability, and then adding data with semantic labels for joint optimization. The optimizer is AdamW with a learning rate of 1e-4, which decays by a factor of 10 when the training steps reach 90%-95%.
[0033] Furthermore, in step S4, the model outputs multi-granularity semantic segmentation results including K different granularities of "semantic labels + segmentation masks".
[0034] Compared with the prior art, the beneficial effects of the present invention are:
[0035] 1. Multi-granular semantic recognition capability:
[0036] By constructing a three-module architecture of "visual encoder - decoupled text encoder - multi-granularity deformable mask decoder" and combining it with a multi-selection learning mechanism, full-granularity semantic recognition of tobacco leaves is achieved, from "tobacco plant" (coarse-grained) to "leaf part" (medium-grained) and then to "mold spots" (fine-grained). Specifically, this is manifested in:
[0037] (1) Identification of different growth stages: Through multi-scale visual feature extraction (Swin Transformer-L), the characteristics of tobacco leaves at different growth stages such as seedling stage, vigorous growth stage, and maturity stage can be accurately identified;
[0038] (2) Different morphology recognition: By decoupling the text encoder, the independent modeling of "tobacco leaf object-component-mold type" is realized, which can accurately distinguish different morphologies such as complete tobacco leaves, coarse broken tobacco leaves, medium broken tobacco leaves, and fine broken tobacco leaves;
[0039] (3) Different parts identification: Through a multi-granularity content embedding generation mechanism, accurate positioning and semantic annotation of different parts such as leaf tip, leaf edge, and leaf vein are achieved;
[0040] (4) Mold area identification: Through fine-grained decoding capabilities, accurately identify and locate diseased areas such as mold spots.
[0041] A single inference iteration can output a complete semantic segmentation result with a 3-level granularity, completely solving the problems of "single granularity and ambiguous semantics" in existing technologies.
[0042] 2. High scene adaptability:
[0043] By constructing a hybrid dataset specifically for tobacco leaves, encompassing different varieties, growth stages, and field scenarios, and employing field scenario adaptation and enhancement strategies such as illumination perturbation, geometric transformation, and noise addition, combined with an anti-interference design using a deformable attention mechanism, the model exhibits strong robustness to complex environments. Its recognition accuracy in complex scenarios such as varying illumination, overlapping leaves, and small spots in the early stages of mold is significantly superior to existing tobacco leaf recognition technologies, effectively addressing the problem of "weak scene adaptability" in existing technologies.
[0044] 3. High data utilization efficiency:
[0045] By innovatively designing a many-to-many Hungarian matching strategy, joint training of "semantically labeled data + semantically unlabeled multi-granularity masked data" is achieved: first, the multi-granularity segmentation capability of the model is pre-trained using semantically unlabeled data (similar to the SA-1B labeling method), and then semantically labeled data is added to jointly optimize the classification loss. This training strategy makes full use of a large amount of readily available semantically unlabeled data, solving the problem of "lack of large-scale multi-granularity labeled data" in the tobacco industry, significantly reducing labeling costs, and improving the model's generalization ability.
[0046] 4. Automation and Practicality:
[0047] It supports interactive input such as clicking and selection, and outputs K semantic tags + segmentation masks of different granularities in a single inference, which can be directly integrated into the intelligent tobacco leaf monitoring system.
[0048] (1) Application of tobacco leaf grading: Automatically classify leaves according to leaf integrity (coarse grain) and component condition (medium grain), greatly improving grading efficiency;
[0049] (2) Application of mold diagnosis: Based on fine-grained semantic tags, locate the location of mold, identify the type, and calculate
[0050] The affected area significantly reduces the rate of missed diagnoses.
[0051] It effectively replaces some manual operations, significantly improves work efficiency, and reduces labor costs in tobacco planting. Attached Figure Description
[0052] Figure 1 This is a flowchart of the multi-particle size identification method for tobacco leaves described in this invention.
[0053] Figure 2The diagrams show in detail the architecture and data flow of the three modules: the visual encoder, the decoupled text encoder, and the multi-granularity deformable mask decoder.
[0054] Figure 3 The illustration shows the effect of semantic labeling at three levels of granularity, with multi-level annotation examples ranging from whole objects to tiny spots, including annotations at different granularities such as whole tobacco leaves, tobacco leaf parts, broken tobacco leaves, and mold spots. Detailed Implementation
[0055] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.
[0056] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.
[0057] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.
[0058] Please see Figure 1-3 As shown, this invention provides a multi-granularity tobacco leaf identification method based on semantic segmentation, comprising the following steps:
[0059] S1. Construct a multi-granularity semantic segmentation dataset for tobacco leaves: Collect tobacco leaf images of different varieties, growth stages, and scenes, label them with multi-level granularity semantic labels and non-semantic multi-granularity masks, and perform data augmentation to adapt to field scenes.
[0060] Specifically, the steps include:
[0061] S1.1 Collect tobacco leaf images of different tobacco varieties (such as flue-cured tobacco and burley tobacco), different growth stages (seedling stage, vigorous growth stage, maturity stage), and different field scenes (sunny day, cloudy day, with or without obstruction). The collected tobacco varieties include complete tobacco leaves and damaged tobacco leaves (including mechanical damage and mold). The image resolution is uniformly set to 1024×1024 to adapt to the subsequent model input.
[0062] S1.2 Multi-granularity semantic annotation: A three-level granularity annotation system is adopted for semantic annotation, refer to... Figure 3 As shown, the annotations are as follows:
[0063] Coarse-grained: Semantic tags include "whole tobacco leaves" and "coarsely crushed tobacco leaves";
[0064] Medium granularity: Semantic tags include "medium broken tobacco leaves";
[0065] Fine-grained: Semantic tags include "fragmented tobacco leaves" and "mold spots," etc.
[0066] S1.3, Multi-granularity mask annotation without semantics: A multi-granularity mask annotation method similar to SA-1B is used to annotate some tobacco leaf images without semantics with masks of different granularities. It should be noted that the annotation is not associated with semantic labels, but only records the "region range containing the target", which is used to expand the multi-granularity segmentation capability.
[0067] S1.4 Data Augmentation: To address interference in field scenes, augmentation strategies such as illumination perturbation, geometric transformation, and noise addition are adopted to improve the model's anti-interference ability. Furthermore, the illumination perturbation range is: brightness ±30%, contrast ±20%, geometric transformation includes rotation ±15°, cropping, and horizontal flipping, and the noise addition parameter is: Gaussian noise σ=0.01.
[0068] S2. Construct a multi-granularity semantic segmentation model for tobacco leaves: This model includes a visual encoder, a decoupled text encoder, and a multi-granularity deformable mask decoder. The principle framework of this model can be found in [reference needed]. Figure 2 As shown, this model converts user input into anchor boxes, generates K multi-granularity content embeddings, and outputs semantic labels and masks for each granularity.
[0069] Specifically, the steps include:
[0070] S2.1 Visual Encoder: The pre-trained Swin Transformer-L is used as the visual backbone to extract multi-scale visual features from tobacco leaf images. Its input is a 1024×1024 tobacco leaf image, and its output is a multi-scale feature map (such as 4×4, 8×8, 16×16, 32×32 resolution), which allows it to capture visual information of tobacco leaves from "overall shape" to "detail texture" (such as mold spots);
[0071] S2.2 Decoupled Text Encoder: Based on the UniCL pre-trained text model, a decoupled semantic encoding module specifically for tobacco leaves is constructed to achieve independent semantic modeling of "tobacco leaf object - tobacco leaf component - mold type".
[0072] Input: Semantic tags for tobacco leaves, such as "tobacco leaf tip" and "tobacco leaf mold spots".
[0073] Output: Two types of embedding vectors:
[0074] Object embedding: corresponds to "tobacco leaf object" (such as "intact tobacco leaf" or "moldy tobacco leaf");
[0075] Component embedding: corresponds to "tobacco leaf component / mildew area" (such as "leaf tip" or "mildew spot");
[0076] By decoupling the text encoder, semantic sharing of the same component can be achieved under different objects (e.g., "leaf tip" can be associated with "complete tobacco leaf tip" and "moldy tobacco leaf tip"), thereby improving semantic transfer capabilities;
[0077] S2.3. To address the multi-granularity requirements of tobacco leaves, a deformable mask decoder based on multi-selection learning is designed to achieve "single input, multi-granularity output":
[0078] Input processing: Convert user interaction input (including click input and selection input) into a unified anchor box:
[0079] In practice: Click to input (x,y): convert to a small anchor frame (x,y,w,h), where w=h=16 pixels (to fit the tobacco leaf detail area);
[0080] Select the input (x1, y1, x2, y2): it will be used directly as the anchor box;
[0081] Multi-granularity embedding generation: For each anchor box, K (K=3 in this example, corresponding to 3 levels of granularity in tobacco leaves) content embedding vectors are generated, and each embedding vector corresponds to a granularity level:
[0082]
[0083] in, Embedded according to particle size (e.g., i=1 corresponds to the fine particle size of "mold spots", i=3 corresponds to the coarse particle size of "intact tobacco leaf"). Embed input types (distinguish between "click input" and "selection input");
[0084] Deformable Decoding: Employing the deformable attention mechanism DeformDec, which combines visual features, positional embeddings, and multi-granularity content embeddings, the prediction results at each granularity are formally represented as follows:
[0085]
[0086] Where O is a subset of K content embedding vectors. Output set; o i c is the prediction tuple for the i-th granularity; i For semantic label prediction at the i-th granularity; m i Let be the segmentation mask for the i-th granularity; Q be the query vector set (K content embedding vectors); b be the anchor box; F be the output features of the visual encoder; c be the segmentation mask for the i-th granularity; i This is the semantic label prediction for the i-th granularity.
[0087] S3. Model Training: Design a loss function and use a many-to-many Hungarian matching strategy to jointly train two types of data: those with semantic annotations and those without semantic masks.
[0088] Specifically, the steps include:
[0089] S3.1 Loss Function Design:
[0090] Masking loss: Dice loss is used to calculate the overlap between the predicted mask and the standard mask, adapting to irregular areas of tobacco leaves (such as leaf edges and mold spots).
[0091] Semantic classification loss: Cross-entropy loss is used to calculate object classification loss (for "intact tobacco leaf / moldy tobacco leaf") and component classification loss (for "leaf tip / moldy spot").
[0092] No semantic data loss: Only mask loss is calculated, and unlabeled multi-granularity mask data is used to improve segmentation power;
[0093] S3.2, Many-to-many matching:
[0094] The Hungarian algorithm is used to establish the optimal match between "K prediction results" and "multiple standard masks (different granularities)" to solve the problem of "a single input corresponding to multiple granularity labels" (e.g., when clicking on the leaf tip, the standard includes "leaf tip mask", "leaf mask", and "complete tobacco leaf mask").
[0095] S3.3 Training Process:
[0096] a. Pre-training: Train the model's multi-granularity segmentation capability using only semantically unsemantic multi-granularity masked data (similar to SA-1B), and initialize the visual encoder and decoder;
[0097] b. Joint training: Add semantically labeled tobacco leaf data, jointly optimize mask loss and semantic classification loss, and learn the association between "segmentation + semantics";
[0098] c. Optimizer: The AdamW optimizer is used with an initial learning rate of 1e-4, which decays by a factor of 10 at 90%-95% of the training steps to avoid overfitting.
[0099] S4. Multi-granularity inference application: After receiving the user's selection input instruction, the model outputs multi-granularity semantic segmentation results, which are integrated into the tobacco intelligent monitoring system for grading and mold diagnosis. The multi-granularity semantic segmentation results output by the model include K different granularities of "semantic labels + segmentation masks".
[0100] For example, the specific implementation process is as follows:
[0101] Enter: Click on "leaf tip mold spots";
[0102] Output:
[0103] Fine-grained: Semantic tag “mold spots on tobacco leaf tips”, masked to cover the spotted area;
[0104] Medium granularity: Semantic tag "tobacco leaf", mask covering the entire leaf;
[0105] Coarse-grained: Semantic tag "moldy tobacco leaf", mask overlaying intact damaged tobacco leaf;
[0106] Application Integration: Integrate the reasoning results into the "Intelligent Tobacco Leaf Monitoring System";
[0107] Tobacco leaf grading: Automatically graded based on "leaf integrity" (coarse grain size) and "component condition" (medium grain size);
[0108] Mold diagnosis: Locate the location and type of mold based on fine-grained semantic tags, and calculate the affected area.
[0109] Example 2
[0110] To verify the technical effects of this invention, the following experiments were conducted. It should be noted that the following experimental data...
[0111] These are preliminary test results used to illustrate the technical advantages of this invention compared to existing technologies. Specific performance indicators may vary depending on factors such as the test environment and dataset size.
[0112] Experiment 1: Performance Comparison
[0113] On a self-made tobacco leaf test set (1000 images covering different scenes), the SAM model, Mask2Former model, existing CNN model, and the method described in this invention were used for recognition and evaluation. The evaluation was conducted using three metrics: "1-IoU@All Granularity" (average IoU across all granularities), "semantic classification accuracy," and "fine-grained (mold spot) IoU." The relevant data are shown in Table 1.
[0114] Table 1 Model training and validation performance
[0115] Model 1-IoU@All Granularity Semantic classification accuracy Fine particle size (mold spots) IoU SAM (General) 78.5% - (no semantic meaning) 65.2% Mask2Former (semantic) 68.3% (single particle size) 82.1% 58.7% Existing tobacco leaf CNN model 72.6% (single particle size) 79.3% 62.5% Method of the present invention 89.2% 94.5% 85.8%
[0116] The results show that the method of the present invention is significantly better than the comparative model in terms of overall accuracy at multiple granularities, semantic accuracy, and fine-granular mold identification accuracy.
[0117] Experiment 2: Scene Adaptability
[0118] Table 2 shows the fine-grained IoU of the method of this invention and the existing tobacco leaf CNN model under three complex sub-scenarios: "light intensity variation", "leaf overlap", and "early stage of mold growth".
[0119] Table 2 Comparison of Scene Adaptability Performance
[0120] Complex scenarios The method IoU of this invention Existing tobacco leaf CNN model IoU Increase Strong light / Weak light 82.3% 59.6% +22.7% The leaves overlap (2-3 leaves). 79.5% 55.8% +23.7% Early stage of mold growth (spots < 5mm) 76.8% 48.2% +28.6%
[0121] The results show that the method of the present invention has significantly improved robustness in complex field scenarios, and is especially suitable for early diagnosis of mold (existing models have difficulty identifying tiny spots).
[0122] Experiment 3: Application Efficiency
[0123] In the tobacco leaf grading scenario, it takes 6 hours to manually grade 1,000 tobacco leaves, while the "intelligent monitoring system" using the method of this invention only takes 30 minutes, increasing efficiency by 12 times; in the mold diagnosis scenario, it takes 2 hours to manually inspect 10 acres of tobacco fields, while the system can automatically identify mold in only 15 minutes, and the missed diagnosis rate is reduced from 8% for manual inspection to 1.2%.
[0124] In summary, the experimental results verified the technical advantages of the method of the present invention in terms of multi-granularity semantic recognition capability, adaptability to complex scenarios, data utilization efficiency, and practical application efficiency, and proved that the technical solution of the present invention can effectively solve the problems existing in the current tobacco leaf recognition technology.
[0125] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A multi-granularity tobacco leaf recognition method based on semantic segmentation, characterized in that, Includes the following steps: S1. Construct a multi-granularity semantic segmentation dataset for tobacco leaves: Collect images of tobacco leaves of different varieties, growth stages, and scenes, label them with semantic labels of three levels of granularity (coarse, medium, and fine) and non-semantic multi-granularity masks, and perform data augmentation to adapt to field scenes. S2. Construct a multi-granularity semantic segmentation model for tobacco leaves: This model includes a visual encoder, a decoupled text encoder, and a multi-granularity deformable mask decoder. Through this model, user input is converted into anchor boxes, generating K multi-granularity content embeddings, and outputting the semantic label and mask for each granularity. S3. Model Training: Design a loss function and use a many-to-many Hungarian matching strategy to jointly train two types of data: those with semantic annotations and those without semantic masks. S4. Multi-granularity inference application: After receiving the user's selection input instruction, the model outputs multi-granularity semantic segmentation results, which are integrated into the tobacco intelligent monitoring system for grading and mold diagnosis.
2. The method for multi-granularity identification of tobacco leaves based on semantic segmentation according to claim 1, characterized in that, In step S1, the three levels of semantic tags—coarse, medium, and fine—are specifically as follows: Coarse-grained: Semantic tags include "whole tobacco leaves" and "coarsely crushed tobacco leaves"; Medium granularity: Semantic tags include "medium broken tobacco leaves"; Fine-grained: Semantic tags include "broken tobacco leaves" and "mold spots"; The semantically nullable multi-granularity mask is labeled using the SA-1B dataset, and the labeled semantically nullable multi-granularity mask is not associated with semantic labels.
3. The method for multi-granularity identification of tobacco leaves based on semantic segmentation according to claim 1, characterized in that, In step S1, the data augmentation for adapting to the field scene specifically involves: using enhancement strategies such as illumination perturbation, geometric transformation, and noise addition to improve the model's anti-interference ability in response to interference in the field scene.
4. The method for multi-granularity identification of tobacco leaves based on semantic segmentation according to claim 1, characterized in that, In step S2, the construction of the multi-granularity semantic segmentation model for tobacco leaves specifically includes: S2.1 Visual Encoder: The pre-trained Swin Transformer-L is used as the visual backbone to extract multi-scale visual features of tobacco leaf images. Its input is a 1024×1024 tobacco leaf image, and its output is a multi-scale feature map. S2.2 Decoupled Text Encoder: Based on the UniCL pre-trained text model, a decoupled semantic encoding module specifically for tobacco leaves is constructed. Its input is the semantic label of tobacco leaves, and its output is two types of embedding vectors: object embedding and component embedding. S2.3 Multi-granularity deformable mask decoder: To address the multi-granularity requirements of tobacco leaves, a deformable mask decoder based on multi-selection learning is designed. This decoder converts user interaction input into unified anchor boxes, generating K content embedding vectors for each anchor box, with each embedding vector corresponding to a granularity level. in, For granular-level embedding, Embed for input type; The deformable mask decoder employs the deformable attention mechanism DeformDec, combining visual features, positional embedding, and multi-granularity content embedding to output the prediction result at each granularity, formally represented as: Where O is the DeformDec output set containing K content embedding vectors, b is the anchor box, F is the visual encoder output feature, and o i c is the prediction tuple for the i-th granularity; i For semantic label prediction at the i-th granularity; m i Let be the segmentation mask for the i-th granularity.
5. The method for multi-granularity identification of tobacco leaves based on semantic segmentation according to claim 1, characterized in that, In step S3, the design loss function specifically includes mask loss, semantic classification loss, and semantic data loss. The mask loss uses Dice loss to calculate the overlap between the predicted mask and the standard mask. The semantic classification loss uses cross-entropy loss to calculate the object classification loss and the component classification loss respectively. The semantic data loss only calculates the mask loss, using unlabeled multi-granularity mask data to improve segmentation capability.
6. The method for multi-granularity identification of tobacco leaves based on semantic segmentation according to claim 1, characterized in that, In step S3, the joint training includes two types of data: semantically labeled data and data without semantic masks. Specifically, it involves first using data without semantic multi-granularity masks to pre-train the model's segmentation capability, and then adding data with semantic labels for joint optimization. The optimizer is AdamW with a learning rate of 1e-4, which decays by a factor of 10 when the training steps reach 90%-95%.
7. The method for multi-granularity identification of tobacco leaves based on semantic segmentation according to claim 1, characterized in that, In step S4, the model outputs multi-granularity semantic segmentation results including K different granularities of "semantic labels + segmentation masks".