A method for quantifying the drying degree of primary cured tobacco leaves based on a constrained CaiT model
By constructing a feature component c to determine image color distortion and using assigned weights to constrain the CaiT-transformer model, the problem of discontinuous predicted values in the identification of the dryness of initially roasted tobacco leaves was solved, the identification accuracy was improved, and the tobacco roasting control was optimized.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- YUNNAN TOBACCO CO CHUXIONG PREFECTURE CO
- Filing Date
- 2023-01-13
- Publication Date
- 2026-05-29
AI Technical Summary
The existing CaiT-transformer model suffers from discontinuous prediction values in the identification of the dryness of newly cured tobacco leaf images, which affects the accuracy of the identification.
A constrained CaiT model-based approach is adopted. By constructing a feature component c, the model is judged to determine whether the image color is distorted. The model is constrained by the assigned weights absolute error weights, and the cross-entropy is used as the objective function to train the constrained CaiT-transformer model to ensure the continuity and accuracy of the predicted values.
It improves the accuracy of identifying the dryness of newly cured tobacco leaves in images. The average accuracy of the constrained CaiT-transformer model reaches 97.76%, which is 2.77% higher than that of the unconstrained model. It can also more accurately control the tobacco curing temperature, shorten the time and save energy.
Smart Images
Figure CN116403065B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of methods for recognizing the degree of dryness in images of freshly cured tobacco leaves, and specifically to a method for quantifying the degree of dryness in freshly cured tobacco leaves based on a constrained CaiT model. Background Technology
[0002] The tobacco curing process is a crucial step in producing high-quality tobacco leaves. The initial curing process employs a three-stage process: yellowing, color fixing, and core drying. Temperature control at each stage is typically adjusted flexibly based on the changes in color and shape of the tobacco leaves during curing. These changes are caused by variations in the degree of dryness of the leaves during curing. Therefore, accurate identification of the dryness level of the initially cured tobacco leaves is key to controlling the temperature at each stage. However, this identification process heavily relies on human experience and is time-consuming and labor-intensive. Therefore, applying machine vision to represent images of initially cured tobacco leaves is of great significance for further understanding the curing process and improving tobacco quality.
[0003] In recent years, with the continuous development of artificial intelligence, various deep learning algorithms have been gradually applied to flue-cured tobacco production. The most widely used deep learning algorithm is the convolutional neural network (CNN). In CNNs, high-level feature representations depend on low-level feature representations, gradually extracting features with higher-level semantic information from shallow to deep layers. However, in the field of primary flue-cured tobacco technology, the existing CNNs have a problem: the actual receptive field is smaller than the theoretical receptive field. This results in the CNN being unable to fully extract global contextual information, limiting its application to the initial curing stage of tobacco leaves and preventing more accurate identification of the degree of dryness.
[0004] Convolutional neural network models based on the Transformer utilize attention mechanisms to capture global contextual information, enabling the extraction of stronger and more advantageous features. This results in higher recognition accuracy compared to traditional convolutional neural networks (CNNs), and they have been widely applied in image classification and recognition. With the rapid development of machine vision, experts have further improved the Transformer model, constructing the CaiT-transformer model. Compared to the Transformer model, the CaiT-transformer model introduces LayerScale and Class-Attention, increasing the model's depth while ensuring that its performance does not prematurely saturate with increasing depth, thereby improving the stability and accuracy of the model in image recognition representation.
[0005] The CaiT-transformer model, introduced into the field of image classification and recognition of newly cured tobacco leaves, can identify the degree of dryness in these images. However, the objective function commonly used in existing CaiT-transformer models is not suitable for the field of newly cured tobacco leaves, leading to discontinuous predicted values and affecting the accuracy of dryness assessment. Therefore, improving the CaiT-transformer model to better identify the degree of dryness in newly cured tobacco leaves is a technical problem to be solved in this field. Summary of the Invention
[0006] To better meet the needs of identifying the dryness of newly cured tobacco leaf images in this field, and to solve the problem of discontinuous prediction values of newly cured tobacco leaf images by the existing CaiT-transformer model, this invention proposes a method for quantifying the dryness of newly cured tobacco leaves based on the constrained CaiT model. This method can ensure the continuity of the predicted values of newly cured tobacco leaf images and improve the accuracy of the model in identifying the dryness of newly cured tobacco leaf images.
[0007] To achieve the above objectives, the present invention adopts the following technical solution:
[0008] A method for quantifying the dryness of first-cured tobacco leaves based on a constrained CaiT model, characterized by the following steps:
[0009] S1: The camera captures a large number of images of newly cured tobacco leaves to create a sample for calibrating the dryness of the images of newly cured tobacco leaves.
[0010] S11: Construct parametric feature component c based on the feature component a representing red and green values and the feature component b representing yellow and blue values in the Lab color space; then establish a criterion for judging whether the image color is distorted through feature component c.
[0011] S12: Create a dryness calibration sample of the first-cured tobacco leaf images: Use a camera to collect a large number of first-cured tobacco leaf images, calculate the feature component c of the large number of first-cured tobacco leaf images, and combine the judgment criteria to determine the distorted and undistorted images in the large number of first-cured tobacco leaf images. Collect the undistorted images in the large number of first-cured tobacco leaf images according to the set collection criteria to make a dryness calibration sample of the first-cured tobacco leaf images.
[0012] The image of the first-cured tobacco leaves has a cardboard background and includes a tobacco leaf portion and a cardboard portion. The tobacco leaf portion is defined as the tobacco leaf area, and the cardboard portion is defined as the non-tobacco leaf area.
[0013] S2: Constraints on CaiT-transformer model construction and training;
[0014] S21: Based on the drying degree calibration sample of the images of the first-cured tobacco leaves, the experts calibrated the drying degree of the images of the first-cured tobacco leaves by observing the changes in the leaf tips, and obtained the drying degree grade label Y of the first-cured tobacco leaves.
[0015] S22: Perform image segmentation preprocessing on the dryness calibration samples of the first-cured tobacco leaf images to obtain segmented image dataset X;
[0016] S23: Using the dryness grade label Y of the first-cured tobacco leaves as the dependent variable and the segmented image dataset X as the independent variable, a CaiT-transformer model is established; based on the CaiT-transformer model, absolute error weights are used to constrain the model, and the product of the assigned weights and cross-entropy is used as the objective function to establish a constrained CaiT-transformer model; the objective function is then used to update the model weight parameters through backpropagation to train the constrained CaiT-transformer model.
[0017] Among them, the dryness grade label Y is used as the true target value of the model. The segmented image dataset X′ obtained by image segmentation preprocessing of the newly acquired images of freshly roasted tobacco leaves is imported into the CaiT-transformer model to obtain the predicted value Y′.
[0018] The absolute error weight is assigned based on the difference between the terms containing the maximum values of the vectors of the true target value Y and the predicted value Y'. The specific method is as follows:
[0019]
[0020] Where I1 is the term containing the maximum value of the predicted value Y', and I0 is the term containing the maximum value of the actual target value Y;
[0021] S3: Real-time acquisition and preprocessing of images of freshly roasted tobacco leaves by camera;
[0022] S31: Use a camera to acquire images of newly cured tobacco leaves in real time, calculate the feature component c of each image of newly cured tobacco leaves, and judge the distorted and undistorted images in the images of newly cured tobacco leaves according to the judgment criteria.
[0023] S32: Correct the color of the distorted image in real time to obtain a corrected image;
[0024] S33: Perform segmentation image preprocessing on the obtained corrected image and the undistorted image to obtain the segmentation image dataset X″;
[0025] S4: Import the segmented image dataset X″ into the constrained CaiT-transformer model to obtain the predicted value Y″ of the first-cured tobacco leaf image;
[0026] S5: Feed back the predicted value Y″ of the initial roasted tobacco leaf image to the roasting control terminal in real time, establish a temperature control mechanism in the roasting control terminal, and control the time when the temperature of different roasting stages in the initial roasting process of tobacco leaves enters the stable temperature stage based on the temperature control mechanism.
[0027] In step S11, the feature component c of the judgment index is constructed as follows: the image is converted from the RGB color space to the Lab color space; then the Lab color space of the image is decomposed to obtain three feature components L, a, and b; and feature component c is constructed based on feature components a and b. The formula for calculating feature component c is:
[0028]
[0029] 'a' represents the red-green value, with positive numbers indicating a reddish tint and negative numbers indicating a greenish tint; 'b' represents the yellow-blue value, with positive numbers indicating a yellowish tint and negative numbers indicating a blue tint.
[0030] The judgment criterion is established as follows: Obtain the feature component c of the image; calculate the difference between the maximum and minimum values of feature component c among any five consecutive and adjacent images; compare this difference with a preset threshold; if the difference is greater than the threshold, then it is determined that there is a distorted image among the five images of the first-cured tobacco leaves. The judgment criterion formula is:
[0031] δc i =max([c i-4 ,c i-3 ,c i-3 ,c i-1 ,c i ])-min([c i-4 ,c i-3 ,c i-3 ,c i-1 ,c i ])
[0032] When δc i If the value is greater than ζ, then image i is determined to be distorted;
[0033] δc i ζ represents the distortion index of the i-th sample; ζ represents the threshold; preferably, the threshold is 1.
[0034] In step S12, the camera captures images at a frequency of one image per minute; the large number of images of initially roasted tobacco leaves captured by the camera include historically captured images of initially roasted tobacco leaves and real-time images of initially roasted tobacco leaves captured during subsequent initial roasting processes; the different roasting stages refer to the three roasting stages in the initial roasting process of tobacco leaves: the yellowing stage, the color-fixing stage, and the dry-rib stage; the collection criteria are specifically as follows: the roasting stage of the initially roasted tobacco leaves in the undistorted images includes the three roasting stages: the yellowing stage, the color-fixing stage, and the dry-rib stage; the undistorted images are at least five initially roasted tobacco leaf images captured consecutively in time; the undistorted images are initially roasted tobacco leaf images with a cardboard background, complete images, and maintaining a fixed and uniform height and angle.
[0035] In step S21, the dryness grade labels of the initial roasted tobacco leaves are 13 categories: 5, 8, 11, 16, 18, 21, 23, 25, 28, 30, 35, 37, and 41. Among them, when the dryness grade is <18, the initial roasted tobacco leaves are in the yellowing stage during the roasting process; when the dryness grade is [18, 35], the initial roasted tobacco leaves are in the color fixing stage during the roasting process; and when the dryness grade is >35, the initial roasted tobacco leaves are in the dry core stage during the roasting process.
[0036] In step S22, the image segmentation preprocessing involves using the SegNet+DeepLabV3 segmentation method to segment all images (including the obtained corrected image and the undistorted image) into tobacco leaf regions and non-tobacco leaf regions, and assigning the non-tobacco leaf regions to white to obtain the segmented image of the first-cured tobacco leaves.
[0037] In step S23, the formula for calculating the cross-entropy is:
[0038]
[0039] Where Y is the true target value and Y' is the sample predicted value.
[0040] In step S23, the training of the constrained CaiT-transformer model also includes: training of model pre-training weights and target key region localization.
[0041] Specifically, the model pre-training weights are first scaled up to the size of the segmented image dataset X to obtain a segmented image of the first-cured tobacco leaves with a scale of [384, 384, 3]. Then, the backbone is used to assign the weights of CaiT-s36-384 in the pretrain 1k data to train and update the constraint cait-Transformer model on the segmented image of the first-cured tobacco leaves.
[0042] Specifically, the target key region localization training involves locating the target key region of the model at the tip of the tobacco leaf. First, the image size of the segmented image dataset X is scaled to obtain a segmented image of the first-cured tobacco leaf at a scale of [384, 384, 3]. Then, the weights of the model.block4[-1].norm1 layer are extracted from the model, and the weights are activated and displayed on the segmented image of the first-cured tobacco leaf. It is then determined whether the activated region is in the target key region, the tip of the tobacco leaf. If the activated region is not in the target key region, the scale of the segmented image of the first-cured tobacco leaf is further increased based on the scale of [384, 384, 3] to update and constrain the cait-Transformer model.
[0043] In step S32, the real-time correction of the distorted image color is specifically performed as follows: First, obtain the feature components L1, a1, b1 of the Lab color space of the cardboard portion above the tobacco leaf area in the distorted image; then, obtain the feature components L0, a0, b0 of the Lab color space of the cardboard portion above the tobacco leaf area in the undistorted image that is closest to the distorted image; finally, divide the two to obtain the correction coefficient, calculated using the following formula:
[0044]
[0045] When dL1 = 1; da1 = 1; db1 = 1, the image is not distorted;
[0046] When dL1≠1 and / or da1≠1 and / or db1≠1, the image is a distorted image. The LAB color space feature components of the tobacco leaf region of the distorted image are multiplied by the correction coefficient to perform real-time correction of the color of the tobacco leaf region of the distorted image, thereby obtaining a corrected image.
[0047] In step S4, the predicted value Y″ of the initial cured tobacco leaf image is imported into the constrained CaiT-transformer model to update the objective function. The objective function updates the parameters of the constrained CaiT-transformer model through backpropagation.
[0048] Furthermore, in step S4, the predicted value Y″ of the initial roasted tobacco leaf image is compared with the label Y of the initial roasted tobacco leaf dryness grade, the recognition accuracy of each dryness grade is calculated, and the average accuracy of the constrained CaiT-transformer model is calculated based on the recognition accuracy.
[0049] Specifically, the constrained CaiT-transformer model achieved an average accuracy of 97.76%, which is 2.77% higher than the average accuracy of the CaiT-transformer model.
[0050] In step S5, the temperature control mechanism includes a heating phase and a temperature stabilization phase. First, a dryness target is preset. During the heating phase, it is observed whether the real-time feedback of the dryness level reaches the preset dryness target. If it does, the temperature mode is adjusted from the heating phase to the temperature stabilization phase.
[0051] Compared with the prior art, the present invention has the following beneficial effects:
[0052] (1) The CaiT-transformer model, compared with traditional CNNs (convolutional neural networks), introduces LayerScale and Class-Attention (CA) modules. While increasing the depth of the Transformer model, it ensures that the model performance will not saturate prematurely with the increase of depth, thereby improving the stability and accuracy of the model in image recognition and representation, enabling the model to be applied to the recognition of the dryness of the first-cured tobacco leaf image; The constrained CaiT-transformer model is based on the CaiT-transformer model and uses absolute error weight to constrain the model. The assigned weight is multiplied by the cross-entropy as the objective function. The objective function is trained by backpropagation to ensure the continuity of the predicted value of the first-cured tobacco leaf image, which greatly improves the stability and accuracy of the model prediction results.
[0053] (2) Because the color of tobacco leaves changes from green to yellow during the initial curing process, the yellow-green color component accounts for a large proportion of the image of initial curing tobacco leaves. Therefore, it is particularly important to accurately judge the change of tobacco leaf color from green to yellow in the image of initial curing tobacco leaves. Feature components a and b represent red-green value and yellow-blue value, respectively. Feature component c is constructed based on feature components a and b. Then, a judgment standard for judging whether the image color is distorted is established through feature component c. Compared with conventional color distortion judgment methods, it is more suitable for the field of initial curing tobacco leaf image recognition and improves the accuracy of color distortion judgment of initial curing tobacco leaf images.
[0054] (3) The predicted value Y″ of the initial curing tobacco leaf image is compared with the label Y of the dryness grade of the initial curing tobacco leaf. The recognition accuracy of each dryness grade is calculated, and the average accuracy of the constrained CaiT-transformer model is calculated based on the recognition accuracy. For the same batch of initial curing tobacco leaf images, the average accuracy of the constrained CaiT-transformer model is 97.76%, and the average accuracy of the CaiT-transformer model is 94.99%. Therefore, the average accuracy of the predicted value obtained by the constrained CaiT-transformer model is 2.77% higher than that of the CaiT-transformer model, and it can play a more stable role in actual production.
[0055] (4) Obtain the image prediction value Y″ of the first roasted tobacco leaves by constraining the CaiT-transformer model and feed it back to the roasting control terminal in real time. Establish a temperature control mechanism in the roasting control terminal. Based on the temperature control mechanism, control the time when the temperature of different roasting stages in the initial roasting process of tobacco leaves enters the stable temperature stage. This can more accurately control the roasting temperature of tobacco leaves, avoid over-roasting, shorten the initial roasting time of tobacco leaves, and save energy. Attached Figure Description
[0056] Figure 1 This is a flowchart of a method for quantifying the degree of dryness of first-cured tobacco leaves based on a constrained CaiT model, according to the present invention.
[0057] Figure 2 This is a weighted activation diagram of the tobacco leaf tip of the constrained CaiT-transformer model according to an embodiment of the present invention.
[0058] Figure 3 This is an image of raw, uncured tobacco leaves captured by a camera according to an embodiment of the present invention.
[0059] Figure 4 This is an embodiment of the present invention showing an online image distortion monitoring abc-value map.
[0060] Figure 5 This is a partially enlarged view of feature component c in one embodiment of the present invention.
[0061] Figure 6 This is a distorted image according to one embodiment of the present invention.
[0062] Figure 7 This is a corrected image according to one embodiment of the present invention.
[0063] Figure 8 This is a segmented image according to one embodiment of the present invention.
[0064] Figure 9 This is a structural diagram of a constrained CaiT-transformer model according to an embodiment of the present invention.
[0065] Figure 10 This is a graph showing the change in the objective function loss of a constrained CaiT-transformer model according to an embodiment of the present invention. Detailed Implementation
[0066] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions in the embodiments of this invention will be clearly and completely described below in conjunction with the embodiments of this invention. Obviously, the described embodiments are only some embodiments of this invention, not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.
[0067] like Figure 1 As shown in the figure, this embodiment of the invention provides a method for quantifying the dryness of newly cured tobacco leaves based on a constrained CaiT model. The specific steps are as follows:
[0068] S1: The camera captures a large number of images of initially cured tobacco leaves to create a sample for calibrating the dryness of the initially cured tobacco leaves: (The term "initial curing" refers to the process of harvesting fresh tobacco leaves with a moisture content of 80-90% that have matured in the field, placing them in specific equipment, and then manually adjusting the temperature, humidity, and ventilation conditions to cure them into raw tobacco with a moisture content of 16-18%.)
[0069] S11: Construct parametric feature component c based on the Lab color space feature components a and b; then establish a criterion for judging whether the image color is distorted through feature component c. This mainly determines whether the image is distorted or discolored, especially since the high-temperature environment during tobacco roasting can cause deviations in the image's white balance and color system, leading to color space distortion.
[0070] Specifically, the method for constructing the feature component c of the judgment index is as follows: The image is converted from the RGB color space to the Lab color space; then, the Lab color space of the image is decomposed to obtain three feature components L, a, and b; and feature component c is constructed based on feature components a and b. The formula for calculating feature component c is:
[0071]
[0072] 'a' represents the red-green value, with positive numbers indicating a reddish tint and negative numbers indicating a greenish tint; 'b' represents the yellow-blue value, with positive numbers indicating a yellowish tint and negative numbers indicating a blue tint.
[0073] Specifically, the method for establishing the judgment criteria is as follows: Obtain the feature component c of the image; calculate the difference between the maximum and minimum values of the feature component c among any five consecutive and adjacent images, i.e., the distortion index; compare this difference with a preset threshold; if the distortion index is greater than the threshold, then it is determined that there is a distorted image among the five images of the first-cured tobacco leaves. The judgment criterion formula is:
[0074] δc i =max([c i-4 ,c i-3 ,c i-3 ,c i-1 ,ci ])-min([c i-4 ,c i-3 ,c i-3 ,c i-1 ,c i ])
[0075] When δc i If the value is greater than ζ, then image i is determined to be distorted;
[0076] δc i ζ represents the distortion index of the i-th sample; ζ represents the threshold; preferably, the threshold is 1.
[0077] S12: Create a dryness calibration sample of the first-cured tobacco leaf images: Use a camera to collect a large number of first-cured tobacco leaf images, calculate the feature component c of the large number of first-cured tobacco leaf images, and combine the judgment criteria to determine the distorted and undistorted images in the large number of first-cured tobacco leaf images. Collect the undistorted images in the large number of first-cured tobacco leaf images according to the set collection criteria to make a dryness calibration sample of the first-cured tobacco leaf images.
[0078] The image of the first-cured tobacco leaves has a cardboard background and includes a tobacco leaf portion and a cardboard portion. The tobacco leaf portion is defined as the tobacco leaf area, and the cardboard portion is defined as the non-tobacco leaf area.
[0079] Specifically, the camera captures images at a frequency of one image per minute; the large number of images of initially roasted tobacco leaves captured by the camera includes historically captured images of initially roasted tobacco leaves and real-time images of initially roasted tobacco leaves captured during subsequent initial roasting processes; the different roasting stages refer to the three roasting stages in the initial roasting process of tobacco leaves: the yellowing stage, the color-fixing stage, and the dry-rib stage; the collection criteria are as follows: the roasting stage of the initially roasted tobacco leaves in the undistorted images includes the three roasting stages: the yellowing stage, the color-fixing stage, and the dry-rib stage; the undistorted images are at least five initially roasted tobacco leaf images captured consecutively in time; the undistorted images are initially roasted tobacco leaf images with a cardboard background, complete images, and maintaining a fixed and uniform height and angle.
[0080] S2: Constraints on CaiT-transformer Model Construction and Training:
[0081] Compared to the Transformer model, the CaiT-transformer model introduces LayerScale and Class-Attention (CA) modules: LayerScale adds a learnable diagonal matrix to the output of each residual block, which is initialized close to 0. Adding this simple layer after each residual block improves the dynamics of training, enabling the training of deeper, high-capacity Transformers. Adding LayerScale not only does not change the structural performance but also makes deeper models converge. The Class-Attention (CA) module extracts information from the processed patches embedding using CLS. CA has lower computational cost than SA because it calculates the attention between the class token and the processed patches embedding.
[0082] S21: Based on the drying degree calibration sample of the first-cured tobacco leaf images, the experts calibrated the drying degree of the first-cured tobacco leaf images by observing the changes in the leaf tips, and obtained the drying degree grade label Y of the first-cured tobacco leaves.
[0083] Specifically, the dryness grade labels for the initial roasted tobacco leaves are 13 categories: 5, 8, 11, 16, 18, 21, 23, 25, 28, 30, 35, 37, and 41. Among them, when the dryness grade is <18, the initial roasted tobacco leaves are in the yellowing stage during the roasting process; when the dryness grade is [18, 35], the initial roasted tobacco leaves are in the color-fixing stage during the roasting process; and when the dryness grade is >35, the initial roasted tobacco leaves are in the dry-core stage during the roasting process. The classification table of the dryness grade labels for initial roasted tobacco leaves is shown in Table 1.
[0084] Classification Table of Dryness Grades of Initially Cured Tobacco Leaves
[0085]
[0086] S22: Perform image segmentation preprocessing on the dryness degree calibration samples of the initially cured tobacco leaf images to obtain the segmented image dataset X:
[0087] Specifically, the image segmentation preprocessing uses the SegNet+DeepLabV3 segmentation method to segment all images (including the obtained corrected image and the undistorted image) into tobacco leaf regions and non-tobacco leaf regions, and assigns the non-tobacco leaf regions to white to obtain the segmented image of the first-cured tobacco leaves.
[0088] S23: Using the dryness grade label Y of the first-cured tobacco leaves as the dependent variable and the segmented image dataset X as the independent variable, a CaiT-transformer model is established; based on the CaiT-transformer model, absolute error weights are used to constrain the model, and the product of the error weights and cross-entropy is used as the objective function to establish a constrained CaiT-transformer model; the objective function is then used to update the model weight parameters through backpropagation to train the constrained CaiT-transformer model.
[0089] In this model, the dryness grade label Y is used as the true target value. The segmented image dataset X′ obtained by preprocessing the newly acquired images of freshly roasted tobacco leaves is imported into the CaiT-transformer model to obtain the predicted value Y′.
[0090] The absolute error weight is assigned based on the difference between the terms containing the maximum values of the vectors of the true target value Y and the predicted value Y'. The specific method is as follows:
[0091]
[0092] Where I1 is the term containing the maximum value of the predicted value Y', and I0 is the term containing the maximum value of the actual target value Y.
[0093] Specifically, the cross-entropy calculation formula is as follows:
[0094]
[0095] Where Y is the true target value and Y' is the sample predicted value.
[0096] Specifically, the training of the constrained CaiT-transformer model also includes: pre-training the model weights and training the target key region localization.
[0097] The model pre-training weights are obtained by first scaling the image size of the segmented image dataset X to obtain a segmented image of the first-cured tobacco leaves with a scale of [384, 384, 3]; then calling the backbone as the weights of CaiT-s36-384 in the pretrain 1k data to train and update the constraint cait-Transformer model on the segmented image of the first-cured tobacco leaves.
[0098] The target key region localization training involves locating the target key region of the model at the tip of the tobacco leaf. First, the image size of the segmented image dataset X is scaled to obtain a segmented image of the initially roasted tobacco leaf with a scale of [384, 384, 3]. Then, the weights of the model.block4[-1].norm1 layer are extracted from the model, and these weights are activated and displayed on the segmented image of the initially roasted tobacco leaf, resulting in a weight activation map of the tobacco leaf tip (e.g., ...). Figure 2 As shown in the figure, it is determined whether the activated region is in the target key region, the tip of the tobacco leaf; if the activated region is not in the target key region, the initial roasted tobacco leaf segmentation image is further enlarged on the basis of scale [384,384,3] to update the constraint Cait-Transformer model.
[0099] S3: Real-time acquisition and preprocessing of images of freshly roasted tobacco leaves by camera;
[0100] S31: Use a camera to acquire images of the first-cured tobacco leaves in real time, calculate the feature component c of each first-cured tobacco leaf image, and determine the distorted and undistorted images in the first-cured tobacco leaf images according to the judgment criteria.
[0101] One embodiment: A camera is used to acquire images of the newly cured tobacco leaves in real time, and the resulting original images of the newly cured tobacco leaves are as follows. Figure 3 As shown, the feature component c of each image of the first-cured tobacco leaf is calculated, and an online image distortion monitoring abc-value map is created (e.g., ...). Figure 4 As shown), and the feature component c has a clear fault location, which is then locally magnified (e.g. Figure 5 As shown), combined with Figure 4 and Figure 5 It can be seen that when the sample number is around 575, a significant break occurs in feature component c, indicating that the distortion index of 5 consecutive points here is greater than the threshold of 1. After extracting the corresponding image of the first-cured tobacco leaf, it was found that the distorted image in it turned green overall (e.g., Figure 6 (As shown).
[0102] S32: Correct the color of the distorted image in real time to obtain a corrected image;
[0103] Specifically, the real-time color correction method for the distorted image is as follows: First, obtain the feature components L1, a1, b1 of the Lab color space of the cardboard portion above the tobacco leaf area in the distorted image; then, obtain the feature components L0, a0, b0 of the Lab color space of the cardboard portion above the tobacco leaf area in the undistorted image that is closest to the distorted image; finally, divide the two to obtain the correction coefficient, and the calculation formula is as follows:
[0104]
[0105] When dL1 = 1; da1 = 1; db1 = 1, the image is not distorted;
[0106] When dL1≠1 and / or da1≠1 and / or db1≠1, the image is a distorted image. The LAB color space feature components of the tobacco leaf region of the distorted image are multiplied by the correction coefficient to perform real-time correction of the color of the tobacco leaf region of the distorted image, thereby obtaining a corrected image.
[0107] One embodiment: First, extract the feature components L1, a1, b1 of the Lab color space from the cardboard portion above the tobacco leaf area in the distorted image (sample number around 575) to obtain: L1 = 62, a1 = 67, b1 = -18. Then, extract the feature components L0, a0, b0 of the Lab color space from the cardboard portion above the tobacco leaf area in the five consecutive relatively normal images preceding the distorted image (sample number around 575) to obtain: L0 = 61.82, a0 = 68.23, b0 = -17.31. Calculate the correction coefficient:
[0108]
[0109] The LAB color space feature components of the tobacco leaf region in the distorted image are multiplied by a correction coefficient to perform real-time color correction on the tobacco leaf region of the distorted image, resulting in a corrected image (e.g., ...). Figure 7 (As shown).
[0110] S33: Perform segmentation image preprocessing on the obtained corrected image and the undistorted image to obtain the segmentation image dataset X″;
[0111] One embodiment: The SegNet+DeepLabV3 segmentation method is used to segment all the above images (including the obtained corrected image and the undistorted image) into tobacco leaf regions and non-tobacco leaf regions, and the non-tobacco leaf regions are assigned white to obtain the segmented image of the first-cured tobacco leaves (e.g., Figure 8 (As shown).
[0112] S4: Import the segmented image dataset X″ into the constrained CaiT-transformer model to obtain the predicted value Y″ of the first-cured tobacco leaf image;
[0113] Furthermore, the predicted value Y″ of the first-cured tobacco leaf image described in S4 is imported into the constrained CaiT-transformer model update objective function. The objective function updates the parameters of the constrained CaiT-transformer model through backpropagation, thereby reducing the loss between the true value and the predicted value, making the predicted value generated by the model move closer to the true value, thus achieving retraining of the model.
[0114] The objective function is used to measure the degree of difference between the model's predicted value Y″ and the true value Y.
[0115] Furthermore, the predicted value Y″ of the initial curing tobacco leaf image is compared with the label Y of the initial curing tobacco leaf dryness grade, and the recognition accuracy of each dryness grade is calculated. Then, the average accuracy of the constrained CaiT-transformer model is calculated based on the recognition accuracy. The accuracy of each dryness grade of the constrained CaiT-transformer model and the CaiT-transformer model are compared.
[0116] As shown in Table 2, for the same batch of images of newly cured tobacco leaves, the average accuracy of the predicted values obtained based on the constrained CaiT-transformer model was 97.76%, while the average accuracy of the predicted values obtained based on the comparative method CaiT-transformer model was 94.99%. This indicates that the constrained CaiT-transformer model used in this invention has an average accuracy of 2.77% higher than that of the CaiT-transformer model, and can achieve more stable recognition and representation results in actual production.
[0117] Table 2. Prediction results of the CaiT-transformer model and the constrained CaiT-transformer model.
[0118]
[0119]
[0120] One embodiment: The predicted values are imported into a constrained CaiT-transformer model (model as follows). Figure 9 (As shown) The objective function is updated, and the parameters of the CaiT-transformer model are updated through backpropagation. The degree of difference between the predicted and actual values is obtained through the objective function, and the loss of the objective function changes as shown. Figure 10 As shown.
[0121] S5: Feed back the predicted value Y″ of the initial roasted tobacco leaf image to the roasting control terminal in real time, establish a temperature control mechanism in the roasting control terminal, and control the time when the temperature of different roasting stages in the initial roasting process of tobacco leaves enters the stable temperature stage based on the temperature control mechanism.
[0122] Specifically, the temperature control mechanism includes a heating phase and a stabilization phase. First, a dryness target is preset. During the heating phase, it is observed whether the real-time feedback of the dryness level reaches the preset dryness target. If it does, the temperature mode is adjusted from the heating phase to the stabilization phase.
[0123] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A method for quantifying the dryness of first-cured tobacco leaves based on a constrained CaiT model, characterized in that, Includes the following steps: S1: The camera captures a large number of images of newly cured tobacco leaves to create a sample for calibrating the dryness of the images of newly cured tobacco leaves. S11: Construct feature component c based on feature component a representing red and green values and feature component b representing yellow and blue values in the Lab color space; then establish a criterion for judging whether the image color is distorted through feature component c. The feature component c is constructed as follows: The image is converted from the RGB color space to the Lab color space; then, the Lab color space of the image is decomposed to obtain three feature components: L, a, and b; and feature component c is constructed based on feature components a and b. The formula for calculating feature component c is: 'a' represents the red-green value, with positive numbers indicating a reddish tint and negative numbers indicating a greenish tint; 'b' represents the yellow-blue value, with positive numbers indicating a yellowish tint and negative numbers indicating a blue tint. The judgment criterion is established as follows: Obtain the feature component c of the image; calculate the difference between the maximum and minimum values of feature component c among any five consecutive and adjacent images; compare this difference with a preset threshold; if the difference is greater than the threshold, then it is determined that there is a distorted image among the five images of the first-cured tobacco leaves. The judgment criterion formula is: when If so, then image i is determined to be distorted; Let represent the distortion index of the i-th sample; This represents a threshold value of 1. S12: Create a dryness calibration sample of the first-cured tobacco leaf images: Use a camera to collect a large number of first-cured tobacco leaf images, calculate the feature component c defined in S11 of the large number of first-cured tobacco leaf images, and combine it with the judgment criteria in S11 to determine the distorted and undistorted images in the large number of first-cured tobacco leaf images. Collect the undistorted images in the large number of first-cured tobacco leaf images according to the set collection criteria to make a dryness calibration sample of the first-cured tobacco leaf images. The image of the first-cured tobacco leaves has a cardboard background and includes a tobacco leaf portion and a cardboard portion. The tobacco leaf portion is defined as the tobacco leaf area, and the cardboard portion is defined as the non-tobacco leaf area. S2: Constraints on CaiT-transformer model construction and training; S21: Based on the drying degree calibration sample of the images of the first-cured tobacco leaves, the experts calibrated the drying degree of the images of the first-cured tobacco leaves by observing the changes in the leaf tips, and obtained the drying degree grade label Y of the first-cured tobacco leaves. S22: Perform image segmentation preprocessing on the dryness calibration samples of the first-cured tobacco leaf images to obtain segmented image dataset X; S23: Using the dryness grade label Y of the first-cured tobacco leaves as the dependent variable and the segmented image dataset X as the independent variable, a CaiT-transformer model is established; based on the CaiT-transformer model, absolute error weights are used to constrain the model, and the product of the error weights and cross-entropy is used as the objective function to establish a constrained CaiT-transformer model; the objective function is then used to update the model weight parameters through backpropagation to train the constrained CaiT-transformer model. Among them, the dryness grade label Y is used as the true target value of the model. The segmented image dataset X′ obtained by image segmentation preprocessing of the newly acquired images of freshly roasted tobacco leaves is imported into the CaiT-transformer model to obtain the predicted value Y′. The absolute error weight is assigned based on the difference between the terms containing the maximum values of the vectors of the true target value Y and the predicted value Y'. The specific method is as follows: in, This refers to the term containing the maximum value of the predicted value Y'. This refers to the term containing the maximum value of the true target value Y; S3: Real-time acquisition and preprocessing of images of freshly roasted tobacco leaves by camera; S31: Use a camera to acquire images of newly cured tobacco leaves in real time, calculate the feature component c of each image of newly cured tobacco leaves, and judge the distorted and undistorted images in the images of newly cured tobacco leaves according to the judgment criteria. S32: Correct the color of the distorted image in real time to obtain a corrected image; S33: Perform segmentation image preprocessing on the obtained corrected image and the undistorted image to obtain the segmentation image dataset X″; S4: Import the segmented image dataset X″ into the constrained CaiT-transformer model to obtain the predicted value Y″ of the first-cured tobacco leaf image; S5: Feed back the predicted value Y″ of the initial roasted tobacco leaf image to the roasting control terminal in real time, establish a temperature control mechanism in the roasting control terminal, and control the time when the temperature of different roasting stages in the initial roasting process of tobacco leaves enters the stable temperature stage based on the temperature control mechanism.
2. The method for quantifying the dryness of first-cured tobacco leaves based on a constrained CaiT model according to claim 1, characterized in that, The camera in S12 captures images at a frequency of one image per minute; the large number of images of initially roasted tobacco leaves captured by the camera include historically captured images of initially roasted tobacco leaves and real-time images of initially roasted tobacco leaves captured during subsequent initial roasting processes; the different roasting stages refer to the three roasting stages in the initial roasting process of tobacco leaves: the yellowing stage, the color-fixing stage, and the dry-rib stage; the collection criteria are as follows: the roasting stage of the initially roasted tobacco leaves in the undistorted images includes the three roasting stages: the yellowing stage, the color-fixing stage, and the dry-rib stage; the undistorted images are at least five initially roasted tobacco leaf images captured consecutively in time; the undistorted images are initially roasted tobacco leaf images with a cardboard background, complete images, and maintaining a fixed and uniform height and angle.
3. The method for quantifying the dryness of first-cured tobacco leaves based on a constrained CaiT model according to claim 2, characterized in that, The drying grade labels for the initial-cured tobacco leaves mentioned in S21 are 13 categories: 5, 8, 11, 16, 18, 21, 23, 25, 28, 30, 35, 37, and 41. Among them, when the drying grade is <18, the initial-cured tobacco leaves are in the yellowing stage during the curing process; when the drying grade is [18, 35], the initial-cured tobacco leaves are in the color-fixing stage during the curing process; and when the drying grade is >35, the initial-cured tobacco leaves are in the dry-core stage during the curing process.
4. The method for quantifying the dryness of first-cured tobacco leaves based on a constrained CaiT model according to claim 3, characterized in that, The image segmentation preprocessing described in S22 involves using the SegNet+DeepLabV3 segmentation method to segment all images, including the obtained corrected image and the undistorted image, into tobacco leaf regions and non-tobacco leaf regions, and assigning the non-tobacco leaf regions to white to obtain the segmented image of the first-cured tobacco leaves.
5. The method for quantifying the dryness of first-cured tobacco leaves based on a constrained CaiT model according to claim 4, characterized in that, The formula for calculating cross-entropy in S23 is as follows: Where Y is the true target value and Y' is the sample predicted value.
6. The method for quantifying the dryness of first-cured tobacco leaves based on a constrained CaiT model according to claim 5, characterized in that, The training of the constrained CaiT-transformer model described in S23 also includes: pre-training the model weights and training the target key region localization.
7. The method for quantifying the dryness of first-cured tobacco leaves based on a constrained CaiT model according to claim 6, characterized in that, The model pre-training weights are obtained by first scaling the image size of the segmented image dataset X to obtain a segmented image of the first-cured tobacco leaves with a scale of [384, 384, 3]; then calling the backbone as the weights of CaiT-s36-384 in the pretrain 1k data to train and update the constraint cait-Transformer model on the segmented image of the first-cured tobacco leaves.
8. The method for quantifying the dryness of first-cured tobacco leaves based on a constrained CaiT model according to claim 6, characterized in that, The target key region localization training involves locating the target key region of the model at the tip of the tobacco leaf. First, the image size of the segmented image dataset X is scaled to obtain a segmented image of the first-cured tobacco leaf at a scale of [384, 384, 3]. Then, the weights of the model.block4[-1].norm1 layer are extracted from the model, and the weights are activated and displayed on the segmented image of the first-cured tobacco leaf. It is then determined whether the activated region is in the target key region, the tip of the tobacco leaf. If the activated region is not in the target key region, the scale of the segmented image of the first-cured tobacco leaf is further increased based on the scale of [384, 384, 3] to update and constrain the cait-Transformer model.
9. A method for quantifying the dryness of first-cured tobacco leaves based on a constrained CaiT model according to claim 8, characterized in that, The real-time color correction of the distorted image described in S32 is specifically performed as follows: First, obtain the feature components L1, a1, b1 of the Lab color space of the cardboard portion above the tobacco leaf area in the distorted image; then, obtain the feature components L0, a0, b0 of the Lab color space of the cardboard portion above the tobacco leaf area in the undistorted image that is closest to the distorted image; finally, divide the two to obtain the correction coefficient, calculated using the following formula: When dL1=1; da1=1; db1=1, the image is not distorted; When dL1≠1 and / or da1≠1 and / or db1≠1, the image is a distorted image. The LAB color space feature components of the tobacco leaf region of the distorted image are multiplied by the correction coefficient to perform real-time correction of the color of the tobacco leaf region of the distorted image, thereby obtaining a corrected image.
10. A method for quantifying the dryness of first-cured tobacco leaves based on a constrained CaiT model according to claim 9, characterized in that, The predicted value Y″ of the first-cured tobacco leaf image described in S4 is imported into the constrained CaiT-transformer model to update the objective function. The objective function updates the parameters of the constrained CaiT-transformer model through backpropagation.
11. A method for quantifying the dryness of first-cured tobacco leaves based on a constrained CaiT model according to claim 10, characterized in that, The predicted value Y″ of the first-cured tobacco leaf image described in S4 is compared with the label Y of the degree of dryness of the first-cured tobacco leaf. The recognition accuracy of each degree of dryness is calculated, and the average accuracy of the constrained CaiT-transformer model is calculated based on the recognition accuracy.
12. The method for quantifying the dryness of first-cured tobacco leaves based on a constrained CaiT model according to claim 11, characterized in that, The constrained CaiT-transformer model achieved an average accuracy of 97.76%, which is 2.77% higher than the average accuracy of the CaiT-transformer model.
13. The method for quantifying the dryness of first-cured tobacco leaves based on a constrained CaiT model according to claim 12, characterized in that, The temperature control mechanism described in S5 includes a heating phase and a temperature stabilization phase. First, a dryness target is preset. During the heating phase, it is observed whether the real-time feedback of the dryness level reaches the preset dryness target. If it does, the temperature mode is adjusted from the heating phase to the temperature stabilization phase.