Tobacco leaf grading method based on residual network, electronic device and storage medium
The tobacco leaf grading method based on a multi-branch attention mechanism residual network solves the problems of time-consuming, labor-intensive, and highly subjective manual grading in existing technologies, achieving efficient and accurate tobacco leaf grading and improving production efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- YUNNAN TOBACCO LEAF
- Filing Date
- 2022-09-14
- Publication Date
- 2026-07-03
AI Technical Summary
Existing tobacco leaf grading methods rely on manual experience, which is time-consuming, labor-intensive, and highly subjective, making it difficult to achieve rapid, stable, and objective tobacco leaf grading.
A tobacco leaf grading method based on a multi-branch attention mechanism residual network is adopted. By image preprocessing, data augmentation, and training of the multi-branch attention mechanism residual network model, the image quality and feature learning ability are improved, and the influence of background and small tobacco leaves is reduced.
It improved the accuracy and efficiency of tobacco leaf grading, reduced manual workload, and achieved rapid, stable, and objective tobacco leaf grading.
Smart Images
Figure CN115393649B_ABST
Abstract
Description
Technical Field
[0001] This document relates to the field of residual network technology, and in particular to a tobacco leaf grading method, electronic device and storage medium based on a multi-branch attention mechanism residual network. Background Technology
[0002] The production of tobacco leaves is influenced by a complex array of factors, resulting in varying quality. Only through grading can tobacco leaves of relatively consistent quality be grouped into the same grade, fully reflecting the quality of different types of tobacco. This allows for the rational utilization of tobacco resources according to the needs of the cigarette industry, maximizing the benefits of high-quality tobacco leaves and better promoting commercial operations. Currently, the grading method is manual, relying primarily on the experience of graders and visual and tactile sensory feedback to determine the grade. This makes grading time-consuming, labor-intensive, and highly subjective. Therefore, a fast, stable, objective, and accurate grading method is crucial.
[0003] In recent years, research on tobacco leaf grading methods has mainly focused on extracting features related to manual grading from tobacco leaf images, such as color, texture, and geometry, and then using certain classification methods for grading. For tobacco leaves of adjacent grades, their image features are similar, which can easily lead to a decrease in recognition rate and unstable grading results. Furthermore, this method relies on manual feature extraction, which is tedious. Summary of the Invention
[0004] This invention provides a tobacco leaf grading method, electronic device, and storage medium based on a multi-branch attention mechanism residual network, aiming to solve the above-mentioned problems.
[0005] This invention provides a tobacco leaf grading method based on a multi-branch attention mechanism residual network, comprising:
[0006] Collect images of tobacco leaves of different grades and classify and store them according to grade, and preprocess the tobacco leaf images;
[0007] The preprocessed tobacco leaf images are divided into training set, validation set and test set according to a specific ratio;
[0008] A multi-branch attention mechanism residual network model was constructed, and an optimizer, loss function, and monitoring metrics were set to assemble the multi-branch attention mechanism residual network model.
[0009] The training set and validation set are fed into the assembled multi-branch attention mechanism residual network model for training, and the trained multi-branch attention mechanism residual network model is obtained.
[0010] The trained multi-branch attention mechanism residual network model is used to classify the tobacco leaf images under test.
[0011] This invention provides an electronic device, comprising:
[0012] Processor; and,
[0013] A memory is configured to store computer-executable instructions, which, when executed, cause the processor to perform the steps of the tobacco leaf grading method based on a multi-branch attention mechanism residual network as described above.
[0014] This invention provides a storage medium for storing computer-executable instructions, which, when executed, implement the steps of the tobacco leaf grading method based on a multi-branch attention mechanism residual network as described above.
[0015] This invention employs a tobacco leaf grading method based on a multi-branch attention mechanism residual network. The residual structure facilitates network training while preventing gradient vanishing and exploding problems. Within the residual block, features are divided into two branches using an attention mechanism, enabling the learning of more features and strengthening the learning of important channel features. Image preprocessing before training and testing effectively reduces the influence of background and small tobacco leaf areas, ensuring image quality. Data augmentation ensures sufficient training data. This invention effectively improves tobacco leaf grading quality, reduces manual workload, and increases production efficiency. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in one or more embodiments of this specification or in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0017] Figure 1 This is a flowchart of the tobacco leaf grading method based on a multi-branch attention mechanism residual network according to an embodiment of the present invention;
[0018] Figure 2 This is a schematic diagram of the residual module in an embodiment of the invention. Detailed Implementation
[0019] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this document.
[0020] Method Implementation Examples
[0021] This invention provides a method for tobacco leaf grading based on a multi-branch attention mechanism residual network, comprising:
[0022] Step S101 involves acquiring images of tobacco leaves of different grades, classifying and storing them according to grade, and preprocessing the tobacco leaf images; Step S101 specifically includes:
[0023] Collect several tobacco leaf image samples of different grades. The specific operation is as follows: Lay multiple tobacco leaves of different grades flat on the tobacco leaf sorting conveyor belt, with the leaf tips and ends facing the same fixed direction, and take pictures for sampling. Classify and store the tobacco leaf images according to grade.
[0024] Data preprocessing includes:
[0025] Read the image and perform background removal processing. The specific steps are as follows: Read the image, convert it to grayscale, use the watershed algorithm in OpenCV to convert the image into a binary image with white tobacco leaves and a black belt background, and then multiply it with the original image to obtain the background-removed image.
[0026] Remove small areas of tobacco leaves and tobacco fragments from the image. Extract the tobacco leaf outlines from the image, calculate the area of each outline, and delete the outlines whose area is less than 20% of the image area. This will give you the image with small areas of tobacco leaves and tobacco fragments removed.
[0027] Resize the image to 448*448 pixels;
[0028] Step S102 involves dividing the preprocessed tobacco leaf images into a training set, a validation set, and a test set according to a specific ratio; Step S102 specifically includes:
[0029] The dataset is divided into training, validation, and test sets in a 15:3:2 ratio. The training set is used to train the model parameters, the validation set is used to provide feedback on the selection of model hyperparameters, and the test set is used to test the model's generalization ability.
[0030] Step S103 involves building a multi-branch attention mechanism residual network model, setting the optimizer, loss function, and monitoring metrics to assemble the multi-branch attention mechanism residual network model; Step S103 specifically includes:
[0031] The residual element is constructed, and its structural diagram is attached. Figure 2 As shown, a residual unit consists of two parts: F(x) is calculated from the input x through a series of operations, and Identity is like a line that directly passes x through it. Finally, they are added together to get F(x) + x. This operation can prevent gradient vanishing in the network.
[0032] Build the residual module, as shown in the attached document. Figure 2 As shown, the input x is first increased in dimensionality by a 1*1 convolution, then split into two branches, which are split into two feature maps F1 and F2 by a split operation. Then F1 and F2 are added together. The new feature map is then processed by an attention mechanism, first by a global polling operation to compress the feature map to 1*1 size, then by a Dense fully connected layer for dimensionality reduction, and then by a Dense fully connected layer for dimensionality increase. Then the r-softmax function is used to select weights, and then they are multiplied by F1 and F2 respectively to obtain new F1 and F2. Then F1 and F2 are added together, and finally F(x) is obtained by a 1*1 convolution, which is then added to the input x.
[0033] Stacked residual modules: Residual networks are composed of stacked residual modules of different numbers. By adjusting the number of stacked modules and the number of channels in different feature maps, ResNets with different numbers of layers can be obtained.
[0034] Replacing the 7x7 convolutions in ResNet with three 3x3 convolutions reduces the number of parameters without changing the receptive field size.
[0035] Step S104 involves feeding the training and validation sets into the assembled multi-branch attention mechanism residual network model for training, thereby obtaining the trained multi-branch attention mechanism residual network model. Step S104 specifically includes:
[0036] Model assembly involves setting the optimizer, loss function, and monitoring metric. The optimizer used is the Adam adaptive optimizer, the monitoring metric is accuracy, and the loss function is the multi-class cross-entropy loss function, as shown in the following formula:
[0037]
[0038] In the above formula, N represents the number of samples in the batch, K represents the number of labels, yi,k represents the true value of the i-th sample as the k-th label, log represents the natural logarithm, and pi,k represents the probability that the i-th sample is predicted as the k-th label value.
[0039] Model training involves feeding the training and validation sets into the network after the model is assembled. During training, data augmentation techniques such as rotation, translation, and flipping are used to augment the training set to fit the model.
[0040] Save the trained model structure and parameters, and you can use it directly by loading the model structure and parameters later;
[0041] Load the saved model, validate it using the validation set, and check the accuracy.
[0042] Step S105 involves classifying the tobacco leaf images under test using a trained multi-branch attention mechanism residual network model. Step S105 specifically includes:
[0043] The image of the tobacco leaf to be tested is preprocessed, and then the saved model is loaded to predict the image to obtain the probability prediction value of each grade of the image. The index with the highest probability is taken as the result of the model prediction.
[0044] In this embodiment of the invention, a certain type of tobacco leaf from region A after 2021 was selected. The tobacco leaf grade was determined manually by grading experts, and finally, six grades of tobacco leaves were selected: upper orange-yellow grade 1 tobacco (B1F), upper orange-yellow grade 2 tobacco (B2F), upper orange-yellow grade 3 tobacco (B3F), middle orange-yellow grade 1 tobacco (C1F), middle orange-yellow grade 2 tobacco (C2F), and middle orange-yellow grade 3 tobacco (C3F).
[0045] The grading model and ResNet50 model of this invention were used to predict the six grades of tobacco leaves, respectively. Table 1 shows the prediction results of the model of this invention, and Table 2 shows the prediction results of the ResNet50 model.
[0046] Table 1 Prediction results of the model of this invention
[0047]
[0048] Table 2. ResNet50 model prediction results
[0049]
[0050] As can be seen from the prediction results in Tables 1 and 2, the accuracy of tobacco leaf grading based on the multi-branch attention mechanism residual network model of this invention is as high as 97.8%, while the accuracy of the ResNet50 model is 92.29%. The method of this invention improves the accuracy by 5%, which verifies the effectiveness of the method of this invention.
[0051] The following beneficial effects are achieved by employing the embodiments of the present invention:
[0052] This invention presents a tobacco leaf grading method based on a multi-branch attention mechanism residual network. The residual structure facilitates network training while preventing gradient vanishing and gradient exploding problems. Within the residual block, features are divided into two branches and an attention mechanism is used, allowing for the learning of more features and strengthening the learning of important channel features. Image preprocessing is performed before image training and testing to effectively reduce the influence of background and small tobacco leaf areas, ensuring image quality. Data augmentation is also performed to ensure sufficient training data. This invention effectively improves the quality of tobacco leaf grading, reduces manual workload, and increases production efficiency.
[0053] Device Example 1
[0054] This invention provides an electronic device, comprising:
[0055] Processor; and,
[0056] A memory is configured to store computer-executable instructions, which, when executed, cause the processor to perform the steps of the method embodiments described above.
[0057] Device Example 2
[0058] This invention provides a storage medium for storing computer-executable instructions, which, when executed, perform the steps of the method embodiments described above.
[0059] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for grading tobacco leaves based on a multi-branch attention mechanism residual network, characterized in that, include: Images of tobacco leaves of different grades are collected, classified and stored according to grade, and the tobacco leaf images are preprocessed. The preprocessed tobacco leaf images are divided into training set, validation set and test set according to a specific ratio; A multi-branch attention mechanism residual network model is constructed, and an optimizer, loss function, and monitoring metric are set to assemble the multi-branch attention mechanism residual network model; The training set and validation set are fed into the assembled multi-branch attention mechanism residual network model for training, and the trained multi-branch attention mechanism residual network model is obtained. The trained multi-branch attention mechanism residual network model is used to classify the tobacco leaf images under test. The construction of the multi-branch attention mechanism residual network model specifically includes: The residual module is constructed as follows: the input x is first enlarged by a 1*1 convolution, then divided into two branches, which are split into two feature maps F1 and F2 by a split operation. The feature maps F1 and F2 are then added together. The new feature map after addition is processed by an attention mechanism. First, a global polling operation is performed to compress the feature map to a 1*1 size. Then, a Dense fully connected layer is used for dimensionality reduction, followed by a Dense fully connected layer for dimensionality enlargement. Then, the r-softmax function is used to select weights, and then they are multiplied by F1 and F2 respectively to obtain new feature maps F1 and F2. The new feature maps F1 and F2 are then added together, and finally, after a 1*1 convolution, F(x) is obtained. F(x) is then added to the input x. Stacking residual modules specifically involves stacking a specific number of completed residual modules to obtain ResNets with different numbers of layers.
2. The method according to claim 1, characterized in that, The acquisition of images of tobacco leaves of different grades specifically includes: Multiple tobacco leaves of different grades are laid flat on the conveyor belt of the tobacco sorting line, with the leaf heads and tips facing the same fixed direction for photographing and sampling.
3. The method according to claim 1, characterized in that, The preprocessing of the tobacco leaf images specifically includes: The image background removal process includes: reading the image, converting the image to grayscale, using the watershed algorithm in OpenCV to convert the image into a binary image with white tobacco leaves and black belt background, and then multiplying it with the original image to obtain the background-removed image. Removing small areas of tobacco leaves and tobacco fragments from an image involves: extracting the outlines of tobacco leaves in the image, calculating the area of each outline, and deleting outlines with an area less than 20% of the image area, thus obtaining an image with small areas of tobacco leaves and tobacco fragments removed. The image size after removing small areas of tobacco leaves and broken tobacco is scaled down to a specific pixel size.
4. The method according to claim 1, characterized in that, The optimizer, monitoring metrics, and loss function specifically include: The Adam adaptive optimizer was selected as the optimizer, accuracy was selected as the monitoring metric, and multi-class cross-entropy was used as the loss function. The cross-entropy was obtained using Equation 1. Official 1; Where N represents the number of samples in the batch, K represents the number of labels, and y i,k This indicates that the i-th sample is the true value of the k-th label, where log represents the natural logarithm, and p i,k This represents the probability that the i-th sample is predicted to be the k-th label value.
5. The method according to claim 1, characterized in that, During the process of feeding the training set and validation set into the network for training, the training set is rotated, translated, and flipped to augment the data for model fitting.
6. The method according to claim 1, characterized in that, After obtaining the trained multi-branch attention mechanism residual network model, the process further includes: validating the trained multi-branch attention mechanism residual network model using a validation set to obtain the accuracy.
7. The method according to claim 1, characterized in that, The classification of the tobacco leaf images under test using the trained multi-branch attention mechanism residual network model specifically includes: By inputting the tobacco leaf to be tested into a multi-branch attention mechanism residual network model, the probability prediction values of each level of the image are obtained, and the index with the highest probability is taken as the model prediction result.
8. An electronic device, comprising: processor; as well as, A memory is configured to store computer-executable instructions, which, when executed, cause the processor to perform the steps of the tobacco leaf grading method based on a multi-branch attention mechanism residual network as described in claims 1-7.
9. A storage medium for storing computer-executable instructions, which, when executed, implement the steps of the tobacco leaf grading method based on a multi-branch attention mechanism residual network as described in claims 1-7.