Bundled tobacco leaf automatic grading method based on fine granularity classification

Abstract

The invention provides a bundled tobacco leaf automatic grading method based on fine-grained classification, and belongs to the technical field of tobacco leaf intelligent grading, and the method comprises the steps: obtaining a bundled tobacco leaf image through a bundled tobacco leaf image collection hardware platform, and carrying out the preprocessing; constructing a bundled tobacco leaf automatic grading network model; guiding data enhancement by using an attention map, and designing fine-grained grading loss; using anti-attention learning to supervise attention map quality; and detecting a to-be-detected bundled tobacco leaf image by using the trained bundled tobacco leaf automatic grading model to obtain a grading result of the bundled tobacco leaves. The method can achieve the grading accuracy of the bundled tobacco leaves to the maximum extent, can effectively improve the efficiency and objectivity of tobacco leaf purchasing, and promotes the process of automatic tobacco leaf purchasing.

Description

Technical Field

[0001] The invention belongs to the technical field of intelligent tobacco leaf grading, and in particular relates to an automatic grading method for bundled tobacco leaves based on fine-grained classification. Background Art

[0002] Tobacco leaves are the main raw material for Chinese cigarettes, but the quality of tobacco leaves varies, and their use value also varies. The purpose of tobacco leaf grading is to distinguish tobacco leaves of different qualities and maximize the value of tobacco leaves. Scientific and reasonable grading of tobacco leaves according to their quality is of great significance for the acquisition of tobacco leaves and the design of cigarette formulas. At present, the grading of tobacco leaves mainly relies on manual experience. The shortcomings of this method are obvious. Manual grading has problems such as subjectivity and environmental impact, low grading efficiency, and poor consistency. Therefore, it is urgent to study the use of technologies such as artificial intelligence and tools such as computers to assist or replace graders in conducting scientific and reliable automatic detection and grading of tobacco leaf quality.

[0003] In recent years, the research on tobacco leaf algorithms can be divided into two categories: methods based on traditional image processing and methods based on deep learning. Although many research results have been obtained by scholars at home and abroad, there are still many shortcomings. Tobacco leaf grading based on traditional methods relies too much on artificial tobacco leaf features, and most of them only extract shallow features (texture, veins, color, etc.), and cannot fully learn the characteristics of tobacco leaves. Tobacco leaf grading methods based on deep learning ignore the fine-grained feature learning of tobacco leaves and the retention of underlying features, and lack research on the automatic grading of bundled tobacco leaves. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides an automatic grading method for bundled tobacco leaves based on fine-grained classification, which solves the problem of low accuracy in tobacco leaf grading and fills the research gap in bundled tobacco leaf grading.

[0005] In order to achieve the above purpose, the technical solution adopted by the present invention is: an automatic grading method for bundled tobacco leaves based on fine-grained classification, comprising the following steps:

[0006] S1. Using a hardware platform for collecting images of bundled tobacco leaves, obtain images of bundled tobacco leaves and perform preprocessing;

[0007] S2. A double attention residual feature extraction network is designed to construct an automatic grading model for bundled tobacco leaves, and a feature map of the preprocessed bundled tobacco leaves image is extracted. A first feature matrix is ​​generated by bilinear attention pooling, and a predicted grade is obtained according to the first feature matrix.

[0008] S3. According to the extracted bundled tobacco leaf feature map, the attention map-guided data enhancement is used to obtain the local image, and a fine-grained grading loss function is defined according to the grade prediction result;

[0009] S4. Define anti-attention learning loss using anti-attention learning, and use anti-attention learning loss to supervise the quality of attention map;

[0010] S5. According to the processing results of step S3 and step S4, a trained automatic grading model for bunches of tobacco leaves is obtained, and the trained automatic grading model for bunches of tobacco leaves is used to detect the image of the bunches of tobacco leaves to be detected, so as to obtain a grading result of the bunches of tobacco leaves.

[0011] Furthermore, the step S1 includes the following steps:

[0012] S101, constructing a hardware platform for collecting images of bundled tobacco leaves; the hardware platform for collecting images of bundled tobacco leaves comprises a fixed light source, an industrial RGB camera, and a computer connected to the industrial RGB camera for collecting images of bundled tobacco leaves;

[0013] S102, using an industrial RGB camera to photograph the bundles of tobacco leaves transported on a conveyor belt, and automatically collecting images of the bundles of tobacco leaves according to a frame difference method;

[0014] S103, labeling the obtained bundled tobacco leaf images and dividing them into a training set, a validation set, and a test set;

[0015] S104: Perform data enhancement processing on the training set and the test set to complete preprocessing.

[0016] Furthermore, step S2 includes the following steps:

[0017] S201, inputting the preprocessed bundled tobacco leaf image into a dual attention residual feature extraction network ResCA to extract features, and obtaining a feature map of the bundled tobacco leaf image;

[0018] S202, generating an attention map using a convolutional layer according to the bundled tobacco leaf feature map, wherein the number of channels of the attention map is 32, and each channel represents a part of an object in the image;

[0019] S203, generating a first feature matrix by performing bilinear attention pooling on the attention map and the feature map;

[0020] S204. According to the first feature matrix, a first-level prediction probability vector is obtained using a fully connected layer, and the maximum probability is selected as the prediction level.

[0021] Furthermore, the dual attention residual feature extraction network ResCA includes a plurality of residual blocks, wherein the dual attention residual feature extraction network ResCA extracts a feature map of the bundled tobacco leaf image, which is specifically:

[0022] According to the preprocessed bundled tobacco leaf image, a convolution module is used to obtain the first shallow feature;

[0023] In the first residual block, the first shallow feature is taken as input, and the first high-level feature is obtained using the convolutional layer;

[0024] According to the first high-level feature, the convolutional attention module is used to perform element-wise addition of the first high-level feature after removing redundant features and the first shallow feature;

[0025] The second shallow features of the residual blocks except the first residual block are used as the output of the previous residual block, and the second high-level features are obtained by using the convolution layer. According to the second high-level features, the second high-level features after removing redundant features and the second shallow features are element-wise added by using the convolution attention module;

[0026] According to the element addition result obtained from the last residual block, the feature map of the bundled tobacco leaves image is obtained.

[0027] Furthermore, step S3 includes the following steps:

[0028] S301, according to the extracted bundle tobacco leaf feature map, using the attention map guided data enhancement to obtain the attention cropped local image and the attention removed local image respectively;

[0029] S302, obtaining prediction results Y2 and Y3 respectively according to the partial images of the attention-cropped partial image and the attention-removed partial image;

[0030] S303: Define a fine-grained grading loss function L according to the prediction level, prediction result Y2 and prediction result Y3. cls .

[0031] Furthermore, the attention map guided data enhancement is specifically as follows:

[0032] The extracted bundled tobacco leaf feature map is obtained and normalized to obtain the normalized attention map A. K * ;

[0033] According to the normalized attention map A K *, using the following formula, the key area of ​​the original bundle of tobacco leaves image is cropped to obtain the cropped area, and the cropped area is enlarged to the same size as the original bundle of tobacco leaves image to obtain the attention cropped local image:

[0034]

[0035] Among them, C K (i,j) represents the cropped element in the original bundled tobacco leaf image. Represents the attention map A K * The elements of θ c represents the critical clipping threshold;

[0036] According to the normalized attention map The following formula is used to remove the key areas of the original bundled tobacco leaf image, obtain the local image with attention removed, and complete the data enhancement processing:

[0037]

[0038] Among them, D K (i,j) represents the retained element in the original bundled tobacco leaf image, θ d Denotes the critical removal threshold.

[0039] Furthermore, the fine-grained classification loss function L cls The expression is as follows:

[0040]

[0041] L ce (x,y)=-[ylogx+(1-y)log(1-x)]

[0042] Among them, L ce () represents the cross entropy loss, x represents the prediction result, y represents the true label of the image, and Y1 represents the prediction level.

[0043] Furthermore, step S4 includes the following steps:

[0044] S401, taking a random attention map as a counter-attention map Among them, the size of the anti-attention map is consistent with the size of the attention map;

[0045] S402, the attention map The second feature matrix is ​​obtained by bilinear attention pooling with the bundled tobacco leaf feature map;

[0046] S403: According to the second feature matrix, a fully connected layer is used to obtain the second level prediction probability vector, and the maximum probability is selected as the prediction result.

[0047] S404, according to the prediction result Y and the prediction result Using the following formula, we can calculate Y effect , where the prediction result Y is the prediction level Y1, the prediction result Y2 and the prediction result Y3;

[0048]

[0049] Among them, Y effect represents the influence of the anti-attention map on the prediction result Y, A represents the attention map generated in step S202, X represents the feature map of the bundled tobacco leaves extracted in step S201, and Y(A=A,X=X) represents Y obtained by A and X. express and X gets

[0050] S405, Y effect Perform cross entropy loss L with the true label y ce () calculation, get the anti-attention learning loss L ca , supervise the quality of the attention map generated in step S202.

[0051] Furthermore, the anti-attention learning loss L ca The expression is as follows:

[0052] L ca =L ce (Y effect ,y)

[0053] Among them, L ce () represents the cross entropy loss, and y represents the true label of the image.

[0054] The beneficial effects of the present invention are:

[0055] (1) The present invention collects a dataset of bundled tobacco leaves in actual tobacco leaf purchasing scenarios. The collected data includes different parts and different grades, which promotes the development of bundled tobacco leaf research;

[0056] (2) Since multiple features (texture, oil content, etc.) are needed to judge the grade of tobacco leaves, the present invention uses a dual attention residual feature extraction network ResCA as a feature extraction network, and uses the attention mechanism to retain the important features of high-level features and effectively integrate them with shallow features to improve the tobacco leaf grading performance;

[0057] (3) We use attention maps to guide data augmentation and design a fine-grained hierarchical loss. We also use anti-attention learning to supervise the quality of attention maps and encourage the network to focus on discriminative fine-grained features. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 The figure is a flow chart of the method of the present invention.

[0059] Figure 2 This is a diagram of the equipment used to collect pictures of bundled tobacco leaves in this embodiment.

[0060] Figure 3 Schematic diagram of images of bundled tobacco leaves of different grades collected in this embodiment.

[0061] Figure 4 This is a network model diagram in this embodiment.

[0062] Figure 5 It is a specific module diagram of the network model in this embodiment.

[0063] Figure 6 This is a prediction confusion matrix diagram obtained by testing the test set using the trained network model proposed in this embodiment. DETAILED DESCRIPTION

[0064] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0065] Example

[0066] like Figure 1 As shown, the present invention provides an automatic grading method for bundled tobacco leaves based on fine-grained classification, and the implementation method thereof is as follows:

[0067] S1. Using the hardware platform for collecting bundled tobacco leaf images, obtain bundled tobacco leaf images and perform preprocessing. The implementation method is as follows:

[0068] S101, constructing a hardware platform for collecting images of bundled tobacco leaves; the hardware platform for collecting images of bundled tobacco leaves comprises a fixed light source, an industrial RGB camera, and a computer connected to the industrial RGB camera for collecting images of bundled tobacco leaves;

[0069] In this embodiment, Figure 2As shown in the grading station, the Youngface LED fill light is used as a fixed light source, the industrial RGB camera is Hikvision's MV-CS050-10GC, and the lens is a FA lens.

[0070] S102, using an industrial RGB camera to photograph the bundles of tobacco leaves transported on a conveyor belt, and automatically collecting images of the bundles of tobacco leaves according to a frame difference method;

[0071] In this embodiment, the original image of the bundled tobacco leaves is collected by:

[0072] The targets transported by the conveyor belt are photographed, including the actual placement and status of the bundled tobacco leaves when purchased. According to the width of the conveyor belt of 1m, the vertical shooting height of the test camera is 0.8m, and the system for collecting pictures uses the frame difference method to determine whether the pixel difference between the picture and the background picture when shooting is greater than the set threshold. The camera is always in the starting state. If the pixel difference between the current frame and the background picture is greater than the threshold, the camera starts taking pictures. If the pixel difference between the current frame and the background picture is not greater than the threshold, the camera stops taking pictures.

[0073] S103, labeling the obtained bundled tobacco leaf images and dividing them into a training set, a validation set, and a test set;

[0074] In this embodiment, the grading experts give image labels according to the GB2635-92 standard. Figure 3 As shown in the figure, a total of 5 grades of tobacco leaves were collected. The original image resolution was 2448x2048, and the image format was jpg. A total of 8194 images were collected. According to each category, they were randomly divided into training set, validation set and test set in a ratio of 8:1:1. There were 6555 images in the training set, 819 images in the validation set, and 820 images in the test set.

[0075] S104: Perform data enhancement processing on the training set and the test set to complete preprocessing.

[0076] In this embodiment, data enhancement is performed on both the training set and the test set. The enhanced training set has a total of 7786 images and the test set has 1639 images. The data size is adjusted to 512*512. Data enhancement includes horizontal flipping, small-angle ([-10°, 10°]) rotation, and adding salt and pepper noise. These are all possible situations that may occur in actual scenarios, so as to balance the samples and enhance the generalization of the model.

[0077] In this embodiment, the bundled tobacco leaves are collected when they are tied into a bundle for grading after grading. The industrial camera photographs the bundled tobacco leaves transported on the conveyor belt, and the system for collecting pictures automatically collects the pictures according to the frame difference method.

[0078] S2. By designing a dual attention residual feature extraction network, an automatic grading model for bundled tobacco leaves is constructed, and the feature map of the preprocessed bundled tobacco leaves image is extracted. The first feature matrix is ​​generated by bilinear attention pooling. According to the first feature matrix, the predicted grade is obtained. The implementation method is as follows:

[0079] S201, inputting the preprocessed bundled tobacco leaf image into a dual attention residual feature extraction network ResCA to extract features, and obtaining a feature map of the bundled tobacco leaf image;

[0080] In this embodiment, the dual attention residual feature extraction network ResCA has to go through 16 residual blocks in total; ResCA first passes the input image through a convolutional layer, a RELU layer, a BN layer, and a maximum pooling layer to obtain the shallow features of the first residual block. Figure 5 (a) In the first residual block, the shallow feature input is subjected to three convolution operations to obtain a high-level feature map. The high-level feature map first passes through the channel attention module. The channel attention is obtained by a dual-branch structure, consisting of maximum pooling, average pooling, multi-layer perceptron and sigmoid layer, to obtain important channel feature information; then it passes through the spatial attention module. The spatial attention is obtained by a single-branch structure, consisting of maximum pooling, average pooling, convolution layer and sigmoid layer, to obtain feature information of important areas; through the attention mechanism, the redundant features of the high-level features are filtered out, and then the elements are added to the input shallow features for feature fusion; the 16 residual blocks are a series structure, and the output of the previous residual block is used as the shallow feature input of the next residual block. According to the output of the last residual block, the feature map is obtained.

[0081] S202, generating an attention map using a convolutional layer according to the bundled tobacco leaf feature map, wherein the number of channels of the attention map is 32, and each channel represents a part of an object in the image;

[0082] In this example, the feature map is passed through a convolution layer, a BN layer, and a RELU layer to generate an attention map. The number of convolution kernels in the convolution layer is 32; each of the 32 channels in the attention map represents a key part of the image.

[0083] S203, generating a first feature matrix by performing bilinear attention pooling on the attention map and the feature map;

[0084] In this embodiment, the bilinear attention pooling (BAP) structure is as follows Figure 5As shown in (b), the number of attention map channels is 32, and each channel represents a part of the object in the image. First, the feature map is element-wise multiplied with each channel feature of the attention map, and the obtained result is normalized to generate 32 different local feature maps. The local feature maps are then globally average pooled to obtain more specific local feature information, and then the feature matrix is ​​obtained through splicing operations.

[0085] S204. According to the first feature matrix, a first-level prediction probability vector is obtained using a fully connected layer, and the maximum probability is selected as the prediction level.

[0086] S3. According to the extracted bundled tobacco leaf feature map, the attention map-guided data enhancement is used to obtain the local image, and the fine-grained grading loss function is defined according to the grade prediction result. The implementation method is as follows:

[0087] S301, according to the extracted bundle tobacco leaf feature map, using the attention map guided data enhancement to obtain the attention cropped local image and the attention removed local image respectively;

[0088] In this embodiment, reference Figure 4 , the attention map guides data enhancement, which is implemented as follows:

[0089] The extracted bundled tobacco leaf feature map is obtained and normalized to obtain the normalized attention map A. K * ;

[0090] According to the normalized attention map A K * , using the following formula, the key area of ​​the original bundle of tobacco leaves image is cropped to obtain the cropped area, and the cropped area is enlarged to the same size as the original bundle of tobacco leaves image to obtain the attention cropped local image:

[0091]

[0092] Among them, C K (i,j) represents the cropped element in the original bundled tobacco leaf image. Represents the attention map A K * The elements of θ c represents the critical clipping threshold;

[0093] According to the normalized attention map The following formula is used to remove the key areas of the original bundled tobacco leaf image, obtain the local image with attention removed, and complete the data enhancement processing:

[0094]

[0095] Among them, D K (i,j) represents the retained element in the original bundled tobacco leaf image, θ d represents the key removal threshold, θ d Belongs to (0.2, 0.5).

[0096] S302, obtaining prediction results Y2 and Y3 respectively according to the partial images of the attention-cropped partial image and the attention-removed partial image;

[0097] In this embodiment, the two local images in step S301 are both subjected to the ResCA network to extract fine-grained features, and then undergo bilinear attention pooling to finally obtain two prediction results Y2 and Y3.

[0098] S303: Define a fine-grained grading loss function L according to the prediction level, prediction result Y2 and prediction result Y3. cls .

[0099] In this embodiment, the fine-grained classification loss L cls As follows, the network is encouraged to learn fine-grained features of images:

[0100]

[0101] L ce (x,y)=-[ylogx+(1-y)log(1-x)]

[0102] Among them, L ce () represents the cross entropy loss, x represents the prediction result, y represents the true label of the image, and Y1 represents the prediction level.

[0103] In this embodiment, the prediction results of the global image and the prediction results of the local image are subjected to cross entropy loss with the true label of the image, and then the average value is taken, which promotes the network to focus on fine-grained features without losing global feature information.

[0104] S4. Define anti-attention learning loss using anti-attention learning, and use anti-attention learning loss to supervise the quality of local images. The implementation method is as follows:

[0105] S401: Use a random attention map as a counter-attention map Among them, the size of the anti-attention map is consistent with the size of the attention map;

[0106] In this embodiment, a shape and attention are first generated Figure 1 The tensor is a consistent tensor, and the elements in the tensor are set to random numbers in (0, 2). This tensor is used as the anti-attention map.

[0107] S402, the attention map The second feature matrix is ​​obtained by bilinear attention pooling with the bundled tobacco leaf feature map;

[0108] S403: According to the second feature matrix, a fully connected layer is used to obtain the second level prediction probability vector, and the maximum probability is selected as the prediction result.

[0109] S404, according to the prediction result Y and the prediction result Using the following formula, we can calculate Y effect , where the prediction result Y is the prediction level Y1, the prediction result Y2 and the prediction result Y3:

[0110]

[0111] Among them, Y effect represents the influence of the anti-attention map on the prediction result Y, A represents the attention map generated in step S202, X represents the feature map of the bundled tobacco leaves extracted in step S201, and Y(A=A,X=X) represents Y obtained by A and X. express and X gets

[0112] S405, Y effect Perform cross entropy loss L with the true label y ce () calculation, get the anti-attention learning loss L ca , supervise the quality of the attention map generated in step S202.

[0113] The anti-attention learning loss L ca The expression is as follows:

[0114] L ca =L ce (Y effect ,y).

[0115] Among them, L ce () represents the cross entropy loss, and y represents the true label of the image.

[0116] S5. According to the processing results of step S3 and step S4, a trained automatic grading model for bunches of tobacco leaves is obtained, and the trained automatic grading model for bunches of tobacco leaves is used to detect the image of the bunches of tobacco leaves to be detected, so as to obtain a grading result of the bunches of tobacco leaves.

[0117] In this embodiment, Figure 6 As shown, the 1639 test sets with data enhancement are subjected to step S2 to obtain prediction results, which are then compared with the true labels to calculate the prediction accuracy of each level and create a confusion matrix.

[0118] In order to verify the effectiveness of the method of the present invention, data collection was carried out at the actual tobacco factory acquisition site, and a data set was made for algorithm research. The data set of the present invention has 5 bundles of tobacco leaf grades, namely B2F, B3F, C2F, C3F, and C4F. After data enhancement, the total number of pictures is 10245. In order to verify the performance of the model, the data set is divided into a training set, a validation set, and a test set, and the training set and the test set are data enhanced, wherein the training set is 7786, the validation set is 820, and the test set is 1639. Regarding the optimization algorithm of the model, the SGD algorithm is adopted, the initial learning rate is set to 0.001, and the learning rate update strategy is StepLR.

[0119] The hardware equipment is as follows: processor Intel(R) Xeon(R) Silver 4110CPU@2.10GHz; memory (RAM) 32.0GB; independent graphics card, NVIDIA GeForce GTX 2080Ti; system type, Windows 10; development tools, Python and pytorch deep learning framework.

[0120] The evaluation indicators used are accuracy (acc) and macro-F1. The following are the two indicators:

[0121]

[0122] For example, the tobacco leaves in this bundle are of grade b2f, TP indicates the total number of b2f predicted as b2f, FP indicates the total number of other grades predicted as b2f, FN indicates the total number of all prediction errors that are not predicted as b2f, TN indicates the total number of other grades predicted correctly, so acc indicates the number of correct predictions in the total samples. Macro-F1 is evolved from F1-score. F1-score is the harmonic mean of precision and recall. Recall indicates the probability of being predicted as a correct positive sample, and recall indicates the proportion of positive samples in the correctly predicted samples. Bundling tobacco leaves is a multi-classification problem, so we use the extended macro-F1 of F1-score, F i represents the F1 score of each category, and n represents the total number of categories.

[0123] The implementation plan verification consists of three parts. 1) Using different classic CNNs plus fine-grained feature extraction, verify the effectiveness of extracting fine-grained features of tobacco leaves for classification; 2) Experimenting with the ResCA module and the anti-attention learning (CAL) module to verify their effectiveness; 3) Due to the lack of existing algorithms for bundled tobacco leaves, verification and comparison are conducted with general classification algorithms and advanced fine-grained classification algorithms.

[0124] 1. Using the classic CNN network as the backbone network, the performance of adding a fine-grained feature network structure (WSDAN) and directly extracting global features are compared, and the statistical experimental results are shown in Table 1. It can be seen that the performance of the two comparison groups has been greatly improved in obtaining fine-grained features, especially in comparison 1. After adding fine-grained feature extraction, the classification accuracy is directly improved by 7.38%, which shows the effectiveness of fine-grained feature extraction and learning for the classification of bundled tobacco leaves. Table 1 is a performance comparison table before and after adding fine-grained features.

[0125] Table 1

[0126]

[0127] 2. In order to verify the effectiveness of the ResCA module and the CAL module in grading the bunched tobacco leaves, this group of experiments conducted separate performance verifications on each module, as shown in Table 2. As can be seen from Table 2, both the ResCA model and the CAL model have improved the performance of the bunched tobacco leaves grading, and the performance of the two modules combined has also improved. The final model has an accuracy increase of 2.32% over the basic model, and the macro-F1 performance has increased by 1.76%, proving that the accuracy has been improved regardless of which category is classified. Table 2 is a table of the effectiveness of the ResCA and CAL modules.

[0128] Table 2

[0129]

[0130] 3. Compare with the existing algorithms to highlight the effectiveness of the method proposed in the present invention. Due to the lack of existing algorithms for grading bundled tobacco leaves, the present invention chooses to conduct experimental comparisons with the classic classification network, WSDAN algorithm, and bilinear pooling fine-grained classification networks HBP and B-CNN similar to WSDAN. The results are shown in Table 3. It can be seen that the performance of the algorithm proposed in the present invention is the best, and both performance indicators are optimal. Table 3 is a performance comparison table of the method of the present invention and other algorithms.

[0131] Table 3

[0132]

Claims

1. A method for automatic grading of bundled tobacco leaves based on fine-grained classification, characterized in that: The following steps are involved: S1. Using a hardware platform for collecting images of bundled tobacco leaves, obtain images of bundled tobacco leaves and perform preprocessing; S2. A double attention residual feature extraction network is designed to construct an automatic grading model for bundled tobacco leaves, and a feature map of the preprocessed bundled tobacco leaves image is extracted. A first feature matrix is ​​generated by bilinear attention pooling, and a predicted grade is obtained according to the first feature matrix. S3. According to the extracted bundled tobacco leaf feature map, the attention map-guided data enhancement is used to obtain the local image, and a fine-grained grading loss function is defined according to the grade prediction result; S4. Define anti-attention learning loss using anti-attention learning, and use anti-attention learning loss to supervise the quality of attention map; S5. According to the processing results of step S3 and step S4, a trained automatic grading model for bunches of tobacco leaves is obtained, and the trained automatic grading model for bunches of tobacco leaves is used to detect the image of the bunches of tobacco leaves to be detected, so as to obtain a grading result of the bunches of tobacco leaves.

2. The method for automatic grading of bundled tobacco leaves based on fine-grained classification according to claim 1 is characterized in that: The step S1 comprises the following steps: S101, constructing a hardware platform for collecting images of bundled tobacco leaves; the hardware platform for collecting images of bundled tobacco leaves comprises a fixed light source, an industrial RGB camera, and a computer connected to the industrial RGB camera for collecting images of bundled tobacco leaves; S102, using an industrial RGB camera to photograph the bundles of tobacco leaves transported on a conveyor belt, and automatically collecting images of the bundles of tobacco leaves according to a frame difference method; S103, labeling the obtained bundled tobacco leaf images and dividing them into a training set, a validation set, and a test set; S104: Perform data enhancement processing on the training set and the test set to complete preprocessing.

3. The method for automatic grading of bundled tobacco leaves based on fine-grained classification according to claim 1 is characterized in that: The step S2 comprises the following steps: S201, inputting the preprocessed bundled tobacco leaf image into a dual attention residual feature extraction network ResCA to extract features, and obtaining a feature map of the bundled tobacco leaf image; S202, generating an attention map using a convolutional layer according to the bundled tobacco leaf feature map, wherein the number of channels of the attention map is 32, and each channel represents a part of an object in the image; S203, generating a first feature matrix by performing bilinear attention pooling on the attention map and the feature map; S204. According to the first feature matrix, a first-level prediction probability vector is obtained using a fully connected layer, and the maximum probability is selected as the prediction level.

4. The method for automatic grading of bundled tobacco leaves based on fine-grained classification according to claim 3 is characterized in that: The dual attention residual feature extraction network ResCA includes a plurality of residual blocks, wherein the dual attention residual feature extraction network ResCA extracts a feature map of a bundle of tobacco leaves, which is specifically: According to the preprocessed bundled tobacco leaf image, a convolution module is used to obtain the first shallow feature; In the first residual block, the first shallow feature is taken as input, and the first high-level feature is obtained using the convolutional layer; According to the first high-level feature, the convolutional attention module is used to perform element-wise addition of the first high-level feature after removing redundant features and the first shallow feature; The second shallow features of the residual blocks except the first residual block are used as the output of the previous residual block, and the second high-level features are obtained by using the convolution layer. According to the second high-level features, the second high-level features after removing redundant features and the second shallow features are element-wise added by using the convolution attention module; According to the element addition result obtained from the last residual block, the feature map of the bundled tobacco leaves image is obtained.

5. The method for automatic grading of bundled tobacco leaves based on fine-grained classification according to claim 3 is characterized in that: The step S3 comprises the following steps: S301, according to the extracted bundle tobacco leaf feature map, using the attention map guided data enhancement to obtain the attention cropped local image and the attention removed local image respectively; S302, obtaining prediction results Y2 and Y3 respectively according to the partial images of the attention-cropped partial image and the attention-removed partial image; S303: Define a fine-grained grading loss function L according to the prediction level, prediction result Y2 and prediction result Y3. cls .

6. The method for automatic grading of bundled tobacco leaves based on fine-grained classification according to claim 5 is characterized in that: The attention map guided data enhancement is specifically as follows: The extracted bundled tobacco leaf feature map is obtained and normalized to obtain the normalized attention map A. K * ; According to the normalized attention map A K * , using the following formula, the key area of ​​the original bundle of tobacco leaves image is cropped to obtain the cropped area, and the cropped area is enlarged to the same size as the original bundle of tobacco leaves image to obtain the attention cropped local image: Among them, C K (i,j) represents the cropped element in the original bundled tobacco leaf image. Represents the attention map A K * The elements of θ c represents the critical clipping threshold; According to the normalized attention map The following formula is used to remove the key areas of the original bundled tobacco leaf image, obtain the local image with attention removed, and complete the data enhancement processing: Among them, D K (i,j) represents the retained element in the original bundled tobacco leaf image, θ d Denotes the critical removal threshold.

7. The method for automatic grading of bundled tobacco leaves based on fine-grained classification according to claim 5 is characterized in that: The fine-grained hierarchical loss function L cls The expression is as follows: L ce (x,y)=-[ylogx+(1-y)log(1-x)] Among them, L ce () represents the cross entropy loss, x represents the prediction result, y represents the true label of the image, and Y1 represents the prediction level.

8. The method for automatic grading of bundled tobacco leaves based on fine-grained classification according to claim 3 is characterized in that: The step S4 comprises the following steps: S401: Use a random attention map as a counter-attention map Among them, the size of the anti-attention map is consistent with the size of the attention map; S402, the attention map The second feature matrix is ​​obtained by bilinear attention pooling with the bundled tobacco leaf feature map; S403: According to the second feature matrix, a fully connected layer is used to obtain the second level prediction probability vector, and the maximum probability is selected as the prediction result. S404, according to the prediction result Y and the prediction result Using the following formula, we can calculate Y effect , where the prediction result Y is the prediction level Y1, the prediction result Y2 and the prediction result Y3: Among them, Y effect represents the influence of the anti-attention map on the prediction result Y, A represents the attention map generated in step S202, X represents the feature map of the bundled tobacco leaves extracted in step S201, and Y(A=A,X=X) represents Y obtained by A and X. express and X gets S405, Y effect Perform cross entropy loss L with the true label y ce () calculation, we get the anti-attention learning loss L ca , supervise the quality of the attention map generated in step S202.

9. The method for automatic grading of bundled tobacco leaves based on fine-grained classification according to claim 8, characterized in that: The anti-attention learning loss L ca The expression is as follows: L ca =L ce (Y effect ,y) Among them, L ce () represents the cross entropy loss, and y represents the true label of the image.

Images