Tobacco leaf raw material stability judgment method based on attention mechanism

Abstract

The present application provides a method for judging the stability of tobacco raw materials based on an attention mechanism, comprising the following steps: Step 1: Select several tobacco leaves of different grades as samples and collect tobacco leaf images; Step 2: Based on tobacco leaf images of different grades, construct a tobacco leaf grade classification model; Step 3: Use the tobacco leaf grade classification model in combination with the attention mechanism to obtain an attention degree heat map matrix; Step 4: Extract features of effective color regions of tobacco leaf images according to the attention degree heat map matrix; Step 5: Perform color analysis on the extracted effective color regions of tobacco leaf images to judge the stability of tobacco leaf grade colors. The present invention is based on a tobacco leaf grade classification model to extract effective tobacco leaf color regions, thereby making the feature extraction results more interpretable. In the present invention, the color of the key areas of the tobacco leaf image is calculated by a moving average method to calculate the average color difference characteristics of the tobacco leaf, which can objectively evaluate the color stability of the tobacco leaf raw materials.

Description

Technical Field

[0001] The present application belongs to the technical field of tobacco leaf detection, and in particular, relates to a method for determining the stability of tobacco leaf raw materials based on an attention mechanism. Background Art

[0002] At present, with the continuous development of artificial intelligence technology, machine vision technology has gradually been applied to various fields. Machine vision technology has the advantages of being fast, non-destructive, stable, and objective, which can greatly improve the accuracy and efficiency of detection. In tobacco leaf grading, machine vision technology can accurately measure and analyze the appearance, color, texture and other characteristics of tobacco leaves through image feature extraction technology, thereby achieving an objective evaluation of tobacco leaf quality.

[0003] Since tobacco leaves are natural products, their appearance and quality vary greatly, which makes it very difficult to extract tobacco leaf features. Traditional image feature extraction methods mainly extract features such as image edges, textures, colors, and shapes, which requires a tedious process of manually designing features. This method is not conducive to feature analysis of batches of tobacco leaves of different grades, especially color feature analysis, which makes it difficult to judge the stability of tobacco leaf raw materials of different batches and grades.

[0004] In summary, it is necessary to provide an improved technical solution to address the above-mentioned deficiencies in the prior art. Summary of the invention

[0005] One of the purposes of this application is to provide a method for judging the stability of tobacco leaf raw materials based on an attention mechanism, which can objectively evaluate the color stability of tobacco leaf raw materials.

[0006] To achieve the above-mentioned purpose and other related purposes, the present application provides a method for judging the stability of tobacco leaf raw materials based on an attention mechanism, comprising the following steps:

[0007] Step 1: Select several tobacco leaves of different grades as samples and collect tobacco leaf images;

[0008] Step 2: Based on the tobacco leaf images of different grades, a tobacco leaf grade classification model is constructed;

[0009] Step 3: Use the tobacco grade classification model and the attention mechanism to obtain the attention heat map matrix;

[0010] Step 4: Extract the features of the effective color regions of the tobacco leaf image based on the attention heat map matrix;

[0011] Step 5: Perform color analysis on the effective color area of ​​the extracted tobacco leaf image to determine the stability of the tobacco leaf grade color.

[0012] This application has at least the following beneficial effects:

[0013] The present invention is based on the tobacco leaf grade classification model, and extracts effective tobacco leaf color areas that have a greater impact on grade classification through an attention mechanism, reducing the interference of irrelevant information, thereby making the feature extraction results more interpretable.

[0014] In the present invention, the color of the key area of ​​the tobacco leaf image is calculated by a moving average method to calculate the average color difference characteristics of the tobacco leaf, which can objectively evaluate the color stability of the tobacco leaf raw material. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 It is a flow chart of a method for judging the stability of tobacco leaf raw materials based on an attention mechanism according to an embodiment of the present application;

[0017] Figure 2 is a tobacco leaf image shown according to an embodiment of the present application;

[0018] Figure 3 A tobacco leaf image, a combination of a tobacco leaf image and an attention heat map, and a corresponding attention heat map according to an embodiment of the present application;

[0019] Figure 4 A tobacco leaf image, a focus area, and a corresponding HSV color space diagram are shown according to an embodiment of the present application;

[0020] Figure 5 is a characteristic value curve of tobacco leaf images of various grades shown according to an embodiment of the present application;

[0021] Figure 6 3 is a color difference curve of the image characteristics of tobacco leaves of various grades shown in the embodiment of the present application. DETAILED DESCRIPTION

[0022] The following describes the implementation methods of the present application through specific specific embodiments, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific implementation methods, and the details in the present application can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0023] In the description of the present application, it should be noted that the terms "center", "longitudinal", "lateral", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0024] Reference Figure 1 This embodiment provides a method for determining the stability of tobacco leaf raw materials based on an attention mechanism, comprising the following steps:

[0025] Step 1: Select several tobacco leaves of different grades as samples and collect tobacco leaf images.

[0026] Different grades of tobacco leaves after industrial grading and sorting can be used as samples, and tobacco leaf images can be collected through online machine vision inspection equipment. The tobacco leaf images can be referred to Figure 2 The number of tobacco leaf images can exceed 10,000 to provide sufficient data support for building a tobacco leaf grade classification model in step 2.

[0027] Step 2: Based on tobacco leaf images of different grades, build a tobacco leaf grade classification model.

[0028] The collected tobacco leaf images are graded and classified to obtain a tobacco leaf image dataset. In this step, the industrial grading system of Shanghai Tobacco Group Co., Ltd. can be used to grade the collected tobacco leaf images to obtain a tobacco leaf image dataset containing five different grade codes C21, C22, C23, CXK and BK. The tobacco leaf image dataset is shown in the following table:

[0029] Detailed breakdown of the quantities of five different grades of tobacco leaves

[0030] Grade code C21 C22 C23 BK CXK Number of images 2896 4295 2565 600 722

[0031] The preset neural network model is trained and adjusted using the tobacco leaf image dataset to obtain a tobacco leaf image grade classification model. A grade classification model can be constructed using a deep learning network with an attention mechanism and the corresponding model parameters can be obtained.

[0032] Step 3: Use the tobacco grade classification model and the attention mechanism to obtain the attention heatmap matrix and attention heatmap. The attention mechanism is a technique in deep learning that is used to enhance the model's attention to key information in the input data. The attention heatmap matrix is ​​a visualization tool used to represent the attention weights assigned by the model to different input positions in the attention mechanism. s (F c ) can be calculated using the following method:

[0033] Calculate the channel attention weight M based on the original feature map F c (F). The calculation formula is M c (F) = sogmoid(MLP(MaxPool(F))+MLP(AvgPool(F))), where the original feature map F comes from the tobacco leaf image data obtained in step 1, MaxPool is the maximum pooling operation, AvgPool is the average pooling operation, MLP is a multilayer perceptron with a fully connected layer, and sigmoid is the activation function.

[0034] According to the channel attention weight F c (F) Calculate the feature map F of the channel attention mechanism c The calculation formula is

[0035] According to the feature map F of the channel attention mechanism c Calculate the attention heat map matrix M s (F c ). The calculation formula is M s (F c )=sigmoid(Conv([maxPool(F c ),AvgPool(F c )])), where MaxPool is the maximum pooling operation, [MaxPool(F c ),AvgPool(F c )] is the feature map F of the channel attention mechanism c The results of maximum pooling and average pooling are concatenated, Conv is the convolution operation, and sigmoid is the activation function.

[0036] In some embodiments, the attention mechanism includes a channel attention mechanism and a spatial attention mechanism. The channel attention mechanism focuses on the channel dimension of the feature map. It enables the model to pay more attention to important feature channels by learning the importance weight of each channel. The spatial attention mechanism focuses on the spatial dimension of the feature map. It enables the model to pay more attention to important areas of the image by learning the importance weight of each spatial position on the feature map. The attention heat map matrix is ​​obtained by combining the channel attention mechanism and the spatial attention mechanism.

[0037] Reference Figure 3 , Figure 3 The left picture is a tobacco leaf image, the right picture is an attention heat map, and the middle picture is a combination of the left and right pictures.

[0038] Step 4: Extract the effective color area features of the tobacco leaf image based on the attention heat map matrix.

[0039] The effective color region of the tobacco leaf image is also called the effective region of the tobacco leaf image. The feature extraction of the effective color region of the tobacco leaf image includes the following steps:

[0040] Using image processing technology to set attention threshold The attention heat map matrix M s (F c ) is converted into a binary image f, and the calculation formula is Among them, i, j corresponds to the specific position of each pixel in the binary image f.

[0041] The binary image f and the original tobacco leaf image I are bitwise ANDed to obtain the tobacco leaf image focus area. The tobacco leaf image focus area is extracted to obtain the tobacco leaf image effective color area U. Among them, the tobacco leaf image effective color area U is the tobacco leaf image area that affects the tobacco leaf grade classification. The calculation formula of the tobacco leaf image effective color area U is U=f.×I, f is the binary image, and I is the original tobacco leaf image.

[0042] The effective color area U of the tobacco leaf image is converted into the HSV color space, and the mean of each non-zero pixel in each channel in the HSV color space is calculated, so as to realize the feature extraction of the effective color area of ​​the tobacco leaf image. The calculation formula is as follows:

[0043] Where h, s, and v are the values ​​of pixels in three different channels of the HSV color space, and n is the number of pixels. Figure 4 , Figure 4 The left picture is a tobacco leaf image, the middle picture is the focus area of ​​the tobacco leaf image, and the right picture is the HSV color space.

[0044] Repeat steps 2 to 4 to obtain the HSV color space features of all tobacco leaf images.

[0045] Step 5: Perform color analysis on the effective color area of ​​the extracted tobacco leaf image. Color analysis is used to determine the stability of the grade and color of the tobacco leaf. Color analysis includes average color difference analysis.

[0046] The specific steps of average color difference analysis are as follows:

[0047] Obtain the HSV color space features of all images, and construct image feature sequences of different levels according to the acquisition time sequence. Examples of tobacco leaf image feature curves of different levels are shown in Figure 5 .

[0048] Based on the HSV channel color features of the tobacco leaf image in step 4, the color difference of tobacco leaf features of different grades can be calculated using the moving average method. Moving average is a method used in statistics to analyze data sequences. It shows the trend of data by creating a series of average values, which are the averages of a fixed number of consecutive data points. When calculating the color difference of tobacco leaf features of different grades, the moving average method can be used to smooth the data and reduce random fluctuations, so as to more clearly observe the trend of color difference over time or other variables.

[0049] The specific steps of calculating the color difference of tobacco leaf characteristics of different grades by moving average method are as follows:

[0050] 1) Determine the size of the window and step, where the window represents the number of data points and the step represents the number of data points. In the following steps, w is used to represent the window size and s is used to represent the step size.

[0051] 2) Starting from the first group of HSV value elements of the tobacco leaf characteristics of this grade, take the elements of the previous window group to calculate the average value and save it, and use the time domain t to distinguish the calculation results of the HSV values ​​under different windows. When t=0, the HSV value calculation formula is:

[0052]

[0053] 3) Move the window backward by the step group element, calculate the average value of the window group element at this time and save it. When t=t+s, the HSV value calculation formula is:

[0054]

[0055] Repeat 3) until the window moves to the end of the tobacco leaf feature sequence of this grade, and obtain the HSV average value sequence HSV of the tobacco leaf feature of this grade avg , if the length of the characteristic sequence of this grade of tobacco is L, the calculation formula is

[0056]

[0057] 4) Calculate the color difference between every two adjacent elements in the HSV average value sequence of the tobacco leaf characteristics of this grade, thereby obtaining the color difference sequence of the tobacco leaf characteristics of this grade and calculating the average color difference ΔE avg ,

[0058] Where ΔE i is the color difference between the i-th adjacent elements, n is the length of the color difference sequence, ΔE avg is the average color difference, then:

[0059] Repeat 2) to 4) to calculate the average color difference ΔE of all grades of tobacco leaves avg .

[0060] The color difference curves of tobacco leaf image features of different grades are shown in Figure 6 The average color difference values ​​of tobacco leaves of different grades are shown in Table 2. Here we take five different grades as examples:

[0061] Table 2 Average color difference of image features of tobacco leaves of five different grades

[0062] grade Average color difference BK 2.48 C21 1.31 C22 1.62 C23 1.59 CXK 2.87

[0063] Whether the tobacco raw materials of different grades are stable is judged according to the average color difference values ​​of the characteristics of tobacco leaves of different grades. If the average color difference value is less than 2, the tobacco raw materials of this grade are relatively stable. In Table 2, the tobacco raw materials of grades C21, C22 and C23 are relatively stable. If the average color difference value is ≥2, the stability of the tobacco raw materials of this grade is poor. In Table 2, the stability of the tobacco raw materials of grades BK and CXK is poor.

[0064] The method for judging the stability of tobacco leaf raw materials based on the attention mechanism provided by the present invention has the following advantages:

[0065] The present invention is based on the tobacco leaf grade classification model, and extracts effective tobacco leaf color areas that have a greater impact on grade classification through an attention mechanism, reducing the interference of irrelevant information, thereby making the feature extraction results more interpretable.

[0066] In the present invention, the color of the key area of ​​the tobacco leaf image is calculated by a moving average method to calculate the average color difference characteristics of the tobacco leaf, which can objectively evaluate the color stability of the tobacco leaf raw material.

[0067] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and substitutions can be made without departing from the technical principles of the present application. These improvements and substitutions should also be regarded as the scope of protection of the present application.

Claims

1. A method for judging the stability of tobacco leaf raw materials based on an attention mechanism, characterized in that: The steps include: Step 1: Select several tobacco leaves of different grades as samples and collect tobacco leaf images; Step 2: Based on the tobacco leaf images of different grades, a tobacco leaf grade classification model is constructed; Step 3: Use the tobacco grade classification model and the attention mechanism to obtain the attention heat map matrix; Step 4: Extract the features of the effective color regions of the tobacco leaf image based on the attention heat map matrix; Step 5: Perform color analysis on the effective color area of ​​the extracted tobacco leaf image to determine the stability of the tobacco leaf grade color.

2. The method for judging the stability of tobacco leaf raw materials according to claim 1, characterized in that: Step two includes classifying the collected tobacco leaf images to obtain a tobacco leaf image data set; using the tobacco leaf image data set to train and adjust the parameters of a preset neural network model to obtain a tobacco leaf image classification model.

3. The method for judging the stability of tobacco leaf raw materials according to claim 2, characterized in that: The steps of training and adjusting parameters of a preset neural network model using a tobacco leaf image dataset include constructing a hierarchical classification model using a deep learning network with an attention mechanism.

4. The method for judging the stability of tobacco leaf raw materials according to claim 1, characterized in that: Step three includes: Calculate channel attention weights based on the original feature map; Calculate the feature map of the channel attention mechanism according to the channel attention weight; Calculate the attention heat map matrix based on the feature map of the channel attention mechanism.

5. The method for judging the stability of tobacco leaf raw materials according to claim 4, characterized in that: The original feature map comes from the tobacco leaf image data obtained in step one.

6. The method for determining the stability of tobacco leaf raw materials according to claim 1, characterized in that: Step three includes: the attention mechanism includes a channel attention mechanism and a spatial attention mechanism, and the attention heat map matrix is ​​obtained by combining the channel attention mechanism and the spatial attention mechanism.

7. The method for determining the stability of tobacco leaf raw materials according to claim 4, characterized in that: Step 4 includes: Set the attention threshold and convert the attention heat map matrix into a binary image by combining image processing technology; Process the binary image and the original tobacco leaf image to obtain the tobacco leaf image focus area; Extract the focus area of ​​the tobacco leaf image to obtain the effective color area of ​​the tobacco leaf image; The effective color area of ​​the tobacco leaf image is converted into a color space, and the mean of each non-zero pixel point in each channel in the color space is calculated, thereby realizing feature extraction of the effective color area of ​​the tobacco leaf image.

8. The method for judging the stability of tobacco leaf raw materials according to claim 1, characterized in that: The color analysis in step five includes average color difference analysis.

9. The method for judging the stability of tobacco leaf raw materials according to claim 7, characterized in that: Step five includes: obtaining color space features of all images, and constructing image feature sequences of different levels according to the order of acquisition time; based on the channel color features of the tobacco leaf image in step four, using the moving average method to calculate the color difference of tobacco leaf features of different levels.

Images