Neural network-based classification method for products in process of tobacco primary processing production line

Abstract

The invention specifically discloses a tobacco primary processing production line in-process product classification method based on a neural network, and the method comprises the steps: carrying out the image collection of various types of tobacco materials of primary processing production line in-process products, and carrying out the data preprocessing of the collected tobacco in-process product images, so as to construct and form a tobacco in-process product data set; establishing a convolutional neural network model, and dividing the tobacco product-in-process data set into a training set, a verification set and a test set to train the convolutional neural network model; and performing morphological feature extraction on a to-be-detected tobacco primary processing production line in-process product image through the trained convolutional neural network model, and performing classification identification according to the morphological features so as to realize accurate classification of tobacco materials. According to the method, the product classification efficiency of the tobacco primary processing production line can be improved, quality information such as formula components and cut tobacco components in the production process can be mastered in real time, and online feedback control and post-event quality tracing are facilitated.

Description

Technical Field

[0001] The present invention relates to the technical field of tobacco processing, and more specifically, to a method for classifying in-process products in a tobacco primary processing production line based on a neural network. Background Art

[0002] Tobacco materials in the primary processing production process, including tobacco leaves, tobacco stems, etc., are input into the production line as tobacco raw materials. After a series of processing procedures, they are transformed into the form of cut tobacco and cut stems. Before being processed into finished cut tobacco, according to the process technical requirements of different cigarette products, a certain proportion of cut stems and reconstituted tobacco leaves need to be blended into the cut tobacco respectively. At the same time, due to the different characteristics of tobacco leaves and tobacco stems of different grades and the influence of the cutting process during the processing, a small proportion of stem chips will be generated. Therefore, the finished cut tobacco contains components such as cut tobacco, cut stems, and reconstituted tobacco leaves. Whether these components are blended in proportion according to the different process technical requirements of different brand specifications of cigarettes has a direct impact on the internal quality stability of cigarettes. At the same time, a small amount of stem chips are negative influencing factors, and too many stem chips will affect both the internal quality stability and the appearance quality of cigarettes. Therefore, it is necessary to clearly identify the formula components and cut tobacco components online, and distinguish the proportions of tobacco leaves and tobacco stems in the formula and the proportions of cut tobacco, cut stems, reconstituted tobacco leaves, and stem chips in the cut tobacco, so as to facilitate the implementation of process control of the production line and post-event quality traceability. At present, the discrimination of the differences in in-process products in the tobacco primary processing production line has always been completed by professional personnel through manual visual distinction. Limited by the experience, state, and environment of professional personnel, the efficiency is low, the cost is high, and the reliability and real-time performance of the classification results are insufficient. Therefore, how to classify the in-process products in the tobacco primary processing production line conveniently and accurately is of great significance. Summary of the Invention

[0003] The present invention provides a method for classifying in-process products in a tobacco primary processing production line based on a neural network, which solves the problems that the classification of in-process product raw materials in the existing tobacco primary processing production line needs to be distinguished by manual visual inspection, with low efficiency, high cost, and insufficient reliability and real-time performance of the classification results. It can improve the convenience and accuracy of classifying in-process products in the tobacco primary processing production line, improve the classification efficiency and accuracy, and contribute to online feedback control and post-event quality traceability.

[0004] To achieve the above objectives, the present invention provides the following technical solutions:

[0005] A method for classifying in-process products in a tobacco primary processing production line based on a neural network, comprising:

[0006] Performing image acquisition on various tobacco materials of in-process products in a tobacco primary processing production line, and performing data preprocessing on the acquired in-process product images to construct an in-process product data set, where the tobacco materials include: tobacco leaves, tobacco stems, cut tobacco, cut stems, reconstituted tobacco leaves, and stem chips;

[0007] Build a convolutional neural network model, and divide the in-process product dataset into a training set, a validation set, and a test set to train the convolutional neural network model;

[0008] Extract morphological features from the in-process product images of the to-be-detected tobacco processing production line through the trained convolutional neural network model, and perform classification and recognition based on the morphological features to achieve accurate classification of materials.

[0009] Preferably, it further includes:

[0010] Introduce the FAC attention mechanism into the convolutional neural network model, and allocate weights through the FAC attention mechanism to perform frequency domain analysis on the in-process product images of the tobacco processing production line, thereby improving the morphological feature extraction ability of the convolutional neural network model.

[0011] Preferably, the step of allocating weights through the FAC attention mechanism to perform frequency domain analysis on the in-process product images of the tobacco processing production line includes:

[0012] Split the input features into n sub-blocks, and allocate different frequency components of the discrete cosine transform to each sub-block to decompose the input feature map into different frequency components, and convert the input channel information into frequency domain information;

[0013] Perform operations on the frequency domain features of each sub-block through a fully connected layer to calculate the corresponding weights;

[0014] For each sub-block, multiply the calculated weight element-wise with its corresponding channel feature to obtain the re-weighted feature;

[0015] Integrate the re-weighted results of all sub-blocks through summation to finally obtain the enhanced channel feature map.

[0016] Preferably, the step of building the convolutional neural network model includes:

[0017] Based on the Shufflenetv2 model, build the overall network structure;

[0018] The overall network structure includes: a basic unit and a downsampling unit. Add the FAC attention mechanism module after the 1×1 convolutional layer Conv in the left and right branches of the downsampling unit respectively, and introduce the FAC attention mechanism module after the 1×1 convolutional layer in the basic unit.

[0019] Preferably, the step of building the convolutional neural network model further includes:

[0020] Adopt the ReLu activation function in the ShuffleNetV2 model, and introduce the LeakyReLU function to improve the model performance.

[0021] Preferably, the overall network structure is provided with an image input layer, a convolutional layer, a max pooling layer, and a global pooling layer, and features are extracted by gradually downsampling through Stage2, Stage3, Stage4, and Stage5.

[0022] Preferably, the trained convolutional neural network model is used to extract morphological features from the in-process product images of the tobacco processing production line to be detected, and classification and recognition are performed according to the morphological features, including:

[0023] Features such as the color, appearance, and area of the in-process products of the tobacco processing production line are extracted according to the input in-process product images of the tobacco processing production line.

[0024] Preferably, the data preprocessing of the collected in-process product images of the tobacco processing production line includes:

[0025] The in-process product images of the tobacco processing production line are adjusted in size, enhanced in data, denoised, and normalized, so as to scale the images to a set size, and more training samples are generated by performing operations such as rotating, translating, flipping, and cropping on the images.

[0026] Preferably, the in-process product dataset is divided into a training set, a validation set, and a test set, including:

[0027] The training set, the validation set, and the test set are divided in a ratio of 80%, 10%, and 10%.

[0028] The present invention provides a method for classifying in-process products of a tobacco processing production line based on a neural network. By inputting the image data of various materials of the in-process products of the tobacco processing production line into a convolutional neural network, morphological features in the images are extracted by the convolutional layer, weights are assigned by the attention mechanism, the feature dimension is reduced by the pooling layer, and information is integrated by the fully connected layer, and finally, accurate classification of the in-process products of the tobacco processing production line is realized. It solves the problems that existing classification requires manual visual distinction, has low efficiency, high cost, and insufficient reliability and real-time performance of classification results, improves the classification efficiency and accuracy, and helps with online feedback regulation and post-event quality traceability. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the specific embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below.

[0030] Figure 1 is a schematic diagram of a method for classifying in-process products of a tobacco processing production line based on a neural network provided by the present invention.

[0031] Figure 2 is a schematic diagram of the structure of the basic unit Unit1 provided by an embodiment of the present invention.

[0032] Figure 3 It is a schematic structural diagram of the Unit2 downsampling unit provided by an embodiment of the present invention. Specific Embodiments

[0033] In order to enable those skilled in the art to better understand the solutions of the embodiments of the present invention, the following further details the embodiments of the present invention in conjunction with the accompanying drawings and embodiments.

[0034] Aiming at the problems of low efficiency, high cost, and insufficient reliability and real-time performance in the classification of various materials of in-process products in the current tobacco leaf processing production line, the present invention provides a method for classifying in-process products in a tobacco leaf processing production line based on a neural network, which solves the problems caused by manual visual distinction in existing classification, improves the classification efficiency and accuracy, and helps with online feedback regulation and post-event quality traceability.

[0035] As Figure 1 shown, a method for classifying in-process products in a tobacco leaf processing production line based on a neural network includes:

[0036] S1: Collect images of various tobacco materials of in-process products in a tobacco leaf processing production line, and perform data preprocessing on the collected in-process product images to construct an in-process product data set, where the tobacco materials include: tobacco leaves, tobacco stems, cut tobacco, stem cut tobacco, reconstituted tobacco, and stem chips.

[0037] S2: Establish a convolutional neural network model, and divide the in-process product data set into a training set, a validation set, and a test set to train the convolutional neural network model.

[0038] S3: Extract morphological features from the in-process product images of the tobacco leaf processing production line to be detected through the trained convolutional neural network model, and perform classification recognition according to the morphological features to achieve accurate classification of tobacco materials.

[0039] Specifically, during classification, first, a high-quality data set of various materials of in-process products in a tobacco leaf processing production line needs to be constructed. The construction and preprocessing of the data set are the key to the training of the deep learning model and are crucial for finally achieving accurate classification. Take pictures of different samples to obtain image data. Input the image data of various materials into the convolutional neural network, use the convolutional layer to extract the morphological features in the image, the attention mechanism to assign weights, the pooling layer to reduce the feature dimension, and the fully connected layer to integrate information, and finally achieve accurate classification of various material samples of in-process products in a tobacco leaf processing production line. The weights of the network will be optimized through the backpropagation algorithm to continuously improve the classification performance. This method not only has strong robustness when processing different tobacco material samples but also provides a stable and accurate classification basis for subsequent processing, improving the efficiency and accuracy of classifying various materials of in-process products in a tobacco leaf processing production line.

[0040] The method further includes: introducing the FAC attention mechanism into the convolutional neural network model, and allocating weights through the FAC attention mechanism to perform frequency domain analysis on the in-process product images of the tobacco processing production line, so as to improve the morphological feature extraction ability of the convolutional neural network model.

[0041] Further, the allocating weights through the FAC attention mechanism to perform frequency domain analysis on the in-process product images of the tobacco processing production line includes:

[0042] The input features are split into n sub-blocks, and different frequency components of the discrete cosine transform are allocated to each sub-block, so that the input feature map is decomposed into different frequency components, and the input channel information is converted into frequency domain information; the frequency domain features of each sub-block are all operated through a fully connected layer to calculate the corresponding weights; for each sub-block, the calculated weights are multiplied element-wise with their corresponding channel features to obtain re-weighted features; the re-weighted results of all sub-blocks are integrated through summation, and finally an enhanced channel feature map is obtained.

[0043] In practical applications, in order to improve the prediction accuracy of the Shufflenetv2 model, the FAC (Frequency Attention Convolution) attention mechanism is considered to be introduced. FAC is an attention mechanism proposed in the field of deep learning, which combines the characteristics of frequency domain analysis and convolutional neural networks to improve the feature extraction ability of the model. The core idea of the FAC mechanism is to apply attention weights in the frequency domain, so as to pay more attention to the frequency components that have important effects on the task in the image or signal. This mechanism has shown good effects in the fields of image processing, computer vision, signal processing, etc.

[0044] Further, the establishing of the convolutional neural network model includes:

[0045] Based on the Shufflenetv2 model, an overall network structure is established;

[0046] The overall network structure includes: basic units and downsampling units. The FAC attention mechanism module is respectively added after the 1×1 convolutional layer Conv of the left and right branches of the downsampling unit, and the FAC attention mechanism module is introduced after the 1×1 convolutional layer in the basic unit.

[0047] Further, the establishing of the convolutional neural network model also includes:

[0048] The ReLu activation function is adopted in the ShuffleNetV2 model, and the LeakyReLU function is introduced to improve the model performance.

[0049] Furthermore, the overall network structure is provided with an image input layer, a convolutional layer, a max pooling layer, and a global pooling layer, and feature extraction is gradually performed through downsampling by Stage2, Stage3, Stage4, and Stage5.

[0050] In practical applications, the image data of various material samples of the in-process products in the tobacco leaf processing production line is collected as a data set. The selected convolutional network classification model is based on Shufflenetv2. Its network is mainly composed of two unit modules, namely the Unit1 basic unit and the Unit2 downsampling unit. Its structure is as Figure 2 and Figure 3 shown. The overall network structure of Shufflenetv2 is shown in Table 1.

[0051]

[0052] The activation function adopted by ShuffleNetv2 is ReLU, and its calculation method is:

[0053] ; Its main advantages are simple calculation and high efficiency, which can avoid problems such as overfitting and gradient disappearance; however, this function still has certain deficiencies. Since the value on one side is 0, the negative gradient is also 0, and there may be a problem that all data cannot activate neurons, that is, there is a necrosis problem.

[0054] The LeakyReLU function can effectively solve this problem while maintaining the advantages of the original activation function by introducing a small slope (usually 0.01) in the negative part. Its calculation method is:

[0055] ; where α is the slope.

[0056] Improve the basic units with a stride of 1 and a stride of 2 in the ShuffleNetV2 model. In view of the unilateral inhibition phenomenon of the ReLU function used in the ShuffleNetV2 model, the LeakyReLU function is introduced to improve the performance of the model. Secondly, combined with the image features of the collected tobacco, the FAC attention module is added after the 1×1 convolutional layer (Conv) of the left and right branches of the downsampling unit respectively, and at the same time, the FAC is introduced after the 1×1 convolutional layer in the basic unit. It can better utilize the frequency domain information to mine the discriminative neurons in the downsampling layer and the basic unit of the network, that is, give greater weights to important feature channels and smaller weights to unimportant feature channels, thereby enhancing the attention of the downsampling unit and the basic unit, as Figure 2 and Figure 3 shown.

[0057] Without significantly increasing the number of parameters, the stacking times of the basic module in stage2, stage3, and stage4 are increased from the original 3, 7, 3 to 5, 9, 5 times. Since there are relatively high similarities among certain types of materials in the in-process products of the tobacco primary processing line, by increasing the stacking times of the basic unit, the network update is more delicate, better capturing the more subtle shape, area features, and tiny color differences among various materials, and fully learning more detailed information features of tobacco.

[0058] When using the FAC attention mechanism module, it is required that the number of channels be divisible by n. In the improvement, the first 16 low-frequency components are used. Therefore, the number of channels in each original layer needs to be modified from [24, 116, 232, 464, 1024] to [24, 128, 256, 512, 1024].

[0059] To demonstrate the advantages of the model, a comparative experiment can be conducted. In one embodiment, Shufflenetv2, Shufflenetv2-FAC, Mobilenetv3, Efficientnetv2, Xception, Squeezenet, Mnasnet, Maxvit, and Ghostnetv2 are trained and deployed in the same environment, and tested and evaluated using the same test dataset and evaluation metrics, as shown in Table 2:

[0060]

[0061] As can be seen from the table, Shufflenetv2-FAC is significantly in the leading position in this group of models. The overall performance of Shufflenetv2 is also very strong and is better than other models except the improved version in this regard, indicating that Shufflenetv2 is well-suited for the task of image classification of various materials in the in-process products of the tobacco primary processing line. Its improved version introduces the frequency domain attention mechanism (FAC) and makes improvements in its details. Compared with the original Shufflenetv2, this may be the key reason for its performance improvement. FAC can process features at different frequencies, making the model more sensitive to information at different scales, thereby enhancing the feature representation ability. This makes Shufflenetv2-FAC perform outstandingly in terms of metrics such as accuracy, recall rate, and F1 value, indicating that it can not only accurately identify positive examples but also maintain a balanced performance in multi-class tasks.

[0062] Further, the method of extracting morphological features from the in-process product images of the to-be-detected tobacco processing production line by using the trained convolutional neural network model and performing classification and recognition according to the morphological features includes: extracting features of the color, appearance, and area of the in-process products of the tobacco processing production line based on the input in-process product images of the tobacco processing production line, so as to perform precise classification of various material samples of the in-process products of the tobacco processing production line.

[0063] Further, the data preprocessing of the collected in-process product images of the tobacco processing production line includes: performing size adjustment, data augmentation, noise removal, and normalization processing on the in-process product images of the tobacco processing production line, so as to scale the images to a set size, and generating more training samples by performing operations such as rotating, translating, flipping, and cropping on the images.

[0064] Further, the method of dividing the in-process product dataset into a training set, a validation set, and a test set includes: dividing the training set, the validation set, and the test set in a ratio of 80%, 10%, and 10%.

[0065] In practical applications, in order to improve the training effect of the model, it is necessary to preprocess the collected images. The preprocessing mainly includes the following aspects:

[0066] ① Size adjustment: Scale the images to a unified size for input into the convolutional neural network. It is necessary to ensure that excessive distortion or blurring is not introduced during the adjustment process.

[0067] ② Data augmentation: Generate more training samples by performing operations such as rotating, translating, flipping, and cropping on the images, so as to improve the generalization ability and robustness of the model.

[0068] ③ Noise removal: Denoise the images to eliminate the noise introduced by factors such as lighting and shooting equipment, and improve the image quality.

[0069] ④ Normalization: Perform normalization processing on the images to make the pixel values distributed within a suitable range, which helps to accelerate the convergence speed of the model.

[0070] Divide the preprocessed dataset into a training set, a validation set, and a test set. The training set is used to train the model, the validation set is used to adjust the model parameters and hyperparameters, and the test set is used to evaluate the generalization performance of the model. Usually, the division can be made in a ratio of 80%, 10%, and 10%, but the specific ratio needs to be adjusted according to the size of the dataset and actual requirements.

[0071] When constructing the dataset of various materials of in-process products in a tobacco leaf processing production line, it may occur that the number of samples in some categories is much larger than that in other categories. This may lead to overfitting of the model to some categories during the training process, thus affecting the generalization performance. To solve this problem, data balancing is required, and the following methods can be adopted:

[0072] ① Resampling: Oversample the samples of the minority categories (such as replication, data augmentation, etc.) or undersample the samples of the majority categories (such as random deletion, clustering, etc.) to make the number of samples in each category close to balance.

[0073] ② Loss function adjustment: Assign higher weights to the samples of the minority categories to make the model pay more attention to these samples during the training process. Methods such as the class-weighted cross-entropy loss function can be used to achieve this.

[0074] After the above steps, the constructed and preprocessed dataset will lay a solid foundation for the training of subsequent deep learning models and the design of classification systems. During the model training process, the dataset can be continuously adjusted and optimized according to the actual situation to further improve the performance of the in-process product classification system of the tobacco leaf processing production line.

[0075] The dataset of various materials of in-process products in the produced tobacco leaf processing production line contains approximately 4,000 images, and data augmentation is used, that is, by performing operations such as transforming, rotating, and changing the hue on the original data, more data volume is generated. This can not only increase the number of the dataset but also increase the diversity of the dataset and improve the robustness of the model.

[0076] It can be seen that the present invention provides a method for classifying in-process products in a tobacco leaf processing production line based on a neural network. By inputting the image data of different materials into a convolutional neural network, using the convolutional layer to extract the morphological features in the image, the attention mechanism to assign weights, the pooling layer to reduce the feature dimension, and integrating information through the fully connected layer, the accurate classification of various material samples of in-process products in the tobacco leaf processing production line is finally realized. It solves the problems that existing various materials need to be distinguished by the naked eye of workers, with low efficiency, high cost, and insufficient reliability and real-time performance of classification results, improves the classification efficiency and accuracy, and helps with online feedback regulation and post-event quality traceability.

[0077] The above are only the preferred embodiments of the present invention, and do not impose any form of limitation on the present invention; any ordinary technician in the industry can smoothly implement the present invention according to what is shown in the accompanying drawings of the specification and the above; however, any equivalent changes made by those familiar with the technology in the industry within the scope of the technical solution of the present invention, using the technical content disclosed above and making some changes, modifications and evolutions, are all equivalent embodiments of the present invention; at the same time, any changes, modifications and evolutions made to the above embodiments based on the substantial technology of the present invention are still within the scope of protection of the technical solution of the present invention.

Claims

1. A method for classifying in-process products in a tobacco primary processing production line based on a neural network, characterized in that, Including: Performing image acquisition on various tobacco materials of the in-process products in the cigarette making production line, and performing data preprocessing on the acquired in-process product images to construct an in-process product dataset. The tobacco materials include: tobacco leaves, tobacco stems, cut tobacco, stem cut tobacco, reconstituted tobacco, and stem tips; Establishing a convolutional neural network model, and dividing the in-process product dataset into a training set, a validation set, and a test set to train the convolutional neural network model; Extracting morphological features from the in-process product images of the to-be-detected cigarette making production line through the trained convolutional neural network model, and performing classification and recognition according to the morphological features to achieve accurate classification of materials.

2. The method for classifying in-process products of a tobacco primary processing production line based on a neural network according to claim 1, wherein Also including: Introducing the FAC attention mechanism into the convolutional neural network model, and assigning weights through the FAC attention mechanism to perform frequency domain analysis on the in-process product images of the cigarette making production line, thereby enhancing the morphological feature extraction ability of the convolutional neural network model.

3. The method for classifying in-process products of a tobacco primary processing production line based on a neural network according to claim 2, wherein The assigning weights through the FAC attention mechanism to perform frequency domain analysis on the in-process product images of the cigarette making production line includes: Splitting the input features into n sub-blocks, and assigning different frequency components of the discrete cosine transform to each sub-block to decompose the input feature map into different frequency components, and converting the input channel information into frequency domain information; Performing operations on the frequency domain features of each sub-block through a fully connected layer to calculate the corresponding weights; For each sub-block, multiplying the calculated weight element-wise with its corresponding channel feature to obtain a re-weighted feature; Integrating the re-weighted results of all sub-blocks through summation to finally obtain an enhanced channel feature map.

4. The method for classifying in-process products of a tobacco processing production line based on a neural network according to claim 3, wherein The establishing of the convolutional neural network model includes: Based on the Shufflenetv2 model, establishing the overall network structure; The overall network structure includes: basic units and downsampling units. Adding the FAC attention mechanism module after the 1×1 convolutional layer Conv of the left and right branches of the downsampling unit respectively, and introducing the FAC attention mechanism module after the 1×1 convolutional layer in the basic unit.

5. The method for classifying in-process products of a tobacco primary processing production line based on a neural network according to claim 4, wherein The establishing of the convolutional neural network model also includes: Adopting the ReLu activation function in the ShuffleNetV2 model, and introducing the LeakyReLU function to improve the model performance.

6. The method for classifying in-process products of a tobacco processing production line based on a neural network according to claim 5, wherein The overall network structure is provided with an image input layer, convolutional layers, max pooling layers, and a global pooling layer, and features are gradually extracted through downsampling by Stage2, Stage3, Stage4, and Stage5.

7. The method for classifying in-process products of a tobacco primary processing production line based on a neural network according to claim 6, wherein The extracting morphological features from the in-process product images of the to-be-detected cigarette making production line through the trained convolutional neural network model, and performing classification and recognition according to the morphological features includes: Extracting morphological features of the color, appearance, and area of the in-process products in the cigarette making production line according to the input in-process product images of the cigarette making production line.

8. The method for classifying in-process products of a tobacco primary processing production line based on a neural network according to claim 7, characterized in that, The performing data preprocessing on the acquired in-process product images of the cigarette making production line includes: Performing size adjustment, data augmentation, noise removal, and normalization processing on the in-process product images of the cigarette making production line to scale the images to a set size, and generating more training samples by performing operations such as rotating, translating, flipping, and cropping the images.

9. The method for classifying in-process products of a tobacco primary processing production line based on a neural network according to claim 8, wherein, Dividing the in-process product data set into a training set, a validation set, and a test set includes: The training set, the validation set, and the test set are divided in a ratio of 80%, 10%, and 10%.

Images