Special-shaped smoke incremental classification method combining pyramid module and transfer learning

Abstract

The invention discloses a special-shaped smoke incremental classification method in combination with a pyramid module and transfer learning. In order to solve the problem that a traditional image classification neural network does not have continuous learning ability for added new classes, the invention provides an image classification method suitable for adding new classes to a training network, and pre-trained ResNet network knowledge is migrated through the thought of migration learning to be used for special-shaped cigarette data set classification. A pyramid module is embedded in the backbone network to improve the image global information extraction capability of the model; and in order to keep the continuous learning performance of the training model on the newly added category, performing fine tuning on the full connection layer of the backbone network to adapt to the learning of the new category of the special-shaped cigarettes. According to the method, the convergence speed and the training precision of the training model are improved to a great extent by utilizing migration of the model structure and the weight parameters and fine adjustment of the model.

Description

Technical Field

[0001] The invention belongs to the field of machine learning and image classification, and in particular relates to an incremental classification method for shaped smoke combining a pyramid module and transfer learning. Background Art

[0002] Special-shaped cigarettes are cigarettes with unique packaging and different sizes. In the tobacco industry, standard cigarettes are of equal size, with a size specification of (270-286)×(86-90)×(44-49)mm. Cigarettes with different specifications are called special-shaped cigarettes, or "special-shaped cigarettes" for short. Special-shaped cigarettes have always been popular among consumers due to their diverse flavors, unique packaging, and low tar content, and their sales have also been increasing year by year. With the increase in sales of special-shaped cigarettes, the classification and identification of special-shaped cigarettes has also brought great challenges to the sorting and distribution of tobacco companies. In order to ensure the normal operation of the sorting process line, it is very necessary to accurately classify the types of special-shaped cigarettes.

[0003] At present, there are several main methods for classifying data of special-shaped cigarette samples: 1. Manual detection classification. The manual detection method is the most primitive image classification method, but manual classification requires a lot of manpower and financial resources, the classification speed is slow, and it is easy to fatigue and cause classification errors when the time is too long. Therefore, in the face of the increasing number of special-shaped cigarette samples, the manual classification effect is not ideal; 2. Barcode classification. The barcode on the surface of special-shaped cigarettes contains more information and is easy to scan and identify. This method has high classification accuracy. However, due to the variety of special-shaped cigarette brands, different sizes, and variable positions, there may be dead zones during scanning, resulting in missed scanning, which causes system missed classification; 3. Neural network classification. With the rise of deep learning, neural networks have developed rapidly. They have been applied to the tobacco industry and achieved good research results. For example, Ding et al. identified incomplete cigarette barcodes by unifying SPA features and graphic space constraints, extracted and used SPA unified image features using feature pyramid networks (FPN), and then introduced graph convolutional networks to construct constraints, and then performed cigarette barcode recognition.

[0004] The classification and recognition algorithm of special-shaped cigarettes based on deep learning neural network can only perform one-time training on all special-shaped cigarette data sets. When faced with new category data, the model does not have the continuous learning ability of new categories. Since it is not possible to fully obtain all special-shaped cigarette category data in practical applications, with the increase of special-shaped cigarette category data, the traditional deep learning neural network method is no longer suitable for the special-shaped cigarette classification task in an open dynamic environment. Therefore, studying a classification network model that can quickly and accurately realize the incremental classification of special-shaped cigarette categories has important research significance and application value for the normal operation of the sorting link in the tobacco industry. Summary of the invention

[0005] The technical problem to be solved by the present invention is that the existing neural network method for classifying irregular smoke images does not have the ability to continuously learn new categories, and proposes an incremental classification method for irregular smoke that combines a pyramid module and transfer learning. The method has an existing neural network image classification program library and can process multiple types of data samples.

[0006] The technical solution of the present invention is an incremental classification method for shaped cigarettes that combines a pyramid module and transfer learning. The method mainly uses the idea of ​​transfer learning to transfer the model structure / weight parameters of the pre-trained network model knowledge in a large data set and use it in the classification task of shaped cigarettes in the target domain, by inserting a pyramid module and fine-tuning the model to adapt to the classification of newly added category data of shaped cigarettes.

[0007] The incremental classification method for irregular smoke combining the pyramid module and transfer learning also includes the following steps:

[0008] (1) Due to the problems of overlapping, incomplete, and severe light pollution in the captured images, it is necessary to clean and screen the samples to generate a dataset of special-shaped smoke images and divide them into a training set and a validation set;

[0009] (2) Using the idea of ​​transfer learning, the model structure and weight parameters of the ResNet network pre-trained in the large dataset ImageNet are transferred as the training network for the special-shaped smoke dataset;

[0010] (3) Since the amount of data collected for special-shaped smoke is too small, the loaded ResNet network needs to be improved to adapt to the learning of a small number of samples as the backbone network of the training model

[0011] (4) Since the special-shaped smoke dataset or the newly added categories may contain some similar sample data with slight differences from the original dataset, if only the ResNet18 network is used for feature extraction, the extracted feature information may be very similar, and the receptive field of the ResNet18 network is not as large as theoretically in actual applications, and the extraction of local detail features of the image is not comprehensive, resulting in the problem that the feature information extracted by the network cannot accurately classify similar categories. The pyramid module is embedded in the backbone network, and the feature maps of different levels are formed through this module to control the size of the receptive field. The feature maps are then upsampled and fused with the feature maps extracted by the backbone network, thereby achieving performance that takes into account both global and local features.

[0012] (5) Since the number of categories of special-shaped cigarettes is less than that of the large dataset ImageNet, in order to ensure that the training network is adapted to a small number of special-shaped cigarette categories and avoid the problem of network overfitting due to the small amount of data in the direct training model, and to adapt to the classification of newly added category data, the present invention effectively avoids model overfitting by fine-tuning the fully connected layer of the model and adding the Drout layer method, thereby realizing image classification.

[0013] The beneficial effects of the present invention compared with the prior art are as follows: Compared with the traditional neural network classification method, the training network of the present invention has the continuous learning ability of the newly added categories. By migrating the network model structure and parameters, it is not necessary to rebuild the classification network and training parameters in the task of special-shaped smoke classification, shortening the model training time, and inserting the pyramid module in the backbone network to expand the model's ability to extract global image information. Finally, the model is fine-tuned to adapt to the classification of the newly added categories, thereby improving the efficiency and accuracy of image classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a flow chart of the incremental classification method of special-shaped cigarettes of the present invention. DETAILED DESCRIPTION

[0015] In order to deepen the understanding of the method of the present invention, the invention will be further described below in conjunction with the drawings of the specification. The embodiment is only used to explain the present invention and does not constitute a limitation on the protection scope of the present invention.

[0016] like Figure 1 The figure shows an incremental classification method for irregular smoke combining a pyramid module and transfer learning according to the present invention, and the specific implementation steps are as follows:

[0017] (1) The dataset of irregular smoke images is formed by manually taking pictures. Due to the problems of overlapping, incomplete, and severe light pollution in the taken pictures, it is necessary to clean and screen the samples to generate the irregular smoke image dataset and divide it into a training set and a validation set.

[0018] (2) Transfer learning is a learning method that applies the knowledge in the known source domain to different but related target domain tasks. Through the idea of ​​transfer learning, the ResNet18 network model structure and parameters pre-trained on the large dataset ImageNet are transferred to the classification task of the heterogeneous smoke dataset.

[0019] (3) In the ResNet network structure, after the data enters the network, it first passes through a 7*7 convolutional layer with a step size of 2, resulting in a large number of data features not being utilized. In order to improve the accuracy of the classification of special-shaped smoke images, this paper improves the first layer of the ResNet18 network and replaces the 7*7 convolutional layer of this layer with three 3*3 stacked convolutional layers as the backbone network, reducing the amount of calculated parameters, adding nonlinear activation functions, and improving the image classification performance of the residual network model.

[0020] (4) A pyramid module is embedded in the backbone network. The module is used to form feature maps of different levels to control the size of the receptive field. The feature maps are then upsampled and fused with the feature maps extracted by the backbone network to achieve performance that takes into account both global and local features. The pyramid module is mainly achieved by fusing the original feature map with the features obtained by bilinear interpolation of upsampling of four pooling modules of different sizes: 1×1, 2×2, 3×3, and 6×6. In order to restore the pooled feature map to the same size as the input feature map, a 1×1 convolution layer is added after each level to perform dimensionality reduction, and the bilinear interpolation method in upsampling is used to obtain features of the same size as the original feature map. Finally, the feature maps are spliced ​​in the channel dimension to extract detailed features of similar categories in the special-shaped smoke dataset.

[0021] (5) Add new category samples. Since the number of categories of special-shaped smoke is less than that of the large dataset ImageNet, in order to ensure that the training network is adapted to a small number of special-shaped smoke categories and avoid the problem of network overfitting due to the small amount of data in the direct training model, it is necessary to fine-tune the fully connected layer of the model and add a Dropt layer to this layer to ignore some neurons to suppress the overfitting of the model. The original data and the new category data are classified. If the classification accuracy reaches the expected effect, the classification result is directly output. If it does not meet the requirements, the parameters are readjusted for training.

[0022] The experimental environment and experimental results used in the present invention are as follows:

[0023] Experimental environment: Windows 10 system and based on x64 processor environment, with 16G memory AMD Ryzen 7 4800H with Radeon Graphics CPU and NVIDIA GeForce RTX 3050 Laptop GPU. The construction, training and verification of the image classification network are carried out in Python language and PyCharm Community Edition 2022.2.1 software.

[0024] Experimental data and results: This paper constructs a special-shaped smoke dataset SmokeNet through artificial photography. The dataset collects 25 types of images with a size of 640×640 and divides them into training sets and validation sets. The pictures are stored in different folders according to different types, and the folders are named from 1 to 25. The experimental results show that this method has the advantages of fast convergence speed and high classification accuracy. The training classification accuracy of its model is over 98%.

[0025] When the experimental environment, screening conditions, data scale, image data set, etc. change, the image classification results will also change accordingly. The above implementation is only used to illustrate the technical solution of the invention rather than to limit it. Ordinary technicians in this field can modify or replace the solution of the invention without departing from the spirit and scope of the invention. The protection of the invention shall be based on the claims.

Claims

1. An incremental classification method for irregular smoke combining pyramid module and transfer learning, characterized in that The idea of ​​transfer learning is used to transfer the model structure / weight parameters of the pre-trained network model knowledge in a large dataset and use it in the classification task of special-shaped cigarettes in the target domain. The pyramid module is inserted and the model is fine-tuned to adapt to the classification of newly added category data of special-shaped cigarettes.

2. The incremental classification method for special-shaped smoke combining a pyramid module and transfer learning according to claim 1, characterized in that: The knowledge of the ResNet18 network model pre-trained in the large data set is transferred to the incremental classification task of special-shaped smoke, and the convolutional layer in front of the network is adjusted and improved as the backbone network of the training model.

3. The incremental classification method for special-shaped smoke combining pyramid module and transfer learning according to claim 1, characterized in that: The inserted pyramid module also includes: in the special-shaped smoke data set or the newly added category and the original data set, there may be similar sample data with some subtle differences. Only the ResNet18 network for feature extraction cannot accurately distinguish similar categories. Therefore, by embedding the pyramid module in the backbone network, the module forms feature maps of different levels to control the size of the receptive field, and then upsamples and fuses with the feature maps extracted by the backbone network, so as to achieve performance that takes into account both global features and local features, further improve the network information extraction capability, and thus improve the model training accuracy.

4. The incremental classification method for special-shaped smoke combining pyramid module and transfer learning according to claim 1, characterized in that: The fine-tuning model also includes: the number of classifications of special-shaped cigarettes is less than that of the large dataset ImageNet. In order to avoid the problem of model overfitting caused by small sample data and make the model adapt to the classification of newly added category data, the fully connected layer of the training network is fine-tuned and the Dropt layer is added to achieve accurate classification of the model.

Images