Cigarette authenticity identification method and system based on deep learning

Abstract

This invention discloses a method and system for identifying genuine and counterfeit cigarettes based on deep learning. The method divides a sample image into multiple image blocks using a sliding window, and then further divides each image block into multiple sub-image blocks using a grid. During model training, a pre-trained convolutional neural network is used to extract fine-grained features, which are then input into a recurrent neural network. The recurrent neural network is used to mine the correlations between the extracted features corresponding to multiple sub-image blocks. Based on the fine-grained features and the correlations between features, the authenticity of each image block is jointly determined, which greatly improves the accuracy of identifying the authenticity of each image block and ensures that the trained model has high recognition accuracy, thus enabling rapid and accurate identification of genuine and counterfeit cigarettes.

Description

Technical Field

[0001] This invention relates to the field of cigarette authenticity identification technology, and in particular, to a method and system for cigarette authenticity identification based on deep learning. Background Technology

[0002] my country has a massive consumption of tobacco products, and the tax revenue generated annually is a significant source of national finance. In recent years, counterfeit and substandard cigarettes have become widespread in the industry, with large quantities entering the market and seriously impacting the national economy, market order, and public health. Therefore, rapid and accurate identification of genuine and counterfeit cigarettes is of paramount importance. Currently, methods for identifying genuine and counterfeit cigarettes based on machine vision and machine learning technologies mainly include cigarette label registration based on feature points, image feature vector similarity calculation, feature vector classification based on traditional machine learning models, and image classification based on multi-layer neural networks. While these methods alleviate the low efficiency of manual identification in cigarette packaging authentication to some extent, they generally classify and identify cigarettes directly based on the overall shape image. The feature extraction granularity is relatively coarse, and the model does not yet learn the correlation between different features, which can lead to insufficient model feature learning ability, thus affecting the model's performance in identifying genuine and counterfeit cigarettes and resulting in low accuracy. Summary of the Invention

[0003] This invention provides a deep learning-based method and system for identifying genuine and counterfeit cigarettes, addressing the technical problems of low accuracy caused by coarse feature extraction granularity and lack of learning of the correlation between different features in existing cigarette authenticity identification methods.

[0004] According to one aspect of the present invention, a deep learning-based method for identifying genuine and counterfeit cigarettes is provided, comprising the following:

[0005] Step S1: Collect positive sample data and negative sample data and label them. Positive sample data includes scanned images of real cigarettes, and negative sample data includes scanned images of fake cigarettes.

[0006] Step S2: Divide each scanned image into multiple image blocks using a sliding window, and then perform gridding on each image block to obtain multiple sub-image blocks, thereby constructing a sub-image block dataset;

[0007] Step S3: Construct a cigarette authenticity identification model, wherein the cigarette authenticity identification model includes a pre-trained convolutional neural network for extracting features and a recurrent neural network for identifying authenticity based on the extracted features;

[0008] Step S4: Train the cigarette authenticity recognition model using the sub-image patch dataset until the model converges. During the training process, first use a pre-trained convolutional neural network to extract feature sets from multiple sub-image patches corresponding to each image patch, and then input the extracted feature sets into a recurrent neural network. The recurrent neural network identifies the authenticity of the image patch based on the correlation between the extracted features of multiple sub-image patches.

[0009] Step S5: Input the scanned image of the cigarette to be identified into the trained cigarette authenticity recognition model and output the cigarette authenticity recognition result.

[0010] Further, step S1 includes the following:

[0011] Step S11: Scan the front and back images of genuine and counterfeit cigarettes using a scanner device, wherein the cigarettes are in the shape of a carton or a box.

[0012] Step S12: Use the Canny edge operator to calculate the main outline information of the cigarette pack or box, and obtain the main area image of the cigarette pack or box to eliminate the background image of the scanner.

[0013] Further, step S2 includes the following:

[0014] Step S21: Fill the boundary of the scanned image to an integer multiple of the sliding window size, and cut the filled scanned image from left to right and from top to bottom with a specific step size and a preset overlap rate to obtain multiple image blocks;

[0015] Step S22: Perform grid-based segmentation on each image block to evenly divide each image block into multiple sub-image blocks, and construct the sub-image dataset corresponding to each image block in order from left to right and from top to bottom. The several sub-image datasets corresponding to all sample data constitute the sub-image block dataset.

[0016] Furthermore, step S2 includes the following content between steps S21 and S22:

[0017] Step S211: Remove featureless image blocks and low-feature image blocks.

[0018] Further, step S211 includes: converting all image blocks into grayscale images, calculating the total number of pixels in each image block, and removing the image block when the total number of pixels in a certain image block is lower than a preset threshold.

[0019] Furthermore, step S211 also includes: performing binary classification training on the ResNet50 neural network model by constructing a training dataset containing feature image patches and low feature image patches, and using the trained ResNet50 neural network model to filter out low feature image patches.

[0020] Alternatively, the image feature set can be extracted from the image patch using a pre-trained MobileNetV3-Small model, and then the K-Means clustering algorithm can be used to cluster the image patch into high-feature image patches and low-feature image patches, thereby completing the filtering of low-feature image patches.

[0021] Further, step S5 includes the following:

[0022] Step S51: Obtain a scanned image of the cigarette to be identified, and select the ROI region in the scanned image;

[0023] Step S52: Randomly crop out multiple non-overlapping image blocks from the ROI region, and perform gridding on each image block to obtain multiple sub-image blocks;

[0024] Step S53: Input the multiple sub-image blocks corresponding to each image block into the trained cigarette authenticity recognition model, and output the authenticity recognition result of each image block;

[0025] Step S54: Based on the authenticity recognition results of multiple image blocks, a voting method is used to obtain the final cigarette authenticity recognition result.

[0026] In addition, the present invention also provides a deep learning-based cigarette authenticity identification system, comprising:

[0027] The sample data collection module is used to collect positive sample data and negative sample data and perform labeling processing. Positive sample data includes scanned images of real cigarettes, and negative sample data includes scanned images of fake cigarettes.

[0028] The dataset construction module is used to divide each scanned image into multiple image blocks using a sliding window, and to perform gridded segmentation on each image block to obtain multiple sub-image blocks, thereby constructing a sub-image block dataset;

[0029] The model building module is used to build a cigarette authenticity identification model, wherein the cigarette authenticity identification model includes a pre-trained convolutional neural network for extracting features and a recurrent neural network for identifying authenticity based on the extracted features.

[0030] The model training module is used to train the cigarette authenticity recognition model using the sub-image patch dataset until the model converges. During the training process, a pre-trained convolutional neural network is first used to extract the feature set from multiple sub-image patches corresponding to each image patch. Then, the extracted feature set is input into the recurrent neural network. The recurrent neural network identifies the authenticity of the image patch based on the correlation between the extracted features of multiple sub-image patches.

[0031] The authenticity recognition module is used to input the scanned image of the cigarette to be identified into the trained cigarette authenticity recognition model and output the authenticity recognition result.

[0032] The present invention has the following effects:

[0033] The deep learning-based cigarette authenticity identification method of this invention divides a sample image into multiple image blocks using a sliding window, and then further divides each image block into multiple sub-image blocks using a grid. During model training, a pre-trained convolutional neural network is used to extract fine-grained features, which are then input into a recurrent neural network. The recurrent neural network is used to mine the correlation between the extracted features corresponding to multiple sub-image blocks, thereby jointly deciding the authenticity of each image block based on the fine-grained features and the correlation between features. This greatly improves the accuracy of authenticity identification for each image block, ensuring that the trained model has high recognition accuracy, thus enabling fast and accurate identification of cigarette authenticity.

[0034] In addition, the deep learning-based cigarette authenticity identification system of the present invention also has the above-mentioned advantages.

[0035] In addition to the objectives, features, and advantages described above, the present invention has other objectives, features, and advantages. The invention will now be described in further detail with reference to the figures. Attached Figure Description

[0036] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0037] Figure 1 This is a flowchart illustrating a preferred embodiment of the deep learning-based method for identifying genuine and counterfeit cigarettes.

[0038] Figure 2 This is a schematic diagram of a sub-process of step S1 in a preferred embodiment of the present invention.

[0039] Figure 3 This is a schematic diagram of a sub-process of step S2 in a preferred embodiment of the present invention.

[0040] Figure 4 This is a schematic diagram of a preferred embodiment of the present invention, in which an image block is uniformly divided into multiple sub-image blocks.

[0041] Figure 5 This is a schematic diagram of a sub-process of step S2 in another embodiment of the present invention.

[0042] Figure 6 This is a schematic diagram of the training log of the cigarette authenticity identification model in a preferred embodiment of the present invention.

[0043] Figure 7 This is a schematic diagram of a sub-process of step S5 in a preferred embodiment of the present invention.

[0044] Figure 8 This is a schematic diagram of randomly cropping multiple non-overlapping image blocks from the ROI region of the cigarette pack to be identified in a preferred embodiment of the present invention.

[0045] Figure 9 This is a schematic diagram of the module structure of a deep learning-based cigarette authenticity identification system according to another embodiment of the present invention. Detailed Implementation

[0046] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, the present invention can be implemented in many different ways as defined and covered below.

[0047] like Figure 1 As shown, a preferred embodiment of the present invention provides a deep learning-based method for identifying genuine and counterfeit cigarettes, comprising the following:

[0048] Step S1: Collect positive sample data and negative sample data and label them. Positive sample data includes scanned images of real cigarettes, and negative sample data includes scanned images of fake cigarettes.

[0049] Step S2: Divide each scanned image into multiple image blocks using a sliding window, and then perform gridding on each image block to obtain multiple sub-image blocks, thereby constructing a sub-image block dataset;

[0050] Step S3: Construct a cigarette authenticity identification model, wherein the cigarette authenticity identification model includes a pre-trained convolutional neural network for extracting features and a recurrent neural network for identifying authenticity based on the extracted features;

[0051] Step S4: Train the cigarette authenticity recognition model using the sub-image patch dataset until the model converges. During the training process, first use a pre-trained convolutional neural network to extract feature sets from multiple sub-image patches corresponding to each image patch, and then input the extracted feature sets into a recurrent neural network. The recurrent neural network identifies the authenticity of the image patch based on the correlation between the extracted features of multiple sub-image patches.

[0052] Step S5: Input the scanned image of the cigarette to be identified into the trained cigarette authenticity recognition model and output the cigarette authenticity recognition result.

[0053] It is understood that the deep learning-based cigarette authenticity identification method in this embodiment divides the sample image into multiple image blocks through a sliding window, and then divides each image block into multiple sub-image blocks through a grid. During the model training process, a pre-trained convolutional neural network is first used to extract fine-grained features, and then the extracted fine-grained features are input into a recurrent neural network. The recurrent neural network is used to mine the correlation between the extracted features corresponding to multiple sub-image blocks, so as to jointly decide the authenticity of each image block based on the fine-grained features and the correlation between features, which greatly improves the authenticity identification accuracy of each image block and ensures that the trained model has a high recognition accuracy, so as to quickly and accurately identify the authenticity of cigarettes.

[0054] Understandable, such as Figure 2 As shown, step S1 includes the following:

[0055] Step S11: Scan the front and back images of genuine and counterfeit cigarettes using a scanner device, wherein the cigarettes are in the shape of a carton or a box.

[0056] Step S12: Use the Canny edge operator to calculate the main outline information of the cigarette pack or box, and obtain the main area image of the cigarette pack or box to eliminate the background image of the scanner.

[0057] Specifically, a high-resolution scanner is used to scan the front and back images of genuine and counterfeit cigarettes, with an image resolution of 1200 DPI. The scanned images of genuine cigarettes serve as positive sample data, and the scanned images of counterfeit cigarettes serve as negative sample data. The positive and negative sample images are manually labeled. Positive and negative sample data from different brands of cigarettes can be collected simultaneously, or only one brand can be collected at a time. To improve the model's recognition accuracy, it is preferable to collect positive and negative sample data from one brand of cigarettes at a time for model training; that is, one model corresponds to the identification of genuine and counterfeit cigarettes for one brand. Furthermore, cigarettes are generally sold in cartons or boxes; this embodiment uses boxes as an example. Since the scanner background area appears in the scanned image, image preprocessing is required to remove it to prevent its features from affecting the subsequent model training accuracy. Specifically, the Canny edge operator is used to calculate the main contour information of the boxed cigarettes. The four vertices of the boxed cigarettes are calculated based on the positional relationships between the contour points, resulting in the main body image of the boxed cigarettes, thus eliminating the scanner background area image.

[0058] Understandable, such as Figure 3 As shown, step S2 includes the following:

[0059] Step S21: Fill the boundary of the scanned image to an integer multiple of the sliding window size, and cut the filled scanned image from left to right and from top to bottom with a specific step size and a preset overlap rate to obtain multiple image blocks;

[0060] Step S22: Perform grid-based segmentation on each image block to evenly divide each image block into multiple sub-image blocks, and construct the sub-image dataset corresponding to each image block in order from left to right and from top to bottom. The several sub-image datasets corresponding to all sample data constitute the sub-image block dataset.

[0061] Specifically, since the resolution of the cigarette box main area image obtained in step S1 is very high, the higher the resolution, the more information it contains, which is more beneficial to improving the accuracy of cigarette authenticity identification. However, if the entire image is input into the neural network for training, it will be a huge challenge to the GPU memory, and may even make it impossible to train the model. Therefore, in step S2, the boundary of the main area image of the cigarette box is first filled to an integer multiple of the sliding window size to facilitate image sliding window segmentation. Then, the image is segmented using a rectangular window size of 2048*2048, dividing the main area image of the cigarette box into multiple image blocks. Moreover, considering the difficulty in obtaining negative sample data and the limited amount of negative sample data, in order to improve the amount of sample data and the accuracy of model training, the main area image of the cigarette box is segmented by sliding window from left to right and from top to bottom with a step size of 1024 (i.e., 50% overlap rate) to obtain several image blocks. However, even after dividing the main area image of the cigarette pack into multiple image blocks, the granularity of the image blocks remains relatively coarse, resulting in coarse-grained features that can be extracted. The model struggles to capture finer-grained features indicating cigarette authenticity and to establish relationships between these fine-grained features. Therefore, in step S22, each image block obtained in step S21 is divided into grids, uniformly dividing the image block into several sub-image blocks. Sub-image datasets are then constructed for each image block in a left-to-right, top-to-bottom order. These sub-image datasets, corresponding to all sample data, constitute the sub-image block dataset. For example, as... Figure 4 As shown, a 2048*2048 image block P is uniformly divided into 16 512*512 sub-image blocks, and the sub-image dataset [P0, P1, P2, ..., P15] corresponding to each image block is constructed in order from left to right and from top to bottom. The features between adjacent sub-image blocks are continuous. For example, P0, P1, and P2 are three sub-image blocks with continuous positions, and their features are continuous. Therefore, the correlation between multiple sub-image blocks with continuous positions can be learned by deep learning model to provide joint features for genuine and counterfeit identification, which is beneficial to improving the accuracy of cigarette genuine and counterfeit identification.

[0062] Understandable, such as Figure 5As shown, in another embodiment of the present invention, step S2 further includes the following between step S21 and step S22:

[0063] Step S211: Remove featureless image blocks and low-feature image blocks.

[0064] Specifically, the multiple image patches obtained by sliding window segmentation in step S21 include feature-rich image patches, featureless image patches, and low-feature image patches. Blank areas are featureless image patches, and low-feature image patches refer to image regions with a small number of extractable true / false distinguishing features. Feature-rich image patches, on the other hand, refer to image regions with a large number of extractable true / false distinguishing features. If both featureless and low-feature image patches are input into the model for training, it increases the amount of data, reducing the training convergence speed, and introduces invalid or unrecognizable features, thus reducing the model's recognition accuracy. Therefore, in step S2 of this embodiment, after obtaining multiple image patches by sliding window segmentation, featureless and low-feature image patches are removed, retaining only feature-rich image patches. This improves the training speed and recognition accuracy of the subsequent model.

[0065] Step S211 includes: converting all image blocks to grayscale images, calculating the total number of pixels in each image block, and removing an image block when the total number of pixels in a certain image block is lower than a preset threshold. The preset threshold can be set based on empirical values ​​or on the statistical regularity of the total number of pixels in all image blocks.

[0066] Furthermore, the aforementioned methods for removing featureless and low-feature image blocks are rather coarse, and the preset thresholds are difficult to set accurately. Therefore, while they are effective at filtering featureless image blocks, they are less effective at removing low-feature image blocks. Thus, step S211 further includes: training a ResNet50 neural network model for binary classification using a training dataset containing both feature and low-feature image blocks; and using the trained ResNet50 neural network model to filter out low-feature image blocks, thereby accurately filtering them out. Additionally, since the above classification scheme requires manual data annotation, as an alternative, step S211 further includes: first extracting image features from image blocks using a pre-trained MobileNetV3-Small model; then clustering the extracted feature sets. Here, the K-Means algorithm is used to cluster image blocks into two categories: high-feature blocks and low-feature blocks, thereby completing the filtering of low-feature image blocks. Compared to the ResNet50 binary classification scheme, this scheme does not require manual data labeling or deep learning model training. After testing, this clustering scheme also has high accuracy and can be used in practical applications.

[0067] Preferably, step S2 further includes the following after step S22:

[0068] Step S23: Perform image enhancement processing on the sub-image blocks, such as scaling the image size, randomly flipping the image, and normalizing the image data, using conventional image enhancement processing methods.

[0069] It is understood that in step S2, by first segmenting the scanned image of the cigarette into multiple image blocks and removing featureless and low-feature image blocks, retaining only the feature-rich image blocks, the training speed and recognition accuracy of the subsequent model are improved. Then, the feature-rich image blocks are further divided into multiple sub-image blocks using a grid. Although the gridded sub-image blocks may contain featureless sub-image blocks, they are not removed or individually labeled. Instead, all gridded sub-image blocks are used for feature extraction and image block authenticity identification. The purpose is to preserve the continuity between adjacent sub-image blocks, thereby preserving the correlation between features and improving the accuracy of image block authenticity identification. Therefore, the two-layer image segmentation and image filtering method of this invention ensures that image detail features are not compressed, preserves the detail features of genuine and counterfeit cigarettes and the correlation between features for model learning, greatly improving the accuracy of cigarette authenticity identification.

[0070] It is understood that in step S3, a pre-trained convolutional neural network is used to extract features in the constructed cigarette authenticity identification model, and a recurrent neural network is used for authenticity identification. The pre-trained convolutional neural network for feature extraction can use neural network models such as VGGNet, GoogLeNet, and DenseNet. This invention preferably uses a deep residual network (ResNet) for image recognition to create the basic feature extractor. This network has been pre-trained on the ImageNet dataset. Transfer learning is performed on the cigarette authenticity training dataset based on the pre-trained model weights until the model converges, ultimately obtaining a basic image feature extraction model. Based on this model, features are extracted from the input sub-image patch dataset to obtain the feature set [F0, F1, F2, ..., F15] for each image patch. Commonly used recurrent neural networks include LSTM, BiLSTM, and GRU models. This invention preferably uses a BiLSTM model to perform authenticity identification on the feature set extracted by the deep residual network (ResNet).

[0071] It is understood that in step S4, the sub-image patch dataset is divided into training and test sets in an 8:2 ratio. The training set is used to train the cigarette authenticity recognition model, and the test set is used to verify the accuracy of the trained model. During training, a pre-trained deep residual network ResNet extracts the corresponding feature set [F0, F1, F2, ..., F15] from multiple sub-image datasets corresponding to each image patch. Then, the feature set [F0, F1, F2, ..., F15] is input into BiLSTM, and feature fusion is performed based on fine-grained features and the correlation between features, thereby performing authenticity recognition for each image patch. This is achieved by combining... Figure 4 As can be seen, the sub-image blocks P0, P1, P2, and P3 are adjacent and continuous in pairs. Therefore, their image features F0, F1, F2, and F3 are also correlated and can make a certain degree of joint decision-making effect on the authenticity of the entire image block. In order to learn the correlation between the sub-image features of the above image blocks and use them simultaneously for the authenticity decision of the entire image block, a recurrent neural network is used to perform feature fusion to obtain the correlation of these features, thereby improving the accuracy of recognition.

[0072] It is understandable that the feature extraction model (i.e., ResNet) and the feature fusion model (i.e., BiLSTM) are trained simultaneously. The test dataset is used to verify the learning effect of the model. The cigarette authenticity recognition model is repeatedly trained by setting the number of training rounds, learning rate, loss function, etc. The accuracy of the cigarette authenticity recognition model after repeated training is recalculated on the test dataset. When the accuracy is higher than the previous model, the current model is automatically saved until the number of iterations ends, and the optimal model is obtained.

[0073] In addition, the inventors of this application conducted relevant experiments to verify the training accuracy of the cigarette authenticity recognition model of this invention. Specifically, the scanned image of a pack of Furongwang cigarettes was divided into multiple 2048*2048 image blocks, and each 2048*2048 image block was further divided into 16 512*512 sub-image blocks, thereby constructing a sub-image block dataset for training. The specific training log for training the cigarette authenticity recognition model is as follows: Figure 6 As shown, when the model has been trained for 30 rounds, the accuracy on the training set has reached 99%, and the accuracy on the validation set has reached almost 100%.

[0074] Understandable, such as Figure 7 As shown, step S5 includes the following:

[0075] Step S51: Obtain a scanned image of the cigarette to be identified, and select the ROI region in the scanned image;

[0076] Step S52: Randomly crop out multiple non-overlapping image blocks from the ROI region, and perform gridding on each image block to obtain multiple sub-image blocks;

[0077] Step S53: Input the multiple sub-image blocks corresponding to each image block into the trained cigarette authenticity recognition model, and output the authenticity recognition result of each image block;

[0078] Step S54: Based on the authenticity recognition results of multiple image blocks, a voting method is used to obtain the final cigarette authenticity recognition result.

[0079] Specifically, after acquiring a single-sided or double-sided scanned image of the cigarette to be identified, since different brands and types of cigarette cartons and boxes have different characteristics, regions with obvious features are preferentially selected as Regions of Interest (ROIs) from the scanned image of a particular brand of cigarette cartons or boxes. The maximum selectable range of the ROI is the entire main area of ​​the cigarette carton or box, and the minimum range is the size of a single image block, such as 2048*2048. Then, N non-overlapping image blocks are randomly cropped from the ROI, where N is generally set between 1 and 10, and different values ​​can be set according to different brands of cigarette cartons or boxes. For example... Figure 8 As shown, 10 non-overlapping image blocks are randomly cropped from the ROI region of the Furongwang cigarette pack. Each image block is then divided into 16 sub-image blocks using a grid. After image enhancement for each sub-image block, the multiple sub-image blocks corresponding to each image block are input into a trained cigarette authenticity recognition model, outputting the authenticity recognition results for the 10 image blocks. Finally, the authenticity recognition results for the 10 image blocks are determined using a voting method, i.e., the minority obeys the majority, to obtain the final cigarette authenticity recognition result.

[0080] Understandably, the inventors of this application also conducted experimental verification, preparing a total of 26 scanned images of the front and back of genuine and counterfeit Furongwang cigarette boxes for model verification, including 16 images of counterfeit cigarettes and 10 images of genuine cigarettes. The identification results of the cigarette authenticity identification model of this invention are shown in Table 1:

[0081] Table 1. Recognition Results of the Cigarette Authenticity Identification Model

[0082] The prediction came true. Prediction is false total Real cigarettes 9(TP) 1(FN) 10 counterfeit cigarettes 1(FP) 15(TN) 16 total 10 16 26

[0083] Therefore, as can be seen from Table 1, the accuracy rate for identifying genuine cigarettes is 90%, the accuracy rate for identifying counterfeit cigarettes is 93.8%, and the overall accuracy rate is 92.3%, indicating that the model has a very high accuracy in identifying genuine and counterfeit cigarettes.

[0084] In addition, such as Figure 9 As shown, another embodiment of the present invention also provides a deep learning-based cigarette authenticity identification system, preferably employing the method described above. The system includes:

[0085] The sample data collection module is used to collect positive sample data and negative sample data and perform labeling processing. Positive sample data includes scanned images of real cigarettes, and negative sample data includes scanned images of fake cigarettes.

[0086] The dataset construction module is used to divide each scanned image into multiple image blocks using a sliding window, and to perform gridded segmentation on each image block to obtain multiple sub-image blocks, thereby constructing a sub-image block dataset;

[0087] The model building module is used to build a cigarette authenticity identification model, wherein the cigarette authenticity identification model includes a pre-trained convolutional neural network for extracting features and a recurrent neural network for identifying authenticity based on the extracted features.

[0088] The model training module is used to train the cigarette authenticity recognition model using the sub-image patch dataset until the model converges. During the training process, a pre-trained convolutional neural network is first used to extract the feature set from multiple sub-image patches corresponding to each image patch. Then, the extracted feature set is input into the recurrent neural network. The recurrent neural network identifies the authenticity of the image patch based on the correlation between the extracted features of multiple sub-image patches.

[0089] The authenticity recognition module is used to input the scanned image of the cigarette to be identified into the trained cigarette authenticity recognition model and output the authenticity recognition result.

[0090] As can be understood, the deep learning-based cigarette authenticity recognition system in this embodiment divides the sample image into multiple image blocks through a sliding window, and then divides each image block into multiple sub-image blocks using a grid. During model training, a pre-trained convolutional neural network is first used to extract fine-grained features, and then the extracted fine-grained features are input into a recurrent neural network. The recurrent neural network is used to mine the correlation between the extracted features corresponding to multiple sub-image blocks, thereby jointly deciding the authenticity of each image block based on the fine-grained features and the correlation between features. This greatly improves the accuracy of authenticity recognition for each image block and ensures that the trained model has high recognition accuracy, thus enabling fast and accurate identification of cigarette authenticity.

[0091] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0092] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0093] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0094] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0095] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0096] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0097] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for identifying genuine and counterfeit cigarettes based on deep learning, characterized in that, Includes the following: Step S1: Collect positive sample data and negative sample data and label them. Positive sample data includes scanned images of real cigarettes, and negative sample data includes scanned images of fake cigarettes. Step S2: Divide each scanned image into multiple image blocks using a sliding window, and then perform gridding on each image block to obtain multiple sub-image blocks, thereby constructing a sub-image block dataset; Step S3: Construct a cigarette authenticity identification model, wherein the cigarette authenticity identification model includes a pre-trained convolutional neural network for extracting features and a recurrent neural network for identifying authenticity based on the extracted features; Step S4: Train the cigarette authenticity recognition model using the sub-image patch dataset until the model converges. During the training process, first use a pre-trained convolutional neural network to extract a set of fine-grained features from multiple sub-image patches corresponding to each image patch. Then, input the extracted set of fine-grained features into a recurrent neural network. The recurrent neural network identifies the authenticity of the image patch based on the correlation between the fine-grained features of multiple sub-image patches. Step S5: Input the scanned image of the cigarette to be identified into the trained cigarette authenticity recognition model and output the cigarette authenticity recognition result; Step S2 includes the following: Step S21: Fill the boundary of the scanned image to an integer multiple of the sliding window size, and cut the filled scanned image from left to right and from top to bottom with a specific step size and a preset overlap rate to obtain multiple image blocks; Step S22: Perform grid-based segmentation on each image block to evenly divide each image block into multiple sub-image blocks, and construct the sub-image dataset corresponding to each image block in the order from left to right and from top to bottom. The several sub-image datasets corresponding to all sample data constitute the sub-image block dataset. The features between adjacent sub-image blocks are continuous. Step S5 includes the following: Step S51: Obtain a scanned image of the cigarette to be identified, and select the ROI region in the scanned image; Step S52: Randomly crop out multiple non-overlapping image blocks from the ROI region, and perform gridding on each image block to obtain multiple sub-image blocks; Step S53: Input the multiple sub-image blocks corresponding to each image block into the trained cigarette authenticity recognition model, and output the authenticity recognition result of each image block; Step S54: Based on the authenticity recognition results of multiple image blocks, a voting method is used to obtain the final cigarette authenticity recognition result.

2. The deep learning-based cigarette authenticity identification method as described in claim 1, characterized in that, Step S1 includes the following: Step S11: Scan the front and back images of genuine and counterfeit cigarettes using a scanner device, wherein the cigarettes are in the shape of a carton or a box. Step S12: Use the Canny edge operator to calculate the main outline information of the cigarette pack or box, and obtain the main area image of the cigarette pack or box to eliminate the background image of the scanner.

3. The deep learning-based cigarette authenticity identification method as described in claim 1, characterized in that, Step S2, between steps S21 and S22, also includes the following: Step S211: Remove featureless image blocks and low-feature image blocks.

4. The deep learning-based cigarette authenticity identification method as described in claim 3, characterized in that, Step S211 includes: converting all image blocks into grayscale images, calculating the total number of pixels in each image block, and removing an image block when the total number of pixels in a certain image block is lower than a preset threshold.

5. The deep learning-based cigarette authenticity identification method as described in claim 4, characterized in that, Step S211 further includes: performing binary classification training on the ResNet50 neural network model by constructing a training dataset containing feature image patches and low feature image patches, and using the trained ResNet50 neural network model to filter out low feature image patches. Alternatively, the image feature set can be extracted from the image patch using a pre-trained MobileNetV3-Small model, and then the K-Means clustering algorithm can be used to cluster the image patch into high-feature image patches and low-feature image patches, thereby completing the filtering of low-feature image patches.

6. A deep learning-based cigarette authenticity identification system, employing the deep learning-based cigarette authenticity identification method as described in any one of claims 1 to 5, characterized in that, include: The sample data collection module is used to collect positive sample data and negative sample data and perform labeling processing. Positive sample data includes scanned images of real cigarettes, and negative sample data includes scanned images of fake cigarettes. The dataset construction module is used to divide each scanned image into multiple image blocks using a sliding window, and to perform gridded segmentation on each image block to obtain multiple sub-image blocks, thereby constructing a sub-image block dataset; The model building module is used to build a cigarette authenticity identification model, wherein the cigarette authenticity identification model includes a pre-trained convolutional neural network for extracting features and a recurrent neural network for identifying authenticity based on the extracted features. The model training module is used to train the cigarette authenticity recognition model using the sub-image patch dataset until the model converges. During the training process, a pre-trained convolutional neural network is first used to extract the feature set from multiple sub-image patches corresponding to each image patch. Then, the extracted feature set is input into the recurrent neural network. The recurrent neural network identifies the authenticity of the image patch based on the correlation between the extracted features of multiple sub-image patches. The authenticity recognition module is used to input the scanned image of the cigarette to be identified into the trained cigarette authenticity recognition model and output the authenticity recognition result.

Images