A method for detecting missing cigarette packs based on deep learning

Through deep learning technology, combined with image acquisition and data enhancement, lightweight convolutional neural network is trained, which solves the problem of traditional detection equipment in identifying cigarette packets with high accuracy and error rate, and realizes the intelligence of cigarette packets with insufficient branches Accurate inspection improves productivity and competitiveness.

CN114092741BActive Publication Date: 2024-12-03THE 41ST INST OF CHINA ELECTRONICS TECH GRP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111381821.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-22
Publication Date
2024-12-03
Estimated Expiration
2041-11-22

AI Technical Summary

Technical Problem

Traditional visual inspection equipment has low accuracy in identifying the lack of cigarette packets on tobacco automated production lines, high error rate, frequent adjustments, uncalibrated potential defects cannot be identified, and it is difficult to identify cigarette packets with abnormal size or appearance. Human intervention factors Large, resulting in high maintenance costs and cannot meet the needs of intelligent manufacturing and lean production.

Method used

Using deep learning-based smoke packet lack of branch detection method, filter side images are collected through image acquisition components, and data enhancement is performed. The classification model is trained using Keras framework and DenseNet lightweight convolutional neural network to generate deep learning filter lack of branch defects. The detection model is used to determine the defective and defect levels of cigarette packets in real time, and release the results through the RESTful interface.

Benefits of technology

It realizes intelligent and precise detection of cigarette pack shortage, improves detection accuracy and robustness, reduces the cost of GPU hardware resources, expands the application scenarios of deep learning models, and is suitable for various cigarette packaging machines.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114092741B_ABST
    Figure CN114092741B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for detecting missing cigarette packs based on deep learning, which belongs to the technical field of detecting missing cigarette packs. The method completes the online intelligent recognition and classification of missing cigarette packs through a missing cigarette pack detection system, and specifically includes the following steps: collecting sample cigarette pack filter side images; using image processing technology to perform data enhancement on the obtained cigarette pack filter side images and expand sample data; training a filter side classification model based on the Keras framework and the DenseNet lightweight convolutional neural network according to the sample data; calling the filter side classification model through local and Web detection methods, collecting the filter side images of the cigarette packs to be detected in real time, and making a judgment on missing cigarette packs. The present invention adopts an online detection algorithm for missing cigarette packs based on a deep learning model, which has high detection accuracy and strong robustness compared to traditional image algorithms; the published RESTful interface can make full use of limited GPU hardware resources and expand the application scenarios of deep learning models.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of cigarette pack missing detection, and specifically relates to a cigarette pack missing detection method based on deep learning. Background Art

[0002] In the process of processing and producing cigarettes in cigarette factories, various reasons lead to various defective cigarettes being transported from the cigarette rolling machine to the cigarette packing line. According to relevant national standards, the lack of cigarettes in cigarette packs is a Class A serious quality defect, so it is very necessary to detect missing cigarette packs on the assembly line and remove unqualified cigarette packs.

[0003] On the automated tobacco production line, traditional visual inspection equipment still uses "threshold calibration" to identify defects. After more than a decade of use in the tobacco industry, this ancient identification method has become increasingly prominent, such as low recognition accuracy, high false rejection rate, frequent threshold adjustment, limited number of defect calibrations, inability to identify uncalibrated potential defects, difficulty in identifying cigarette packages with abnormal size or appearance, too many human intervention factors, etc. These shortcomings are obviously not conducive to the company's improvement of intelligent manufacturing and lean production models, and the subsequent maintenance costs are also very high. As cigarette companies continue to improve their indicators and requirements for product quality, product loss, production automation, intelligence, and informationization, traditional production equipment, production processes, testing equipment, and testing methods can no longer meet the growing actual needs.

[0004] Deep learning technology is becoming the mainstream research trend in the field of machine vision inspection. Applying the theoretical methods of deep learning to the defect detection algorithm of machine vision, combined with related issues in image defect detection and classification, using theories and methods based on deep learning to study defect detection algorithms, solve the shortcomings of current detection algorithms, and make industrial defect detection more accurate, efficient, intelligent, and low-cost, thereby improving product production efficiency and competitiveness. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a cigarette pack missing detection method based on deep learning to achieve intelligent and accurate detection of cigarette pack missing.

[0006] The technical solution of the present invention is as follows:

[0007] A cigarette pack missing detection method based on deep learning, which completes the online intelligent recognition and classification of cigarette pack missing status through a cigarette pack missing detection system;

[0008] The cigarette pack missing detection system comprises an image acquisition component, a host computer software, and a Web server;

[0009] The cigarette pack missing detection method specifically comprises the following steps:

[0010] S1. Collect the filter side image of the sample cigarette pack through the image acquisition component and transmit it to the host computer software;

[0011] S2. Using image processing technology, data enhancement is performed on the obtained cigarette pack filter side image to expand the sample data;

[0012] S3. Based on the sample data, the filter side classification model is trained based on the Keras framework and the DenseNet lightweight convolutional neural network to generate a deep learning filter missing defect detection model file;

[0013] S4. The host computer software loads the deep learning filter missing defect detection model file and the preset defect grade category file, and determines in real time whether the cigarette pack missing defect exists and the defect grade based on the filter side image of the cigarette pack to be detected collected by the image acquisition component;

[0014] S5. Synchronously publish the RESTful interface. The Web server calls the deep learning filter missing defect detection model to provide the existence and defect level of the cigarette pack missing defect in JSON data format for the filter side image of the cigarette pack transmitted through the HTTP protocol.

[0015] Preferably, in step S1, among the collected filter side images of sample cigarette packs, 50% of the cigarette packs have defects and 50% of the cigarette packs do not have defects.

[0016] Preferably, in step S2, a series of random transformations are used to perform data enhancement on the cigarette filter image; the random transformations used include horizontal translation, vertical translation, perspective transformation, scaling, channel conversion, and random horizontal flipping.

[0017] Preferably, in step S3, the specific process of model construction is: after creating the network, load the pre-trained weights, then flatten the convolution output, add a fully connected layer, Dropout regularization and a classification layer, and finally set the freezing layer and the training layer, compile the model using the Adam optimizer, and then carry out training, and continuously adjust the number of frozen network layers, learning rate, and decay rate parameters.

[0018] Preferably, step S4 completes the local online detection of the deep learning filter missing defect detection model, and the specific content is: first, the model format is converted, and the hdf5 model generated by Keras needs to be converted into a pb model supported by the host computer software program; then, the necessary deep learning dynamic link library is introduced into the host computer software program, the defect category file is read in, the converted pb model file is loaded, the input image information is converted into an input tensor, and finally the model input is set, and network reasoning is executed to determine in real time whether a cigarette pack missing defect exists and the defect level.

[0019] Preferably, step S5 completes the publishing of the deep learning filter missing defect detection model to a RESTful interface service, and the specific content is: the system Web server reads in the defect category file and the trained deep learning filter missing defect detection model file, and performs a missing filter judgment on the filter side image of the cigarette pack transmitted through the HTTP protocol, so that the user can transmit the image, and the Web server performs network reasoning, and gives the existence of the cigarette pack missing defect and the defect level result in the JSON data format.

[0020] Preferably, the front-end languages ​​accessed by the RESTful interface include android, ios, and html5, and the Web server is only responsible for processing data.

[0021] Preferably, the image acquisition component includes an LED lighting lamp, an optical plane mirror, and an optical lens; the optical lens is located on the upper oblique side of the optical plane mirror, the LED lighting lamp provides light source to illuminate the cigarette pack, and the optical lens collects image information on the filter side of the cigarette pack after reflection from the optical plane mirror.

[0022] Beneficial technical effects brought by the present invention:

[0023] Data enhancement is performed through image processing technology to expand sample data for model training, solving the problem of overfitting of models trained with limited or too few production line sample sets. An online missing cigarette pack detection algorithm based on a deep learning model is used, which has high detection accuracy and strong robustness compared to traditional image algorithms. The published RESTful interface can make full use of limited GPU hardware resources while expanding the application scenarios of deep learning models. The missing cigarette pack detection method based on deep learning can develop missing cigarette detectors for various cigarette packaging machines, which has a wide range of applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of the image acquisition component of the present invention;

[0025] Figure 2 This is a flow chart of the deep learning model building process of the present invention;

[0026] Figure 3 This is a curve chart of the training accuracy of the deep learning model of the present invention;

[0027] Figure 4 This is a graph of the loss function for training the deep learning model of the present invention;

[0028] Figure 5 A diagram showing the steps of applying the local online detection of the deep learning model of the present invention;

[0029] Figure 6 A schematic diagram of publishing a RESTful interface service for the deep learning model of the present invention;

[0030] Among them, 1-LED lighting; 2-optical plane mirror; 3-low distortion optical lens; 4-cigarette package to be inspected; 5-industrial camera. DETAILED DESCRIPTION

[0031] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0032] The present invention proposes a cigarette pack missing detection method based on deep learning, which completes the online intelligent recognition and classification of the cigarette pack missing status through a cigarette pack missing detection system. The cigarette pack missing detection system mainly includes an image acquisition component, a host computer software, and a Web server.

[0033] Among them, Figure 1 As shown, the industrial camera 5 is equipped with an LED lighting lamp 1 and a low-distortion optical lens 3. An optical plane mirror 2 is placed obliquely below the right side of the low-distortion optical lens 3. The angle between the low-distortion optical lens 3 and the optical plane mirror 2 can be adjusted according to actual conditions to reflect the complete image of the cigarette pack filter side. The industrial camera 5 and the optical plane mirror 2 together form an image acquisition component. The LED lighting lamp 1 provides a light source for illumination, which illuminates the cigarette pack 4 to be inspected, so that the low-distortion optical lens 3 can collect the image information of the cigarette pack filter side after being reflected by the optical plane mirror 2.

[0034] The process of online intelligent identification and classification of missing cigarette packs is as follows:

[0035] Step 1: After the image acquisition component receives the trigger signal, the low-distortion optical lens collects the image information of the cigarette pack filter to be inspected after being reflected by the plane mirror and transmits it to the host computer software. A total of 3,000 sample data were collected and 3,000 sample data sets were constructed, of which 50% were defective and 50% were non-defective images.

[0036] Step 2: Use image processing technology to enhance the collected sample data and expand the sample data to train the cigarette filter classification model. For example, expand a sample image in the data set to fifteen through random transformation, and use 80% of each data set as a training set and 20% as a validation set. The random transformation processing methods used include horizontal translation, vertical translation, perspective transformation, scaling, channel conversion, random horizontal flipping, etc. By expanding the sample data, it is possible to prevent the trained model from overfitting, that is, the model's generalization ability is insufficient. By improving the data through a series of random transformations, there will be no two identical images in the sample input of the deep learning model.

[0037] Step 3: Based on the Keras framework and DenseNet lightweight neural network, after loading the training set and validation set, the feature extractor at the bottom of the DenseNet network is used. It is composed of a series of convolutional layers and pooling layers. The model is small in size and fast in speed. Then, feature extraction and classifier design are performed on the sample data set on the filter side through transfer learning. The specific process of model construction is as follows: Figure 2 As shown in the figure, after creating the network, load the pre-trained weights, flatten the convolution output, add the fully connected layer, Dropout regularization and classification layer, and finally set the freezing layer and training layer. Use the Adam optimizer to compile the model, and then conduct training, constantly adjust the number of frozen network layers, learning rate, decay rate and other parameters to improve the accuracy of the neural network model. After the training is completed, the training accuracy and loss function curve of the data set are generated. Figure 3 As shown in the figure, the horizontal axis represents the number of model training iterations, and the vertical axis represents the accuracy rate; with the iteration of model training, the accuracy rate approaches 100%. Figure 4 As shown in the figure, the horizontal axis represents the number of model training iterations, and the vertical axis represents the loss value; with the iteration of model training, the loss converges close to 0.

[0038] Step 4: Figure 5 As shown in the figure, the local online detection application steps of the deep learning model. First, the model format is converted. The hdf5 model generated by Keras needs to be converted into a pb model supported by the host computer software program; then the necessary deep learning dynamic link library is introduced into the host computer software program, the defect category file is read in, the converted pb model file is loaded, the input image information is converted into an input tensor, and finally the model input is set, the network inference is executed, and the existence and defect level of the cigarette pack missing defect are judged in real time.

[0039] Step 5: Figure 6 As shown in the figure, it is a schematic diagram of the deep learning model publishing RESTful interface service. The system Web server reads the defect category file and the trained deep learning model file, and makes a missing judgment on the filter side of the cigarette pack passed in through the HTTP protocol. When the user passes in the picture, the Web server performs network reasoning and gives the existence of the missing cigarette pack defect and the defect level result in JSON data format. By publishing the RESTFUL interface, it has the advantage of platform independence. The server is only responsible for processing data, and the front end can be in any language (android, ios, html5).

[0040] Through the above steps 4 and 5, local and Web detection methods are realized, achieving the purpose of expanding the application scenarios of deep learning models. Among them, in the Web online detection service, the Web server runs independently and provides a RESTful interface. It can perform online detection on images sent via HTTP. It has the advantage of platform independence. The Web server is only responsible for processing data, and the front end can be any language (android, ios, html5), that is, it can be connected to the PC, mobile phone (including Android and IOS), and Pad. The real-time performance is slightly lower than the local real-time detection.

[0041] In addition, the content of the defect category file directly reflects the defect level, including: qualified, general defects (half a stick missing, one stick missing), and serious defects (more than one stick missing).

[0042] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions or substitutions made by technicians in this technical field within the essential scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A method for detecting missing cigarette packs based on deep learning, characterized in that: The online intelligent identification and classification of missing cigarette packs can be completed through the missing cigarette pack detection system; The cigarette pack missing detection system comprises an image acquisition component, a host computer software, and a Web server; The cigarette pack missing detection method specifically comprises the following steps: S1. Collect the filter side image of the sample cigarette pack through the image acquisition component and transmit it to the host computer software; S2. Using image processing technology, data enhancement is performed on the obtained cigarette pack filter side image to expand the sample data; S3. Based on the sample data, the filter side classification model is trained based on the Keras framework and the DenseNet lightweight convolutional neural network to generate a deep learning filter missing defect detection model file; S4. The host computer software loads the deep learning filter missing defect detection model file and the preset defect grade category file, and determines in real time whether the cigarette pack missing defect exists and the defect grade based on the filter side image of the cigarette pack to be detected collected by the image acquisition component; S5. Synchronously publish the RESTful interface. The Web server calls the deep learning filter missing defect detection model to provide the existence and defect level of the cigarette pack missing defect in JSON data format for the filter side image of the cigarette pack transmitted through the HTTP protocol.

2. The method for detecting missing cigarette packs based on deep learning according to claim 1, characterized in that: In the step S1, among the collected filter side images of the sample cigarette packs, 50% of the cigarette packs have defects and 50% of the cigarette packs do not have defects.

3. The method for detecting missing cigarette packs based on deep learning according to claim 1, characterized in that: In the step S2, a series of random transformations are used to perform data enhancement on the cigarette filter image; the random transformations used include horizontal translation, vertical translation, perspective transformation, scaling, channel conversion, and random horizontal flipping.

4. The method for detecting missing cigarette packs based on deep learning according to claim 1, characterized in that: In step S3, the specific process of model construction is: after creating the network, load the pre-trained weights, then flatten the convolution output, add the fully connected layer, Dropout regularization and classification layer, and finally set the freezing layer and training layer, compile the model using the Adam optimizer, and then carry out training, and continuously adjust the number of frozen network layers, learning rate, and decay rate parameters.

5. The method for detecting missing cigarette packs based on deep learning according to claim 1, characterized in that: The step S4 completes the local online detection of the deep learning filter missing defect detection model, and the specific content is: first, the model format is converted, and the hdf5 model generated by Keras needs to be converted into a pb model supported by the host computer software program; then, the necessary deep learning dynamic link library is introduced into the host computer software program, the defect category file is read in, the converted pb model file is loaded, the input image information is converted into an input tensor, and finally the model input is set, and network reasoning is executed to determine in real time whether a cigarette pack missing defect exists and the defect level.

6. The method for detecting missing cigarette packs based on deep learning according to claim 1, characterized in that: The step S5 completes the release of the deep learning filter missing defect detection model to a RESTful interface service. Specifically, the system Web server reads in the defect category file and the trained deep learning filter missing defect detection model file, and performs a missing filter judgment on the filter side image of the cigarette pack transmitted via the HTTP protocol. The user transmits the image, and the Web server performs network reasoning, and gives the existence of a cigarette pack missing defect and the defect level result in the JSON data format.

7. The method for detecting missing cigarette packs based on deep learning according to claim 1, characterized in that: The front-end languages ​​accessed by the RESTful interface include android, ios, and html5, and the Web server is only responsible for processing data.

8. The method for detecting missing cigarette packs based on deep learning according to claim 1, characterized in that: The image acquisition component includes an LED lighting lamp, an optical plane mirror, and an optical lens; the optical lens is located on the upper oblique side of the optical plane mirror, the LED lighting lamp provides light source to illuminate the cigarette pack, and the optical lens collects image information on the filter side of the cigarette pack after reflection from the optical plane mirror.