A cigarette box defect detection method, device, equipment and medium

Abstract

The present application relates to the field of cigarette box defect detection, and in particular to a cigarette box defect detection method, device, equipment and medium, the method comprising: obtaining a cigarette box image to be detected in real time; extracting image features of the cigarette box image to be detected, and matching the image features with preset universal defect features to obtain a first image category of the cigarette box image to be detected; if the first image category is a universal defect category, then determining that the cigarette box corresponding to the cigarette box image to be detected is a universal defect cigarette box; if the first image category is other categories, then inputting the cigarette box image to be detected into a pre-trained neural network model for second category recognition to obtain a second image category; if the second image category is any major defect category, then determining that the cigarette box corresponding to the cigarette box image to be detected is a major defect category cigarette box; if the second image category is a normal category, then determining that the cigarette box corresponding to the cigarette box image to be detected is a normal cigarette box. The present application can improve the efficiency of cigarette box defect detection.

Description

Technical Field

[0001] The present application relates to the field of cigarette box defect detection, and in particular to a cigarette box defect detection method, device, equipment and medium. Background Art

[0002] In the tobacco industry, cigarettes generally have three manufacturing processes: shredding, rolling and packaging. The packaging process is to pack the rolled cigarettes into cigarette boxes, and assemble the cigarette boxes into cigarette rods. Since the products formed by the packaging process are directly facing consumers, the quality of the packaging directly affects the shopping experience of consumers, so the quality inspection of the packaging process is particularly important.

[0003] In the related art, when performing quality inspection on the packaging process, a deep learning neural network model is generally used to detect all types of defects on the cigarette box image to determine whether the cigarette box packaging has defects. In the actual cigarette box packaging process, a large number of cigarette boxes need to be quality inspected to determine whether there are cigarette box defects. However, when the neural network model performs cigarette box defect detection, it is necessary to perform multiple convolutions on the cigarette box features to obtain the cigarette box features, and the detection efficiency is low.

[0004] Therefore, how to provide a solution to the above technical problems is a problem that those skilled in the art need to solve at present. Summary of the invention

[0005] In order to improve the efficiency of cigarette box defect detection, the present application provides a cigarette box defect detection method, device, equipment and medium.

[0006] In a first aspect, the present application provides a cigarette box defect detection method, which adopts the following technical solution:

[0007] Acquire the cigarette box image to be detected in real time;

[0008] Extracting image features of the cigarette box image to be detected, and matching the image features with preset universal defect features to obtain a first image category of the cigarette box image to be detected;

[0009] If the first image category is a common defect category, determining that the cigarette box corresponding to the to-be-detected cigarette box image is a common defect cigarette box;

[0010] If the first image category is other categories, inputting the cigarette box image to be detected into a pre-trained neural network model for second category recognition to obtain a second image category;

[0011] If the second image category is any defect category, then the cigarette box corresponding to the to-be-detected cigarette box image is determined to be a defect category cigarette box, and the defect categories include: wrinkles, peeling, glue spitting, corner overlap, missing, small box wrinkles, crushed, and irregular shape;

[0012] If the second image category is a normal category, it is determined that the cigarette box corresponding to the to-be-detected cigarette box image is a normal cigarette box.

[0013] By adopting the above technical solution, after acquiring the image of the cigarette box to be detected in real time, the image features of the cigarette box image to be detected are extracted, and the image features are matched with the preset universal defect features to perform rapid screening of the universal defect features to obtain the first image category of the cigarette box image to be detected. If the first image category is the universal defect category, the cigarette box image to be detected is marked as the universal defect cigarette box image. If the first image is other categories, the cigarette box image to be detected is input into the neural network model for second image category recognition to determine whether the cigarette box corresponding to the cigarette box image to be detected is a defective major category cigarette box corresponding to any major defect category or a normal cigarette box. It can be seen that the image features of the cigarette box image to be detected are first matched with the preset universal defect features to determine whether it is a universal defect category cigarette box with obvious features, which occupies low computer resources and has a faster response. When the first image category is other categories, the neural network model can be used to perform second category recognition on the cigarette box image to be detected to obtain the second image category, thereby reducing the number of cigarette box images to be detected input into the neural network model and improving the detection efficiency.

[0014] In a possible implementation, after acquiring the cigarette box image to be detected in real time, the method further includes:

[0015] Preprocessing the cigarette box image to be detected, wherein the preprocessing method includes: positioning correction and / or noise reduction processing;

[0016] Accordingly, the step of extracting the image features of the cigarette box image to be detected includes:

[0017] Extract the image features of the preprocessed cigarette box image to be detected.

[0018] By adopting the above-mentioned technical scheme, positioning correction and / or noise reduction processing is performed on the cigarette box image to be detected, which can remove some interference information of the cigarette box image to be detected, retain the target shape, size and specific features in the cigarette box image to be detected, and make the cigarette box image to be detected clearer and more accurate. The noise-reduced or corrected cigarette box image to be detected is then used for image detection, which can improve the effectiveness and accuracy of the detection result.

[0019] In a possible implementation, the extracting the image features of the cigarette box image to be detected includes:

[0020] Extracting first feature data of the cigarette box image to be detected by using a feature pyramid structure; extracting second feature data of the cigarette box image to be detected by using SIFT (Scale-invariant feature transform);

[0021] The first feature data and the second feature data are fused to obtain fused data, and the fused data is screened using ROI (region of interest) to obtain the image feature.

[0022] By adopting the above technical solution, the feature pyramid structure and SIFT are used to extract and fuse the features of the cigarette box image with detection. The fused features are screened using ROI to obtain image features. Compared with related technologies, there is no need to repeatedly convolute the image to extract it, which improves the speed of extracting image features.

[0023] In a possible implementation, the training process of the neural network model includes:

[0024] Acquire a first training sample, wherein the first training sample includes a plurality of cigarette box image samples and respective corresponding category information labels, wherein the category information labels include a normal category label and category labels of each major defect category;

[0025] Obtain the neural network model to be trained;

[0026] The neural network model to be trained is trained based on the first training sample to obtain the neural network model.

[0027] By adopting the above technical solution, a training method for a neural network model is provided. Compared with the related technology, the neural network model to be trained can be trained based on the first training sample, and finally a neural network model that meets the requirements is obtained, which is used to identify the normal category of cigarette boxes or the category information of each defect category.

[0028] In one possible implementation, it also includes:

[0029] When the cigarette box corresponding to the cigarette box image to be detected is a common defective cigarette box or a defective major category cigarette box, the cigarette box information corresponding to the cigarette box image to be detected is sent to the console, so that the console removes the corresponding cigarette box according to the cigarette box information corresponding to the cigarette box image to be detected.

[0030] By adopting the above technical solution, when the general defect categories of cigarette boxes are detected using the general detection model, or when the major defect categories of cigarette boxes are detected using the neural network model, the cigarette box information corresponding to the cigarette box image to be detected is sent to the control console. The control console determines the position of the cigarette box according to the cigarette box information corresponding to the cigarette box image to be detected, and removes the corresponding cigarette box, thereby realizing automated operation and improving the production efficiency of cigarette boxes.

[0031] In a possible implementation, after determining that the cigarette box corresponding to the to-be-detected cigarette box image is a defective cigarette box, the method further includes:

[0032] Input the cigarette box image to be inspected into a pre-trained defect information recognition neural network model to perform cigarette box defect information recognition, and obtain a cigarette box defect sub-category image including defect information, wherein the defect information includes the specific defect location, defect size, and defect magnitude of the defect major category cigarette box;

[0033] The defect level of the defective cigarette box of the major category is determined according to the defect information. If the defect level is the target defect level, the cigarette box corresponding to the to-be-detected cigarette box image is determined to be a normal cigarette box.

[0034] By adopting the above technical solution, after detecting the specific major defect category, the major defect category image is input into the corresponding defect information recognition neural network model to detect the minor defect category, and the specific defect level is determined according to the detected minor defect category. If the defect level is the target defect level, it is determined that the cigarette box corresponding to the cigarette box image to be detected meets the factory standards and can be used normally, thereby improving the recall rate of defect detection.

[0035] In a possible implementation, the defect information recognition neural network model is a fusion neural network model. Accordingly, the training process of the fusion neural network model includes:

[0036] Acquire a second training sample, wherein the second training sample includes multiple defect sub-category images and respective corresponding defect location information labels;

[0037] Obtaining a fusion neural network model to be trained, wherein the fusion neural network model to be trained includes: an initial first neural network model, an initial second neural network model and an initial data fusion module, wherein the initial first neural network model and the initial second neural network model are used to calculate the input defect subclass image to obtain the first matrix sequence and the second matrix sequence corresponding to each other, and the initial data fusion module is used to fuse the first matrix sequence and the second matrix sequence;

[0038] The fused neural network model to be trained is trained based on the second training sample to obtain the fused neural network model.

[0039] By adopting the above-mentioned technical scheme, the second training sample is input into the fusion neural network model to be trained for training to obtain a fusion neural network model, and the fusion neural network model to be trained adopted by the present technical scheme includes: an initial first neural network model, an initial second neural network model and a fusion module. The obtained matrix series is richer, and the learned data is more comprehensive. The obtained fusion neural network model can effectively reduce the missed detection rate of small categories of cigarette box defects, and ensure the accuracy of detection of small categories of cigarette box defects.

[0040] In a second aspect, the present application provides a cigarette box defect detection device, which adopts the following technical solution:

[0041] A cigarette box defect detection device, comprising:

[0042] Image acquisition module: used to acquire the image of the cigarette box to be detected in real time;

[0043] A first image category acquisition module: used for extracting image features of the cigarette box image to be detected, and matching the image features with preset universal defect features to obtain a first image category of the cigarette box image to be detected;

[0044] A first determination module: for determining that the cigarette box corresponding to the cigarette box image to be detected is a common defect cigarette box if the first image category is a common defect category;

[0045] A second image category acquisition module: used for inputting the cigarette box image to be detected into a pre-trained neural network model for second category recognition to obtain a second image category if the first image category is other categories;

[0046] The second determination module is used to determine that the cigarette box corresponding to the cigarette box image to be detected is a defective cigarette box if the second image category is any defect category. The defect categories include: wrinkles, peeling, glue spitting, corner overlap, missing, small box wrinkling, crushing, and irregular shape;

[0047] The third determination module is configured to determine that the cigarette box corresponding to the cigarette box image to be detected is a normal cigarette box if the second image category is a normal category.

[0048] By adopting the above technical scheme, after acquiring the cigarette box image to be detected in real time, the image features of the cigarette box image to be detected are extracted, and the image features are matched with the preset universal defect features to perform rapid screening of the universal defect features to obtain the first image category of the cigarette box image to be detected. If the first image category is the universal defect category, the cigarette box image to be detected is marked as the universal defect cigarette box image. If the first image is other categories, the cigarette box image to be detected is input into the neural network model for second image category recognition to determine whether the cigarette box corresponding to the cigarette box image to be detected is a defective major category cigarette box corresponding to any major defect category or a normal cigarette box. It can be seen that the technical scheme first matches the extracted image features of the cigarette box image to be detected with the preset universal defect features to determine whether it is a universal defect category cigarette box with obvious features. Only when it is other categories can the neural network model be used to determine the second image category, which can reduce the number of cigarette box images to be detected input into the neural network model and improve detection efficiency.

[0049] In a third aspect, the present application provides an electronic device, which adopts the following technical solution:

[0050] An electronic device, comprising:

[0051] at least one processor;

[0052] Memory;

[0053] At least one application, wherein the at least one application is stored in a memory and configured to be executed by at least one processor, and the at least one application is configured to: execute the above-mentioned cigarette box defect detection method.

[0054] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution:

[0055] A computer-readable storage medium includes: a computer program that can be loaded by a processor and execute the above-mentioned cigarette box defect detection method.

[0056] In summary, the present application includes at least one of the following beneficial technical effects:

[0057] 1. After acquiring the image of the cigarette box to be detected in real time, extract the image features of the cigarette box image to be detected, and match the image features with the preset universal defect features to quickly screen the universal defect features to obtain the first image category of the cigarette box image to be detected. If the first image category is the universal defect category, mark the cigarette box image to be detected as the universal defect cigarette box image. If the first image is other categories, input the cigarette box image to be detected into the neural network model for second image category recognition to determine whether the cigarette box corresponding to the cigarette box image to be detected is a defective major category cigarette box corresponding to any defect major category or a normal cigarette box. It can be seen that first matching the extracted image features of the cigarette box image to be detected with the preset universal defect features to determine whether it is a universal defect category cigarette box with obvious features occupies low computer resources and has a fast response. When the first image category is other categories, the neural network model can be used to perform second category recognition on the cigarette box image to be detected to obtain the second image category, thereby reducing the number of cigarette box images to be detected input into the neural network model and improving detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a flow chart of a cigarette box defect detection method provided in an embodiment of the present application;

[0059] Figure 2 It is a structural schematic diagram of a cigarette box defect detection device provided in an embodiment of the present application;

[0060] Figure 3 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] The following is combined with Figure 1 -Attached Figure 3 This application is described in further detail.

[0062] After reading this specification, those skilled in the art may make modifications to this embodiment without creative contribution as needed, but they are protected by patent law as long as they are within the scope of the embodiments of this application.

[0063] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0064] In addition, the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally means that the associated objects before and after are in an "or" relationship.

[0065] In the related art, computer vision is generally used to detect defects in cigarette box images when performing quality inspection on packaging processes. With respect to the above-mentioned related art, the inventors found that since the neural network model is used in computer vision technology, multiple convolutions are required when extracting the defect features of cigarette boxes, resulting in slow detection speed and low detection efficiency.

[0066] In order to solve the above technical problems, an embodiment of the present application provides a cigarette box defect detection method, which first matches the extracted image features of the cigarette box image to be detected with the preset general defect features to determine whether it is a general defect category cigarette box with obvious features. It occupies low computer resources and has a fast response. When the first image category is other categories, the neural network model can be used to perform second category recognition on the cigarette box image to be detected to obtain the second image category, thereby reducing the number of cigarette box images to be detected input into the neural network model and improving detection efficiency.

[0067] Specifically, the embodiment of the present application provides a cigarette box defect detection method, which is performed by an electronic device, which can be a server or a terminal device, wherein the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal device can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., but is not limited thereto. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiment of the present application.

[0068] Combination Figure 1 , Figure 1 : is a flow chart of a cigarette box defect detection method provided by an embodiment of the present application, wherein the method comprises steps S100, S101, S102, S103, S104, and S105, wherein:

[0069] Step S100, acquiring the image of the cigarette box to be detected in real time.

[0070] After receiving the detection request, the cigarette box defect detection process is executed. Specifically, a monitoring program is pre-integrated in the electronic device, and the monitoring program is used to monitor the triggering behavior of the detection request. Once the detection request is monitored to be triggered, the cigarette box defect detection process is executed. Specifically, when the user confirms the detection, a detection instruction is automatically generated. The method of confirming the detection may include the user confirming the detection by clicking a detection button on the application, and the user confirming the detection by voice. When the electronic device detects that the user triggers the detection request, the electronic device executes the cigarette box defect detection process.

[0071] Specifically, the image defect detection method provided in the embodiment of the present application can be applied to a cigarette box production line, and one or more camera devices for collecting cigarette box images are pre-set in the area to be detected of the cigarette box production line to obtain the cigarette box image to be detected collected by the camera device in real time. The camera device can be a CCD (Charge-coupled Device) camera to obtain a high-resolution cigarette box image to be detected, and the area to be detected can be an area selected by the user according to actual needs, or can be set by the user based on experience.

[0072] Step S101 , extracting image features of the cigarette box image to be detected, and matching the image features with preset universal defect features to obtain a first image category of the cigarette box image to be detected.

[0073] Among them, after the image features are extracted, the image features are matched with preset universal defect features to determine whether the category of the cigarette box to be detected is a universal defect category or other categories. Specifically, as long as it is matched that the cigarette box has any one or more of the seal stains, damages, and abrasions, the first image category of the cigarette box image to be detected is a universal defect category; otherwise, the first image category of the cigarette box image to be detected is other categories.

[0074] Furthermore, the method may include: acquiring multiple cigarette box images with common defects, extracting standard features of the multiple cigarette box images with common defects, using the multiple standard features as preset common defect features, and periodically updating the preset common defect features.

[0075] Step S102: If the first image category is a common defect category, it is determined that the cigarette box corresponding to the cigarette box image to be detected is a common defect cigarette box.

[0076] Specifically, when the first image category is a general defect category, it indicates that the cigarette box corresponding to the cigarette box image to be detected is a general defect cigarette box, and the cigarette box quality inspection fails, and no further inspection is required. It can be understood that in the conventional cigarette box quality inspection process, common defects are general defect cigarette boxes. This embodiment first uses the general defect detection model to perform a preliminary screening of the cigarette box images to be inspected, which can eliminate cigarette box images with general defects and reduce the number of cigarette box images to be inspected that are input into the neural network model. In the case of a large number of cigarette box defect inspections, the inspection speed can be improved.

[0077] Step S103: if the first image category is other categories, the cigarette box image to be detected is input into a pre-trained neural network model for second category recognition to obtain a second image category.

[0078] Among them, the neural network model is obtained after training based on multiple cigarette box images and their corresponding first category information labels, wherein the first category information labels include normal category information and major defect category information, wherein the major defect category information includes at least: any one of the following information: wrinkles, peeling, glue spitting, overlapped corners, missing, small box wrinkles, crushed, and irregular shape.

[0079] In the embodiment of the present application, the neural network model can be a VGG (visual geometry group, super-resolution test sequence) neural network model. The VGG neural network model includes a convolution layer, a pooling layer and a fully connected layer. When detecting the cigarette box image to be detected, the cigarette box image to be detected is input into the VGG neural network. Specifically, the feature vector is extracted through the convolution layer; the feature vector is input into the pooling layer for feature calculation to obtain the calculation result; the calculation result is input into the fully connected layer for classification and image regression to output the second image category. It can be understood that when the neural network model performs the second category recognition, the categories that can be recognized include: normal category and various major defect categories.

[0080] Step S104, if the second image category is any defect category, then determine that the cigarette box corresponding to the cigarette box image to be detected is a defect category cigarette box, and the defect categories include: wrinkles, peeling, glue spitting, overlapped corners, missing, small box wrinkles, crushed, and irregular shape.

[0081] Step S105: if the second image category is a normal category, then determining that the cigarette box corresponding to the cigarette box image to be detected is a normal cigarette box.

[0082] Specifically, when the second image category is a normal category, the cigarette box corresponding to the cigarette box image to be inspected is determined to be a normal cigarette box and passes the cigarette box quality inspection; when the second image category is any one of the defective categories, the cigarette box corresponding to the cigarette box image to be inspected is determined to be a defective cigarette box, that is, the cigarette box quality inspection fails.

[0083] Based on the above embodiment, after acquiring the cigarette box image to be detected in real time, the image features of the cigarette box image to be detected are extracted, and the image features are matched with the preset universal defect features to perform rapid screening of the universal defect features to obtain the first image category of the cigarette box image to be detected. If the first image category is the universal defect category, the cigarette box image to be detected is marked as the universal defect cigarette box image. If the first image is other categories, the cigarette box image to be detected is input into the neural network model for second image category recognition to determine whether the cigarette box corresponding to the cigarette box image to be detected is a defective major category cigarette box corresponding to any major defect category or a normal cigarette box. It can be seen that first matching the extracted image features of the cigarette box image to be detected with the preset universal defect features to determine whether it is a universal defect category cigarette box with obvious features occupies low computer resources and has a faster response. Only when the first image category is other categories can the neural network model be used to perform second category recognition on the cigarette box image to be detected to obtain the second image category, thereby reducing the number of cigarette box images to be detected input into the neural network model and improving detection efficiency.

[0084] Furthermore, in the embodiment of the present application, after acquiring the image of the cigarette box to be detected in real time, the method further includes:

[0085] Preprocessing the cigarette box image to be detected, the preprocessing method includes: positioning correction and / or noise reduction processing;

[0086] Accordingly, extracting the image features of the cigarette box image to be detected includes: extracting the image features of the preprocessed cigarette box image to be detected, including:

[0087] The preprocessed multiple cigarette box images to be inspected are input into the universal defect detection model for first category recognition to obtain the first image categories corresponding to each of them.

[0088] Specifically, since the cigarette box will produce slight translation and jitter when it is transmitted on the production line, the cigarette box image to be detected is positioned and corrected to eliminate the translation and jitter of the cigarette box image to be detected. The specific positioning and correction steps may include: pre-saving the complete cigarette box image, establishing coordinates and determining correction parameters based on the same features existing in the pre-saved complete cigarette box image and the cigarette box image to be detected, and positioning and correcting the cigarette box image to be detected according to the correction parameters. Since it is easily polluted by various noises during imaging, it may disturb the observable information of the cigarette box image to be detected. Therefore, the cigarette box image to be detected is subjected to noise reduction processing to eliminate interference information. Specifically, a low-pass filter algorithm can be used for rapid noise reduction, which can remove part of the interference information of the cigarette box image to be detected, retain the target shape, size and specific features in the cigarette box image to be detected, so that the cigarette box image to be detected is clearer and more accurate, and then the noise-reduced or corrected cigarette box image to be detected is subjected to image detection, which can improve the effectiveness and accuracy of the detection result.

[0089] Further, in a possible case, extracting image features of the cigarette box image to be detected includes: extracting first feature data of the cigarette box image to be detected using a feature pyramid structure; extracting second feature data of the cigarette box image to be detected using SIFT;

[0090] The first feature data and the second feature data are fused to obtain fused data, and the fused data is screened using ROI to obtain image features.

[0091] Specifically, the first feature data obtained by the feature pyramid structure is to extract the features of each scale of the cigarette box image to be detected. SIFT, i.e., Scale-invariant feature transform (SIFT), is a description used in the field of image processing. This description has scale invariance and can detect key points, i.e., the second feature data, in the cigarette box image to be detected.

[0092] The fused data obtained by fusing the first feature data and the second feature data has the characteristics of different scales and continuity. The fused data is then feature screened using ROI (region of interest) to remove abnormal parts, and finally image features are obtained.

[0093] Specifically, the feature pyramid structure and SIFT are used to extract and fuse the features of the cigarette box image with detection. The fused features are screened using ROI to obtain image features. Compared with related technologies, there is no need to perform repeated convolution extraction on the image, which improves the speed of extracting image features.

[0094] Furthermore, in another possible case, extracting image features of the cigarette box image to be detected includes: extracting image features of the cigarette box image to be detected using a HOG (Histogram of Oriented Gradient) algorithm or a LBP (Local Binary Pattern) algorithm.

[0095] Furthermore, the embodiment of the present application provides a training method for a neural network model, which may include step S10 (not shown in the drawings), step S20 (not shown in the drawings), and step S30 (not shown in the drawings), wherein:

[0096] S10, obtaining a first training sample, wherein the first training sample includes a plurality of cigarette box image samples and their corresponding category information labels, and the category information labels include a normal category label and category labels of each major defect category.

[0097] Specifically, a variety of cigarette box image samples are obtained by shooting with a web crawler and / or a production line camera device, and the category information label can be based on a manual method or an automatic machine method. For example, an image annotation tool can be used to annotate the category information of cigarette box defects in the cigarette box image sample to obtain the cigarette box category information label corresponding to each cigarette box image sample, and the category information labels correspond to each defect category label and normal category label. Among them, each defect category label includes wrinkles, peeling, spitting glue, overlapping corners, missing, small box wrinkles, crushing, irregular shape, and normal category labels.

[0098] Step S20, obtaining the neural network model to be trained.

[0099] The neural network model to be trained may be a VGG neural network model, which may include: a convolutional layer, a pooling layer, and a fully connected layer.

[0100] Step S30: training the neural network model to be trained based on the first training sample to obtain a neural network model.

[0101] Specifically, each cigarette box image sample is input into the neural network model to be trained to obtain a first category to be compared; based on the first category to be compared and the corresponding category information label, a first loss value is determined; according to the first loss value, it is determined whether the current neural network model to be trained has converged; if so, the current neural network model to be trained is determined as the neural network model; if not, the weight parameters are updated by back propagation using the first loss value, and the neural network model to be trained after the weight parameters are updated is iteratively trained until the current neural network model to be trained converges, so as to determine the converged neural network model to be trained as the neural network model.

[0102] In summary, the embodiment of the present application provides a training method for a neural network model. Compared with the related art, the embodiment of the present application can train the neural network model to be trained based on the first training sample, and finally obtain a neural network model that meets the requirements, which is used to identify the normal category of cigarette boxes or the category information of each major defect category.

[0103] Furthermore, in the embodiment of the present application, in order to realize automated operation and improve the production efficiency of cigarette boxes, it also includes:

[0104] When the cigarette box corresponding to the cigarette box image to be detected is a common defective cigarette box or a defective major category cigarette box, the cigarette box information corresponding to the cigarette box image to be detected is sent to the console, so that the console removes the corresponding cigarette box according to the cigarette box information corresponding to the cigarette box image to be detected.

[0105] The cigarette box information corresponding to the cigarette box image to be detected may be the corresponding cigarette box number information and / or the position information of the cigarette box in the image. Specifically, after the general defect category of the cigarette box is detected using the general detection model, or the major category of the cigarette box defect is detected using the neural network model, the cigarette box information corresponding to the cigarette box image to be detected is sent to the control console, and the control console determines the cigarette box position according to the cigarette box information corresponding to the cigarette box image to be detected, and controls the rejection tool to reject the corresponding cigarette box, thereby realizing automated operation and improving the production efficiency of cigarette boxes.

[0106] Furthermore, after determining that the cigarette box corresponding to the cigarette box image to be inspected is a defective cigarette box, the method further includes:

[0107] Inputting the cigarette box image to be inspected into a pre-trained defect information recognition neural network model to perform cigarette box defect information recognition, and obtaining a cigarette box defect sub-category image including defect information, wherein the defect information includes the specific defect location, defect size, and defect magnitude of the defect major category cigarette box;

[0108] The defect level of the defective cigarette box is determined according to the defect information. If the defect level is the target defect level, the cigarette box corresponding to the to-be-detected cigarette box image is determined to be a normal cigarette box.

[0109] Among them, the defect information recognition neural network model is used to identify the defect information of the cigarette box to obtain a cigarette box defect subcategory image including the defect information. The embodiment of the present application does not limit the defect information recognition neural network model. The user can set it according to actual needs, as long as it can achieve the purpose of this embodiment. For example, the defect information recognition neural network model can be a fusion neural network model, which can specifically include: a trained first neural network model, a second neural network model and a data fusion module, wherein the data fusion module is connected to the first neural network model and the second neural network model respectively, and the cigarette box image to be detected is used as the input information of the first neural network model and the second neural network model. Specifically, the first neural network model and the second neural network model can be Fast RCNN (Fast Region-Convolutional Neural Networks) neural network model and Faster RCNN (Faster Region-Convolutional Neural Networks) neural network model, respectively. Specifically, the cigarette box image to be inspected is input into the defect information recognition neural network model, and the feature extraction and pooling of the suggested boxes of the first neural network model and the second neural network model are performed in turn to obtain the first matrix series and the second matrix series corresponding to each other; the first matrix series and the second matrix series are fused by using the data fusion module; the fused data matrix is ​​regressed by bounding box; a few boxes are selected from the matrix series after the box regression by using the non-maximum suppression method; the cigarette box image to be inspected is annotated according to the obtained few boxes, and mapped back to the cigarette box image to be inspected to obtain the cigarette box defect sub-category image including defect information. It can be understood that the defect sub-category results detected by using the FastRCNN neural network model, the Faster RCNN neural network model and the data fusion module are more accurate than those detected by using the Fast RCNN neural network model or the Faster RCNN neural network model, and can improve the recall rate of defect detection.

[0110] For another example, the defect information recognition neural network model is Fast RCNN or Faster RCNN. Specifically, the suggestion box features of FastRCNN or Faster RCNN are extracted and pooled to obtain matrix data, and then the fully connected layer is used for recognition, and finally the cigarette box defect sub-category image including the defect information is output.

[0111] It is understandable that there are certain production standards according to the production defect level, and defects are divided into different levels of defects, such as X, A, B, C, and D level defects. Generally speaking, defects such as X or A can be tolerated and can be considered normal. Therefore, in the embodiment of the present application, after obtaining the defect information, the defect position, defect size, and defect size of the defect information are matched with the correspondence between the pre-stored defect information and the defect level to obtain the defect level corresponding to the defect information. When the defect level is the target level, it is determined that the defect of the cigarette box is a tolerable defect, and the cigarette box is regarded as a normal cigarette box.

[0112] The corresponding relationship between defect information and defect level can be measured and stored in advance by the user according to actual conditions, or set based on experience, and the target level can also be selected by the user according to actual needs, as long as the purpose of this embodiment can be achieved.

[0113] Specifically, after detecting the specific major defect categories, the major defect category images are input into the corresponding defect information recognition neural network model to detect minor defect categories. The specific defect grade is determined based on the detected minor defect categories and the industrial production defect grade assessment standards. The cigarette boxes corresponding to the minor defect categories that meet the requirements are used normally, thereby improving the recall rate of defect detection.

[0114] Furthermore, the defect information recognition neural network model is a fusion neural network model. Accordingly, the training process of the fusion neural network model may include step S40 (not shown in the drawings), step S50 (not shown in the drawings), and step S60 (not shown in the drawings), wherein:

[0115] Step S40, obtaining a second training sample, wherein the second training sample includes multiple defect sub-category images and their corresponding defect position information labels.

[0116] Specifically, numerous images of cigarette box defect subcategories are acquired through web crawlers and / or production line cameras, and information labeling can be based on manual methods or automatic machine methods. For example, image labeling tools can be used to label the cigarette box defect subcategory information in the cigarette box images to obtain the defect location information label corresponding to each cigarette box sample image.

[0117] Step S50, obtaining a fusion neural network model to be trained, wherein the fusion neural network model to be trained includes: an initial first neural network model, an initial second neural network model and an initial data fusion module, the initial first neural network model and the initial second neural network model are used to calculate the input defect subclass image to obtain the first matrix sequence and the second matrix sequence corresponding to each other, and the initial data fusion module is used to fuse the first matrix sequence and the second matrix sequence;

[0118] Step S60: training the fused neural network model to be trained based on the second training sample to obtain a fused neural network model.

[0119] The initial first neural network model and the initial second neural network model are used to calculate the input image to obtain the corresponding first training matrix sequence and second training matrix sequence, and the initial data fusion module is used to fuse the first training matrix sequence and the second training matrix sequence.

[0120] Specifically, the initial first neural network model and the initial second neural network model can be a Fast RCNN neural network model and a Faster RCNN neural network model, respectively. The Fast RCNN neural network model includes a convolution layer and a pooling layer.

[0121] When the initial first neural network model is Fast RCNN, the second training sample is input into Fast RCNN for training of cigarette box defect information, specifically including:

[0122] Use selective search to select the suggestion box; input the cigarette box image to be detected into the convolution layer to obtain a feature map; use the suggestion box to extract the feature box from the feature map. Specifically, using the suggestion box to extract the feature box from the feature map can specifically be to find the position corresponding to each suggestion box from the feature map, and cut out the feature box, at this time the depth remains unchanged. It can be understood that the suggestion boxes selected by RCNN have many overlapping parts, and the convolution is seriously repeated. The embodiment of the present application uses Fast RCNN to perform a convolution on each position of the cigarette box to be detected, which reduces the number of convolutions and the amount of calculation.

[0123] The extracted feature frames are input into the ROI pooling layer to convert the proposal frames of different sizes into feature frames of the same size, and the pooled feature frames are converted into the first training matrix series. Specifically, each feature frame is divided into H×W grids, and pooling is performed in each grid to obtain the maximum value in each grid to finally obtain the first training matrix series. The difference between the Faster RCNN neural network model and the Fast RCNN neural network model is that the method of selecting the proposal frame is different. Faster RCNN uses RPN (Region Proposal Network). RPN takes the image feature map as input and generates a series of object proposals.

[0124] When the initial second neural network model is Faster RCNN, the second training sample is input into the initial second neural network model. Specifically, the feature of the second training sample is extracted through the convolution layer of the initial second neural network model to obtain a feature map; the feature map of the second training sample is used as input by RPN to generate a series of suggestion boxes; the suggestion boxes are mapped to the last convolution feature map of the convolution layer, and the extracted feature boxes are input into the ROI pooling layer to convert suggestion boxes of different sizes into feature boxes of the same size, and the feature boxes after pooling are converted into the second training matrix series.

[0125] The first training matrix series and the second training matrix series obtained by the Fast RCNN neural network model and the Faster RCNN neural network model in the pooling layer are fused and sorted using a fusion module. Finally, the fused matrix series are regressed using a bounding box. A small number of boxes are selected from the matrix series after the box regression using a non-maximum suppression method. The second training samples are labeled according to the obtained small number of boxes, and finally mapped back to their respective original images to obtain small category images of cigarette box defects including defect information.

[0126] Then, based on the cigarette box defect sub-category image including defect information and the corresponding defect location information label, determine the second loss value; determine whether the current fusion neural network detection model to be trained converges according to the second loss value, if so, determine the current fusion neural network detection model to be trained as the fusion neural network detection model; if not, iterate the training until the current fusion neural network detection model to be trained converges. Specifically, the algorithm for the second loss value can be any one of: MSE: Mean Square Error, RMSE: Root Mean Square Error, MAE: Mean Absolute Error, MAPE: Mean Absolute Percentage Error.

[0127] Specifically, the defect information recognition neural network model adopted in the embodiment of the present application is to input the second training sample into the initial first neural network model and the initial second neural network model respectively, fuse the first matrix series and the second matrix series obtained by the ROI pooling layer respectively, and input the fused matrix data into the fully connected layer to obtain specific defect subcategories. The use of the fusion model can effectively reduce the missed detection rate of cigarette box defects and improve the detection accuracy of defect subcategories.

[0128] In another possible embodiment, the defect information recognition neural network model is a Fast RCNN neural network model.

[0129] Specifically, selective search is used to select suggestion boxes; the cigarette box image to be detected is input into the convolution layer to obtain a feature map; the suggestion box is used to extract the feature box from the feature map, and the extracted feature box is input into the ROI pooling layer to convert suggestion boxes of different sizes into feature boxes of the same size. The feature boxes after pooling are converted into the first training matrix series, and the first matrix series is classified and calculated to obtain the first matrix series after classification calculation, which is finally mapped back to the defect subcategory image to determine the defect subcategory.

[0130] In another possible embodiment, the defect information recognition neural network model is a Faster RCNN neural network model.

[0131] Specifically, RPN (Region Proposal Network) is used to select RPN proposal boxes; the cigarette box image to be detected is input into the convolution layer to obtain a feature map; the feature box is extracted from the feature map using the RPN proposal box, and the extracted feature box is input into the ROI pooling layer to convert proposal boxes of different sizes into feature boxes of the same size, and the feature boxes after pooling are converted into a second training matrix series, and the second matrix series is classified and calculated to obtain a second matrix series after classification calculation, and finally mapped back to the defect subcategory image to determine the defect subcategory.

[0132] A cigarette box defect detection device provided in an embodiment of the present application is introduced below. The cigarette box defect detection device described below and the cigarette box defect detection method described above can refer to each other. The cigarette box defect detection device in the embodiment of the present application is arranged in an electronic device.

[0133] Please refer to Figure 2 , Figure 2 : is a schematic diagram of the structure of a cigarette box defect detection device provided in an embodiment of the present application, comprising:

[0134] Image acquisition module 210: used to acquire the image of the cigarette box to be detected in real time;

[0135] The first image category acquisition module 220 is used to extract the image features of the cigarette box image to be detected, and match the image features with the preset universal defect features to obtain the first image category of the cigarette box image to be detected;

[0136] The first determination module 230 is used to determine that the cigarette box corresponding to the cigarette box image to be detected is a common defect cigarette box if the first image category is a common defect category;

[0137] The second image category acquisition module 240 is used to input the cigarette box image to be detected into a pre-trained neural network model for second category recognition to obtain the second image category if the first image category is other categories;

[0138] The second determination module 250 is used to determine that the cigarette box corresponding to the cigarette box image to be detected is a defective cigarette box if the second image category is any defect category. The defect categories include: wrinkles, peeling, glue spitting, corner overlap, missing, small box wrinkling, crushing, and irregular shape;

[0139] The third determination module 260 is configured to determine that the cigarette box corresponding to the cigarette box image to be detected is a normal cigarette box if the second image category is a normal category.

[0140] A possible implementation of the embodiment of the present application also includes:

[0141] Image preprocessing module: used to preprocess the cigarette box image to be detected, and the preprocessing method includes: positioning correction and / or noise reduction processing;

[0142] Accordingly, when the first image category acquisition module 220 extracts the image features of the cigarette box image to be detected, it is specifically used to:

[0143] Extract the image features of the preprocessed cigarette box image to be detected.

[0144] In a possible implementation of the embodiment of the present application, when the first image category acquisition module 220 extracts image features of the cigarette box image to be detected, it is specifically used to:

[0145] The first feature data of the cigarette box image to be detected is extracted by using the feature pyramid structure; the second feature data of the cigarette box image to be detected is extracted by using SIFT;

[0146] The first feature data and the second feature data are fused to obtain fused data, and the fused data is screened using ROI to obtain image features.

[0147] A possible implementation of the embodiment of the present application further includes: a neural network model training module, which is used to:

[0148] Obtaining a first training sample, wherein the first training sample includes a plurality of cigarette box image samples and respective corresponding category information labels, wherein the category information labels include a normal category label and category labels of each major defect category;

[0149] Obtain the neural network model to be trained;

[0150] The neural network model to be trained is trained based on the first training sample to obtain a neural network model.

[0151] A possible implementation of the embodiment of the present application also includes:

[0152] Defective cigarette box processing module: when the cigarette box corresponding to the cigarette box image to be detected is a common defective cigarette box or a defective large category cigarette box, the cigarette box information corresponding to the cigarette box image to be detected is sent to the console, so that the console can reject the corresponding cigarette box according to the cigarette box information corresponding to the cigarette box image to be detected.

[0153] A possible implementation of the embodiment of the present application also includes:

[0154] Cigarette box defect sub-category image acquisition module: used to input the cigarette box image to be detected into the pre-trained defect information recognition neural network model to perform cigarette box defect information recognition, and obtain the cigarette box defect sub-category image including defect information, wherein the defect information includes the specific defect location, defect size, and defect size of the defect major category cigarette box;

[0155] The fourth determination module is used to determine the defect level of the defective cigarette box according to the defect information. If the defect level is the target defect level, the cigarette box corresponding to the cigarette box image to be detected is determined to be a normal cigarette box.

[0156] A possible implementation of the embodiment of the present application further includes: a defect information recognition neural network model training module, which is used to:

[0157] Acquire a second training sample, wherein the second training sample includes multiple defect sub-category images and respective corresponding defect position information labels;

[0158] Obtaining a fusion neural network model to be trained, wherein the fusion neural network model to be trained includes: an initial first neural network model, an initial second neural network model and an initial data fusion module, the initial first neural network model and the initial second neural network model are used to calculate the input defect subclass image to obtain the first matrix sequence and the second matrix sequence corresponding to each other, and the initial data fusion module is used to fuse the first matrix sequence and the second matrix sequence;

[0159] The fused neural network model to be trained is trained based on the second training sample to obtain a fused neural network model.

[0160] An electronic device provided in an embodiment of the present application is introduced below. The electronic device described below and the cigarette box defect detection method described above can be referenced to each other.

[0161] An electronic device is provided in an embodiment of the present application, such as Figure 3 As shown, Figure 3 is a structural diagram of an electronic device provided in an embodiment of the present application, Figure 3The electronic device 300 shown includes: a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, such as through a bus 302. Optionally, the electronic device 300 may also include a transceiver 304. It should be noted that in actual applications, the transceiver 304 is not limited to one, and the structure of the electronic device 300 does not constitute a limitation on the embodiments of the present application.

[0162] The processor 301 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of the embodiments of the present application. The processor 301 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0163] The bus 302 may include a path to transmit information between the above components. The bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 302 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0164] The memory 303 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0165] The memory 303 is used to store application code for executing the solution of the embodiment of the present application, and the execution is controlled by the processor 301. The processor 301 is used to execute the application code stored in the memory 303 to implement the content shown in the above method embodiment.

[0166] Among them, electronic devices include but are not limited to: mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0167] A cigarette box defect detection system provided in an embodiment of the present application is introduced below. The cigarette box defect detection system described below and the cigarette box defect detection method described above can be referenced to each other.

[0168] The present application embodiment provides a system for detecting cigarette box defects, including:

[0169] A camera device, used for collecting images of cigarette boxes to be inspected;

[0170] An electronic device comprises a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and execute the above-mentioned box defect detection method.

[0171] Among them, the processor can realize the reading of the cigarette box image to be detected, the image preprocessing of the general defect detection model using the general defect detection model, and the image recognition of the neural network model using the neural network model.

[0172] In another possible implementation of the embodiment of the present application, the system for detecting cigarette box defects further includes:

[0173] Optical lens: used to capture the image of the cigarette box to be tested and form an image;

[0174] Light source: used to improve the imaging effect and stability of optical lenses;

[0175] Strobe: used to improve the accuracy of taking images of cigarette boxes to be tested;

[0176] Conveying device: used for transporting cigarette boxes;

[0177] Rejection device: used to reject abnormal cigarette boxes detected by the cigarette box defect detection method.

[0178] In order to ensure timely response of the camera device and the light source, the embodiment of the present application may also include a high-precision optical fiber position sensor. When a cigarette box passes by, the sensor triggers a signal to start the camera device and the light source to capture the image signal.

[0179] A computer-readable storage medium provided in an embodiment of the present application is introduced below. The computer-readable storage medium described below and the method described above can be referenced to each other.

[0180] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above cigarette box defect detection method are implemented. Compared with the related art, in the embodiment of the present application, after acquiring the cigarette box image to be detected in real time, the image features of the cigarette box image to be detected are extracted, and the image features are matched with preset universal defect features to perform rapid screening of the universal defect features to obtain the first image category of the cigarette box image to be detected. If the first image category is the universal defect category, the cigarette box image to be detected is marked as the universal defect cigarette box image. If the first image is other categories, the cigarette box image to be detected is input into the neural network model for second image category recognition to determine whether the cigarette box corresponding to the cigarette box image to be detected is a defective major category cigarette box corresponding to any major defect category or a normal cigarette box. It can be seen that the image features of the cigarette box image to be detected after extraction are first matched with the preset universal defect features to determine whether it is a universal defect category cigarette box with obvious features, which occupies low computer resources and has a faster response. Only when the first image category is other categories can the neural network model be used to perform second category recognition on the cigarette box image to be detected to obtain the second image category, thereby reducing the number of cigarette box images to be detected input into the neural network model and improving detection efficiency.

[0181] Since the embodiments of the computer-readable storage medium part and the embodiments of the method part correspond to each other, the embodiments of the computer-readable storage medium part refer to the description of the embodiments of the method part.

[0182] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0183] The above are only some implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A cigarette box defect detection method, characterized in that: include: Acquire the cigarette box image to be detected in real time; Extracting image features of the cigarette box image to be detected, and matching the image features with preset universal defect features to obtain a first image category of the cigarette box image to be detected; If the first image category is a common defect category, determining that the cigarette box corresponding to the to-be-detected cigarette box image is a common defect cigarette box; If the first image category is other categories, inputting the cigarette box image to be detected into a pre-trained neural network model for second category recognition to obtain a second image category; If the second image category is any defect category, then the cigarette box corresponding to the to-be-detected cigarette box image is determined to be a defect category cigarette box, and the defect categories include: wrinkles, peeling, glue spitting, corner overlap, missing, small box wrinkles, crushed, and irregular shape; If the second image category is a normal category, determining that the cigarette box corresponding to the cigarette box image to be detected is a normal cigarette box; After determining that the cigarette box corresponding to the cigarette box image to be inspected is a defective cigarette box, the method further includes: Input the cigarette box image to be inspected into a pre-trained defect information recognition neural network model to perform cigarette box defect information recognition, and obtain a cigarette box defect sub-category image including defect information, wherein the defect information includes the specific defect location, defect size, and defect magnitude of the defect major category cigarette box; Determining the defect level of the defective cigarette box according to the defect information, and if the defect level is the target defect level, determining that the cigarette box corresponding to the to-be-detected cigarette box image is a normal cigarette box; The defect information recognition neural network model is a fusion neural network model. Accordingly, the training process of the fusion neural network model includes: Acquire a second training sample, wherein the second training sample includes multiple defect sub-category images and respective corresponding defect location information labels; Obtaining a fusion neural network model to be trained, wherein the fusion neural network model to be trained includes: an initial first neural network model, an initial second neural network model and an initial data fusion module, wherein the initial first neural network model and the initial second neural network model are used to calculate the input defect subclass image to obtain the first matrix sequence and the second matrix sequence corresponding to each other, and the initial data fusion module is used to fuse the first matrix sequence and the second matrix sequence; The fused neural network model to be trained is trained based on the second training sample to obtain the fused neural network model.

2. The cigarette box defect detection method according to claim 1, characterized in that: After the real-time acquisition of the cigarette box image to be detected, the method further includes: Preprocessing the cigarette box image to be detected, wherein the preprocessing method includes: positioning correction and / or noise reduction processing; Accordingly, the step of extracting the image features of the cigarette box image to be detected includes: Extract the image features of the preprocessed cigarette box image to be detected.

3. The cigarette box defect detection method according to claim 1, characterized in that: The step of extracting image features of the cigarette box image to be detected includes: Extracting first feature data of the cigarette box image to be detected by using a feature pyramid structure; extracting second feature data of the cigarette box image to be detected by using SIFT; The first feature data and the second feature data are fused to obtain fused data, and the fused data is screened using ROI to obtain the image feature.

4. The cigarette box defect detection method according to claim 1, characterized in that: The training process of the neural network model includes: Acquire a first training sample, wherein the first training sample includes a plurality of cigarette box image samples and respective corresponding category information labels, wherein the category information labels include a normal category label and category labels of each major defect category; Obtain the neural network model to be trained; The neural network model to be trained is trained based on the first training sample to obtain the neural network model.

5. The cigarette box defect detection method according to claim 1, characterized in that: Also includes: When the cigarette box corresponding to the cigarette box image to be detected is a common defective cigarette box or a defective major category cigarette box, the cigarette box information corresponding to the cigarette box image to be detected is sent to the console, so that the console removes the corresponding cigarette box according to the cigarette box information corresponding to the cigarette box image to be detected.

6. A device for detecting cigarette box defects, characterized in that: The device comprises: Image acquisition module: used to acquire the image of the cigarette box to be detected in real time; A first image category acquisition module: used for extracting image features of the cigarette box image to be detected, and matching the image features with preset universal defect features to obtain a first image category of the cigarette box image to be detected; A first determination module: for determining that the cigarette box corresponding to the cigarette box image to be detected is a common defect cigarette box if the first image category is a common defect category; A second image category acquisition module: used for inputting the cigarette box image to be detected into a pre-trained neural network model for second category recognition to obtain a second image category if the first image category is other categories; The second determination module is used to determine that the cigarette box corresponding to the cigarette box image to be detected is a defective cigarette box if the second image category is any defect category. The defect categories include: wrinkles, peeling, glue spitting, corner overlap, missing, small box wrinkling, crushing, and irregular shape; A third determination module: configured to determine that the cigarette box corresponding to the cigarette box image to be detected is a normal cigarette box if the second image category is a normal category; Wherein, the device for detecting cigarette box defects further comprises: Cigarette box defect sub-category image acquisition module: used to input the cigarette box image to be detected into a pre-trained defect information recognition neural network model to perform cigarette box defect information recognition, and obtain a cigarette box defect sub-category image including defect information, wherein the defect information includes the specific defect location, defect size, and defect size of the defect major category cigarette box; a fourth determination module, configured to determine the defect level of the defective cigarette box according to the defect information, and if the defect level is a target defect level, determine that the cigarette box corresponding to the to-be-detected cigarette box image is a normal cigarette box; The device for detecting cigarette box defects further comprises: a defect information recognition neural network model training module, which is used to: Acquire a second training sample, wherein the second training sample includes multiple defect sub-category images and respective corresponding defect location information labels; Obtaining a fusion neural network model to be trained, wherein the fusion neural network model to be trained includes: an initial first neural network model, an initial second neural network model and an initial data fusion module, wherein the initial first neural network model and the initial second neural network model are used to calculate the input defect subclass image to obtain the first matrix sequence and the second matrix sequence corresponding to each other, and the initial data fusion module is used to fuse the first matrix sequence and the second matrix sequence; The fused neural network model to be trained is trained based on the second training sample to obtain the fused neural network model.

7. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executes any one of the methods of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: A computer program is stored which can be loaded by a processor and execute the method according to any one of claims 1 to 5.

Images