Cigarette defect number character screening method based on image similarity

Abstract

The invention discloses a cigarette defect number character screening method based on image similarity, which comprises the following steps of: acquiring cigarette defect pictures from a rolling production line visual inspection terminal, and calculating similarity measurement based on correlation coefficients through detection of a deep learning model so as to judge whether the current defect pictures can be added into a defect number character library or not; therefore, typical digital samples meeting production requirements are screened out from massive defect pictures and added into the digital sample library, a large amount of labor is saved, and the digital samples can be used for replacing entity samples in production to be used for checking whether equipment is normal or not. According to the invention, the production efficiency is improved, the production process of cigarette detection is improved, and the stability of cigarette quality is ensured.

Description

Technical Field

[0001] The invention belongs to the field of machine vision, and in particular to a method for screening digital samples of cigarette defects based on image similarity. Background Art

[0002] In the inspection of cigarette appearance defects, in order to test whether the appearance inspection equipment is operating normally, it is necessary to input physical samples of defective cigarettes. These physical samples are made of specific defects on cigarettes based on the experience of the operator. Then the equipment is inspected. If it can be detected smoothly, it is considered that the equipment is operating normally, otherwise the equipment needs to be repaired and debugged. However, in actual production, cigarette defects are of various types and in different forms. Operators cannot produce all of them on physical samples. This requires a new technical means to make full use of the defective images detected in production and generate a defective digital sample library to guide production. However, in actual production, the equipment side generally generates a large number of defective images, and the classification of these images does not meet the tobacco standards. Therefore, a method is needed to automatically filter out defective digital samples that meet the requirements from a large number of defective images and save these samples in the defective digital sample library. In actual production, these defective digital samples are transmitted to the appearance inspection equipment. If the inspection equipment can correctly detect these defective digital samples, it means that the equipment is operating normally, otherwise, the equipment needs to be repaired. Summary of the invention

[0003] The present invention aims to solve the deficiencies of the above-mentioned prior art and proposes a method for screening digital samples of cigarette defects based on image similarity, in order to screen out typical digital samples that meet production requirements from a large number of defective images generated by the equipment end and add them to the digital sample library to replace the physical samples, so as to check whether the equipment is normal, thereby improving production efficiency, improving the production process of cigarette detection, and ensuring the stability of cigarette quality.

[0004] In order to achieve the above-mentioned purpose, the present invention adopts the following technical scheme:

[0005] The invention discloses a method for screening digital samples of cigarette defects based on image similarity, which comprises the following steps:

[0006] Step 1: Collect the cigarette defect image S on the visual inspection terminal and input it into the defect detection model for processing, obtain the category j and confidence p of the defect in the cigarette defect image S, and the position coordinates of the defect detection box in S are (x min , y min , x max , y max ), where x min and min are the horizontal and vertical coordinates of the upper left corner of the defect detection frame, respectively.max and max They are the horizontal and vertical coordinates of the lower right corner of the defect detection frame respectively;

[0007] Step 2: Segment the defect detection frame from S according to the position coordinates of the defect detection frame, thereby obtaining a cigarette defect area map R;

[0008] Step 3: Use a bilinear interpolation algorithm to adjust the dimension of R to obtain a normalized cigarette defect area image R' with a dimension of W×H, where W and H are the width and height of R' respectively;

[0009] Step 4: Convert R' into a cigarette defect grayscale image G;

[0010] Step 5. Get the feature matrix F in G with a width of W-1 and a height of H respectively:

[0011] Compare the pixel value G(m,n) in the mth row and nth column of G with the pixel value G(m,n+1) in the mth row and n+1th column of G. If G(m,n)< G(m,n+1), set the eigenvalue F(m,n) in the mth row and nth column of the feature matrix to 0. Otherwise, set F(m,n) to 1, where 1≤m≤H and 1≤n≤W-1.

[0012] Step 6: If the number of digital samples with defect category j in the defect digital sample library is 0, then after setting the initial value of the storage probability prob, a digital sample is formed together with S, G and F and added to the defect digital sample library with defect category j, and the process ends;

[0013] If the number of digital sample libraries with defect category j in the defect digital sample library is greater than 0, execute step 7;

[0014] Step 7: Use formula (1) to calculate the average eigenvalue Fmean(m,n) of the mth row and nth column in the average feature matrix Fmean of all digital samples with defect category j in the defect digital sample library:

[0015] (1)

[0016] In formula (1), K is the number of all digital samples with defect category j in the defect digital sample library; The eigenvalue of the mth row and nth column in the characteristic matrix F of the kth digital sample;

[0017] Step 8: Calculate the mean of the feature matrix F in a digital sample using equations (2) and (3) and the mean of the average feature matrix Fmean :

[0018] (2)

[0019] (3)

[0020] Step 9: Calculate the correlation coefficient corr between F and Fmean using formula (4):

[0021] (4)

[0022] Step 10: Use formula (5) to normalize corr to (0,1) to obtain the normalized correlation coefficient :

[0023] (5)

[0024] Step 11: Use formula (6) to normalize K to (0,1), so as to obtain the normalized number of all digital samples with defect category j. :

[0025] (6)

[0026] In formula (6), K_MAX is the maximum number of digital samples of defect categories in the digital sample library;

[0027] Step 12: Use formula (7) to calculate the storage probability prob of the digital sample with defect category j:

[0028] (7)

[0029] In formula (7), w1, w2, w3 are three weights, and w1+w2+w3=1;

[0030] Step 13: If prob>th_p, then prob, S, G and F are combined into a digital sample and added to the defect digital sample library of defect category j, and K+1 is assigned to K. Otherwise, it is not added to the library and the process ends, where th_p is the threshold for digital sample addition;

[0031] Step 14: If K>K_MAX, delete the digital sample with the smallest probability of being put into storage among the digital samples with defect category j; otherwise, do not delete it and end the process.

[0032] An electronic device of the present invention includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the method for screening digital samples of cigarette defects, and the processor is configured to execute the program stored in the memory.

[0033] The present invention provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program executes the steps of the method for screening digital samples of cigarette defects when the computer program is run by a processor.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] 1. The present invention adopts a characteristic matrix to describe the defect area, and the result is stable and reliable.

[0036] 2. The present invention uses correlation coefficient to calculate the similarity between the defect image and other defect images in the digital sample library. The result is highly accurate and can eliminate noise interference.

[0037] 3. The present invention adopts weighted processing on three items: confidence, number of digital samples, and similarity, and comprehensively considers multiple factors to screen digital samples, so that the results are more reliable. DETAILED DESCRIPTION

[0038] In this embodiment, a method for screening digital samples of cigarette defects based on image similarity includes the following steps:

[0039] Step 1: Collect the cigarette defect image S on the visual inspection terminal and input it into the deep learning-based defect detection model for processing, obtain the category j and confidence p of the defect in the cigarette defect image S, and the position coordinates of the defect detection box in S are (x min , y min , x max , y max ), where x min and min are the horizontal and vertical coordinates of the upper left corner of the defect detection frame, respectively. max and max They are the horizontal and vertical coordinates of the lower right corner of the defect detection box. The deep learning defect detection model is obtained through offline training, which requires manual annotation of the defect areas of a large number of defect images and the completion of model training. In the process of annotating data, the number of defect samples of different categories needs to be balanced.

[0040] Step 2: Segment the defect detection frame from S according to the position coordinates of the defect detection frame, thereby obtaining the cigarette defect area map R; in this way, only the defect area map R needs to be processed in subsequent processing.

[0041] Step 3: Use a bilinear interpolation algorithm to adjust the dimension of R to obtain a normalized cigarette defect area image R' with a dimension of W×H, where W and H are the width and height of R' respectively;

[0042] Step 4: Convert R' into a cigarette defect grayscale image G;

[0043] Step 5. Get the feature matrix F in G with a width of W-1 and a height of H respectively:

[0044] Compare the pixel value G(m,n) of the mth row and nth column in G with the pixel value G(m,n+1) of the mth row and n+1th column in G. If G(m,n)< G(m,n+1), let the eigenvalue F(m,n) of the mth row and nth column in the feature matrix = 0. Otherwise, let F(m,n) = 1, where 1≤m≤H, 1≤n≤W-1. The feature matrix extracted in this way can effectively describe the texture features of the defect area and provide feature data for the subsequent similarity measurement.

[0045] Step 6: If the number of digital samples with defect category j in the defect digital sample library is 0, then after setting the initial value of the storage probability prob, a digital sample is formed together with S, G and F and added to the defect digital sample library with defect category j, and the process ends;

[0046] If the number of digital sample libraries with defect category j in the defect digital sample library is greater than 0, execute step seven.

[0047] Step 7: Use formula (1) to calculate the average eigenvalue Fmean(m,n) of the mth row and nth column in the average feature matrix Fmean of all digital samples with defect category j in the defect digital sample library:

[0048] (1)

[0049] In formula (1), K is the number of all digital samples with defect category j in the defect digital sample library; Represents the eigenvalue of the mth row and nth column in the characteristic matrix F of the kth digital sample.

[0050] Step 8: Calculate the mean of the feature matrix F in a digital sample using equations (2) and (3) and the mean of the average feature matrix Fmean :

[0051] (2)

[0052] (3)

[0053] Step 9: Calculate the correlation coefficient corr between F and Fmean using formula (4):

[0054] (4)

[0055] Step 10: Use formula (5) to normalize corr to (0,1) to obtain the normalized correlation coefficient :

[0056] (5)

[0057] Step 11: Use formula (6) to normalize K to (0,1), so as to obtain the normalized number of all digital samples with defect category j. :

[0058] (6)

[0059] In formula (6), K_MAX is the maximum number of digital samples of defect categories in the digital sample library.

[0060] Step 12: Use formula (7) to calculate the storage probability prob of the digital sample with defect category j:

[0061] (7)

[0062] In formula (7), w1, w2, and w3 are three weights, and w1+w2+w3=1. Here, the number of existing samples in the digital sample library, the confidence of the new sample, and the similarity between the new sample and the existing samples of the same category are considered at the same time to ensure that the digital samples stored in the library are more representative.

[0063] Step 13: If prob>th_p, then prob, S, G and F are combined into a digital sample and added to the defect digital sample library of defect category j, and K+1 is assigned to K. Otherwise, it is not added to the library and the process ends, where th_p is the threshold for digital sample addition;

[0064] Step 14: If K>K_MAX, delete the digital sample with the lowest probability of being stored in the digital sample of defect category j, otherwise, do not delete it and end the process. This step ensures that the digital sample library samples will not increase infinitely, and also has the effect of balancing samples of different categories.

[0065] In this embodiment, an electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0066] In this embodiment, a computer-readable storage medium stores a computer program on the computer-readable storage medium, and the computer program executes the steps of the above method when executed by a processor.

Claims

1. A method for screening digital samples of cigarette defects based on image similarity, characterized in that: The steps include: Step 1: Collect the cigarette defect image S on the visual inspection terminal and input it into the defect detection model for processing, obtain the category j and confidence p of the defect in the cigarette defect image S, and the position coordinates of the defect detection box in S are (x min , y min , x max , y max ), where x min and min are the horizontal and vertical coordinates of the upper left corner of the defect detection frame, respectively. max and max They are the horizontal and vertical coordinates of the lower right corner of the defect detection frame respectively; Step 2: Segment the defect detection frame from S according to the position coordinates of the defect detection frame, thereby obtaining a cigarette defect area map R; Step 3: Use a bilinear interpolation algorithm to adjust the dimension of R to obtain a normalized cigarette defect area image R' with a dimension of W×H, where W and H are the width and height of R' respectively; Step 4: Convert R' into a cigarette defect grayscale image G; Step 5. Get the feature matrix F in G with a width of W-1 and a height of H respectively: Compare the pixel value G(m,n) in the mth row and nth column of G with the pixel value G(m,n+1) in the mth row and n+1th column of G. If G(m,n)< G(m,n+1), set the eigenvalue F(m,n) in the mth row and nth column of the feature matrix to 0. Otherwise, set F(m,n) to 1, where 1≤m≤H and 1≤n≤W-1. Step 6: If the number of digital samples with defect category j in the defect digital sample library is 0, then after setting the initial value of the storage probability prob, a digital sample is formed together with S, G and F and added to the defect digital sample library with defect category j, and the process ends; If the number of digital sample libraries with defect category j in the defect digital sample library is greater than 0, execute step 7; Step 7: Use formula (1) to calculate the average eigenvalue Fmean(m,n) of the mth row and nth column in the average feature matrix Fmean of all digital samples with defect category j in the defect digital sample library: (1) In formula (1), K is the number of all digital samples with defect category j in the defect digital sample library; The eigenvalue of the mth row and nth column in the characteristic matrix F of the kth digital sample; Step 8: Calculate the mean of the feature matrix F in a digital sample using equations (2) and (3) and the mean of the average feature matrix Fmean : (2) (3) Step 9: Calculate the correlation coefficient corr between F and Fmean using formula (4): (4) Step 10: Use formula (5) to normalize corr to (0,1) to obtain the normalized correlation coefficient : (5) Step 11: Use formula (6) to normalize K to (0,1), so as to obtain the normalized number of all digital samples with defect category j. : (6) In formula (6), K_MAX is the maximum number of digital samples of defect categories in the digital sample library; Step 12: Use formula (7) to calculate the storage probability prob of the digital sample with defect category j: (7) In formula (7), w1, w2, w3 are three weights, and w1+w2+w3=1; Step 13: If prob>th_p, then prob, S, G and F are combined into a digital sample and added to the defect digital sample library of defect category j, and K+1 is assigned to K. Otherwise, it is not added to the library and the process ends, where th_p is the threshold for digital sample addition; Step 14: If K>K_MAX, delete the digital sample with the smallest probability of being put into storage among the digital samples with defect category j; otherwise, do not delete it and end the process.

2. An electronic device, comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the method for screening digital samples of cigarette defects as claimed in claim 1, and the processor is configured to execute the program stored in the memory.

3. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for screening digital samples of cigarette defects according to claim 1 are executed.

Images