A method and system for detecting defects in printed bottles
Through the maximum stable extreme value area extraction and Soble operator edge detection combined with weight sharing twin network, the problem of insufficient data for printing vase defect detection based on deep learning is solved, and efficient detection of printed vases is achieved, reducing human errors and improving production efficiency.
Patent Information
- Application Number
- CN202411147770.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-08-21
AI Technical Summary
Existing printed vase defect detection methods based on deep learning require a large amount of labeled data for training. For rare defects or emerging defects, insufficient data affects model performance, and traditional image processing methods are difficult to accurately separate defects when processing complex patterns and textures. Areas are easily disturbed by noise.
The maximum stable extreme value area is used for image area extraction, combined with Soble operator edge detection and weight sharing twin network, build a training data set and detection model, detect the detected images through the trained model, and judge the clarity of the printed pattern and text. and detect surface defects by comparing pixel intensity and texture.
Effectively reduce human errors, improve the production efficiency of printed vases, and be able to detect printing defects and printed text information at the same time, improving detection accuracy and efficiency.
Smart Images

Figure CN118657780B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of visual inspection, and in particular relates to a method and system for detecting defects in printed bottles. Background Art
[0002] Problems such as unstable humidity, dirty nozzles or printing platforms, and circuit board sensor failures may affect the printing quality, resulting in unclear or defective patterns. Such problems seriously affect the value of printed bottles, and manual inspection is usually used to remove such defective products. However, manual inspection requires a lot of time and human resources, and is limited by human vision and attention, especially in large-scale production. It is prone to missed inspections and false inspections, and the speed is far less than that of automated inspection systems.
[0003] In order to address these shortcomings, more and more companies are beginning to adopt automated inspection systems, such as machine vision inspection, artificial intelligence image analysis and other technologies, to improve inspection efficiency and accuracy, reduce costs, and achieve real-time monitoring and feedback. Existing methods for printing defect detection are relatively few, and most of them focus on the detection of printing defects on fabrics. There is a lack of system implementation that can detect printing and printed text defects at the same time. At the same time, traditional image processing methods may not be able to accurately separate defective areas when processing complex patterns and textures, and are easily affected by noise. Defect detection methods based on deep learning require a large amount of labeled data for training. For rare defects or newly emerging defects, there may be insufficient data, which affects model performance. Summary of the invention
[0004] The purpose of the present invention is to provide a printed bottle defect detection method, which aims to solve the problem that the defect detection method based on deep learning requires a large amount of labeled data for training, and for rare defects or newly emerging defects, there may be insufficient data, which affects the model performance.
[0005] The present invention is implemented as follows: a method for detecting defects in printed bottles, the method comprising:
[0006] Collect training samples, build training data sets, and obtain images to be detected;
[0007] The maximum stable extreme value region is used to extract the image region, and the text image is extracted;
[0008] Soble operator edge detection is used to evaluate the text contained in the text image to determine whether the printed text is clear;
[0009] A weight-sharing twin network is constructed and trained using the training data set to obtain a trained print image detection model. The print image detection model is used to detect the print pattern in the image to be detected.
[0010] Preferably, the step of constructing a training data set includes collecting different batches and different styles of printed patterns from the production line, recording the time and production number information of the samples, preprocessing the collected data, and marking defects thereon to obtain a training data set.
[0011] Preferably, the step of using the maximum stable extreme value region to extract the image region and extracting the text image includes: graying the image to be detected, performing binarization processing with different thresholds on the grayscale image, judging how the detection region changes with the threshold, using NMS to eliminate repeated regions, screening out the maximum stable extreme value region that does not contain overlapping parts, and using the known aspect ratio of the printed text and the Gabor features of the image to screen out the text region from MSER to obtain the text image.
[0012] Preferably, in the step of extracting the image region using the maximum stable extremum region and extracting the text image, the aspect ratio of the text is obtained from the information before printing; the Gabor feature is obtained by Gabor transformation, and the window function is used to decompose the non-stationary signal into a series of signal combinations that are stable in a short period of time, and the text region is screened out by setting a high-frequency component threshold.
[0013] Preferably, the change of the detection area with the threshold is determined by calculating the change rate of the gray value, and the change rate is expressed as:
[0014] ,
[0015] in, represents the grayscale threshold, The threshold is A connected area, is a small change in the gray threshold. When the change rate When it is a local minimum, It is the maximum stable extreme value area.
[0016] Preferably, the step of using the Soble operator edge detection to evaluate the text contained in the text image and determining whether the printed text is clear includes: using the Soble operator to calculate the gray value of each location in the text image,
[0017] ,
[0018] ,
[0019] ,
[0020] in, represents the original image, and Respectively represent the image grayscale values for horizontal and vertical detection, For the convolution operation, the grayscale images of the reference image and the image to be detected are obtained respectively. and , the difference between the two is determined by the structural similarity index:
[0021] ,
[0022] in, represents the mean value of the grayscale image of the reference image, Represents the mean value of the grayscale image to be detected, Represents the standard deviation of the reference image grayscale image, Represents the standard deviation of the grayscale image to be detected, Represents the covariance of the grayscale image of the reference image and the image to be detected, and is a constant to avoid the denominator being 0. Select multiple clear images to calculate Determine the clarity threshold to determine whether the target image is clear.
[0023] Preferably, a weight-sharing twin network is constructed, and the network is trained using a training data set to obtain a trained print image detection model. The step of detecting a print pattern in an image to be detected by using the print image detection model includes:
[0024] Train a weight-sharing Siamese network, inputting stamp images from the training set With high quality sample images , use the backbone network to extract image features, and convert the multi-dimensional image feature information into a one-dimensional vector through the fully connected layer and , calculate the distance between two vectors:
[0025] ,
[0026] Among them, n represents the number of features after the vector is converted into a one-dimensional vector. Indicates data, Is a printed image The characteristic data of is a high-quality sample image The feature data is normalized to obtain the similarity score , use cross entropy to measure the similarity with the image category label, and use the result as the loss function. Use the trained stamp image detection model to calculate the similarity score between the feature vectors of the image to be detected and the template image, determine the similarity between the image to be detected and the sample image, and judge the quality of the image to be detected.
[0027] Preferably, a weight-sharing twin network is constructed and trained using a training data set to obtain a trained print image detection model. In the step of detecting the print pattern in the image to be detected by the print image detection model, the weight-sharing twin network is reversely updated according to the loss function:
[0028] ,
[0029] in, The value of is 0 or 1, where 0 indicates the worst quality of the printed image, and 1 indicates the highest quality of the printed image.
[0030] Preferably, the method further comprises detecting surface defects, cracks and spots on the printed bottle by comparing the pixel intensity of each area and checking for discontinuity of color or texture, and confirming the location of the defects.
[0031] Another object of the present invention is to provide a printed bottle defect detection system, the system comprising:
[0032] The sample collection module is used to collect training samples, build training data sets, and obtain images to be detected;
[0033] A text image extraction module is used to extract the image region using the maximum stable extreme value region to obtain the text image;
[0034] The text recognition module is used to evaluate the text contained in the text image using the Soble operator edge detection to determine whether the printed text is clear;
[0035] The print detection module is used to construct a weight-sharing twin network, train it using a training data set, obtain a trained print image detection model, and detect the print pattern in the image to be detected through the print image detection model.
[0036] The method for detecting defects in printed bottles provided by the present invention can reduce human errors, and simultaneously detect printing defects and printed text information errors, which can effectively improve the production efficiency of printed bottles and has good engineering application value. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 A flow chart of a method for detecting defects in printed bottles provided by an embodiment of the present invention;
[0038] Figure 2 A flowchart of text quality assessment provided by an embodiment of the present invention;
[0039] Figure 3 An architectural diagram of a weight-sharing twin network provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0040] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.
[0041] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first xx script may be referred to as a second xx script, and similarly, a second xx script may be referred to as a first xx script.
[0042] like Figure 1 FIG. 1 is a flow chart of a method for detecting defects in printed bottles provided by an embodiment of the present invention, the method comprising:
[0043] Collect training samples, build training data sets, and obtain images to be detected.
[0044] In the present invention, if Figure 1 As shown in the figure, firstly, the area where text and patterns may exist in the input image is extracted based on MESR, and the extracted area is judged. If the area contains text information, the quality is evaluated based on the Soble operator. It is judged whether it meets the printing requirements. If the area contains pattern information, the twin network is used to evaluate the distance between the input image and the standard image, and then the defect judgment is performed. If there is a defect, the image is detected for contour and color to determine the approximate location and type of the defect, and the expert is asked to judge and analyze the cause.
[0045] In this step, the target detection method based on deep learning requires a large number of fully annotated data sets, and there are many situations in which defects in printed images may occur, and the data set production is difficult. Therefore, the present invention uses an image defect detection method based on a twin network to collect and annotate defective printed images and high-quality printed images as standard templates. Print images of different batches and styles are regularly collected from the production line to ensure data diversity. During the collection process, information such as the time and production line number of each sample is recorded to facilitate subsequent management and analysis. Finally, the collected data set is enhanced by translation, flipping, noise disturbance, etc. The collected images are preliminarily divided into two categories: defect-free images and images that may have defects. The quality inspection experts review the preliminarily classified images to confirm the defects. Mark the defect type, such as color difference, missing printing, and printing misalignment.
[0046] The maximum stable extremum region is used to extract the image region and the text image is extracted.
[0047] In this step, the image region is extracted using the Maximally Stable Extremal Regions (MSER), the duplicate frames are removed using the Non-Max Suppression (NMS), the region aspect ratio is set, and the character region is extracted using the Gabor filter. The specific operation process is as follows:
[0048] First, grayscale the target image (the image to be detected), and perform binarization with different thresholds on the grayscale image. Determine how the detection area changes with the threshold. This is shown in the following formula:
[0049] ,
[0050] in, represents the grayscale threshold, The threshold is A connected area, is a small change in the grayscale threshold. When it is a local minimum, It is the maximum stable extreme value area.
[0051] Then, NMS is used to remove duplicate regions and filter out the maximum stable extreme value region that does not contain overlapping parts.
[0052] Finally, the text area is screened out from the MSER using the known aspect ratio of the printed text and the Gabor features of the image. The aspect ratio of the text is obtained from the information before printing; the Gabor features are obtained through Gabor transformation, that is, the window function is used to decompose the non-stationary signal into a series of short-term stable signal combinations. Then, the high-frequency information of the text area is enriched, and the high-frequency component threshold is set to screen out the text area.
[0053] In this step, if Figure 2 As shown in the figure, after preprocessing the image, MSER detection and connected regions after non-maximum suppression are used to determine whether it is character information by the area aspect ratio, and then Gabor is used to extract the texture feature values of the four strokes in the character string area. Using the four sets of texture feature values as feature vectors, a classifier is designed to determine whether the connected domain string area is a character string, and then the quality of the character is evaluated to determine whether it has defects.
[0054] The image is converted into a grayscale image, and the image is smoothed using a two-dimensional Gaussian function through Gaussian blur to remove noise in the image and improve the stability of MSER detection. The two-dimensional Gaussian function is expressed as:
[0055] ,
[0056] In the formula, is the amplitude, and is the center point coordinate, and is the standard deviation. The best effect is obtained by adjusting the standard deviation. Then the MSER algorithm is used to detect the extreme value area in the smoothed grayscale image. The coordinates of the lower right corner of the extreme value area are put into a list from small to large, and then the lower right corner box is taken out, and the overlapping area ratio of the remaining boxes and the lower right corner box is calculated. The repeated rectangular boxes are deleted according to the threshold of the overlapping ratio. The non-text area cannot be completely filtered by limiting the aspect ratio of the area, so further classification methods are needed. The Gabor filter is used to extract the texture feature values of the four strokes in the character string area. With four sets of texture feature values as feature vectors, a classifier is designed to determine whether the connected domain string area is a character.
[0057] The Soble operator edge detection is used to evaluate the text contained in the text image to determine whether the printed text is clear.
[0058] In this step, the quality of the text depends largely on whether the edge of the text is clear. Therefore, the present invention uses the Soble operator edge detection as the quality score to evaluate the text and determine whether the text is clear and visible.
[0059] First, use the Soble operator to calculate the grayscale size of each part of the image. As shown in the following formula:
[0060] ,
[0061] ,
[0062] ,
[0063] in, represents the original image, and Respectively represent the image grayscale values for horizontal and vertical detection, is the convolution operation.
[0064] Then obtain the grayscale images of the reference image and the image to be detected respectively , Finally, we use the idea of structural similarity index to judge the difference between the two, as shown in the following formula:
[0065] ,
[0066] in, represents the mean value of the grayscale image of the reference image, Represents the mean value of the grayscale image to be detected, Represents the standard deviation of the reference image grayscale image, Represents the standard deviation of the grayscale image to be detected, Represents the covariance of the grayscale image of the reference image and the image to be detected, and is a constant to avoid the denominator being 0. Select multiple clear images to calculate Determine the clarity threshold to determine whether the target image is clear.
[0067] In this step, the image containing text is read and converted into a grayscale image for edge detection. Then the Sobel edge strength is calculated for each region, and the average value is calculated as the quality score of the region. The clarity of the characters is judged based on the quality score.
[0068] A weight-sharing twin network is constructed and trained using the training data set to obtain a trained print image detection model. The print pattern in the image to be detected is detected through the print image detection model.
[0069] In this step, a weight-sharing twin network (i.e., a stamp image detection model) is trained. The specific training steps are as follows: Input a stamp image from the training set With high quality sample images . Use the backbone network to extract image features and convert the multi-dimensional image feature information into a one-dimensional vector through the fully connected layer and . Use the L1 norm to calculate the distance between two vectors:
[0070] ,
[0071] Among them, n represents the number of features after the vector is converted into a one-dimensional vector. Indicates data, Is a printed image The characteristic data of is a high-quality sample image The feature data is normalized to obtain the similarity score , use cross entropy to measure the similarity with the image category label, and use the result as the loss function. Use the trained stamp image detection model to calculate the similarity score between the feature vectors of the image to be detected and the template image, determine the similarity between the image to be detected and the sample image, and judge the quality of the image to be detected.
[0072] ,
[0073] in The value of is 0 or 1, where 0 indicates the worst quality of the printed image, and 1 indicates the highest quality of the printed image.
[0074] The trained twin neural network is used to calculate the similarity score between the feature vectors of the printed image to be detected and the template image, and the similarity between the image to be detected and the sample image is determined, thereby judging the quality of the image to be detected.
[0075] In this step, if Figure 3 As shown, after two weight-sharing convolutional neural networks extract features from the image, the L1 norm is used to discriminate the difference between the two, thereby determining whether there are defects.
[0076] Construct a twin network consisting of two sub-networks with the same structure and shared parameters, then use the two images as network input and output their feature representations. Use the dataset to train the twin network. During training, for each pair of input images, calculate the loss function according to their labels to adjust the network parameters. Use the L1 norm to calculate the difference between the target image and the sample image to verify whether they have the same defects.
[0077] As a preferred embodiment of the present invention, in order to further confirm the location of the defect, the detection of defects, cracks or spots on the surface of the printed bottle is achieved by comparing the pixel intensity of each area and checking the discontinuity of color or texture. The image is detected for defects by comparing the pixel intensity of each area and checking the color. The gradient of the image is calculated using the Sobel operator, and then the gradient image is thresholded, and the contour of the image is found to detect defects. The image is converted to the Lab color space, and then the a and b channels are extracted, the gradients of the a and b channels are calculated, and they are merged into a gradient amplitude image, and thresholding is performed. Finally, the contour of the image is found to detect the defective area. After that, the production line record information of the data set is combined for analysis to troubleshoot the cause of the fault.
[0078] The present invention also provides a printed bottle defect detection system, the system comprising:
[0079] The sample collection module is used to collect training samples, build training data sets, and obtain images to be detected;
[0080] A text image extraction module is used to extract the image region using the maximum stable extreme value region to obtain the text image;
[0081] The text recognition module is used to evaluate the text contained in the text image using the Soble operator edge detection to determine whether the printed text is clear;
[0082] The print detection module is used to construct a weight-sharing twin network, train it using a training data set, obtain a trained print image detection model, and detect the print pattern in the image to be detected through the print image detection model.
[0083] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.
[0084] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0085] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0086] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.
[0087] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for detecting defects in printed bottles, characterized in that: The method comprises: Collect training samples, build training data sets, and obtain images to be detected; The maximum stable extreme value region is used to extract the image region, and the text image is extracted; Soble operator edge detection is used to evaluate the text contained in the text image to determine whether the printed text is clear; Construct a weight-sharing twin network, train it using a training data set, obtain a trained print image detection model, and use the print image detection model to detect the print pattern in the image to be detected; The step of collecting training samples and constructing a training data set includes collecting different batches and different styles of printed patterns from the production line, recording the time and production number information of the samples, preprocessing the collected data, and marking defects on the data to obtain a training data set; The step of extracting the image region by using the maximum stable extreme value region to obtain the text image comprises: graying the image to be detected, performing binarization processing with different thresholds on the gray image, judging the change of the detection region with the threshold, removing repeated regions by using NMS, screening out the maximum stable extreme value region that does not contain overlapping parts, and screening out the text region from MSER by using the known aspect ratio of the printed text and the Gabor features of the image to obtain the text image; In the step of extracting the image region by using the maximum stable extremum region, the aspect ratio of the text is obtained from the information before printing; the Gabor feature is obtained by Gabor transformation, and the window function is used to decompose the non-stationary signal into a series of short-term stable signal combinations, and the text region is screened out by setting the high-frequency component threshold; The change of the detection area with the threshold is determined by calculating the change rate of the gray value, which is expressed as: , in, represents the grayscale threshold, The threshold is A connected area, is a small change in the gray threshold. When the change rate When it is a local minimum, is the maximum stable extreme value area; The steps of evaluating the text contained in the text image by using the Soble operator edge detection and determining whether the printed text is clear include: using the Soble operator to calculate the gray value of each part of the text image, , , , in, Show the original image, and Respectively represent the image grayscale values for horizontal and vertical detection, For the convolution operation, the grayscale images of the reference image and the image to be detected are obtained respectively. and , the difference between the two is determined by the structural similarity index: , in, represents the mean value of the grayscale image of the reference image, Represents the mean value of the grayscale image to be detected, Represents the standard deviation of the reference image grayscale image, Represents the standard deviation of the grayscale image to be detected, Represents the covariance of the grayscale image of the reference image and the image to be detected, and is a constant to avoid the denominator being 0. Select multiple clear images to calculate Determine the clarity threshold to determine whether the target image is clear; The steps of constructing a weight-sharing twin network, training it with a training data set, obtaining a trained print image detection model, and detecting the print pattern in the image to be detected by the print image detection model include: Train a weight-sharing Siamese network, inputting stamp images from the training set With high quality sample images , use the backbone network to extract image features, and convert the multi-dimensional image feature information into a one-dimensional vector through the fully connected layer and , calculate the distance between two vectors: , Among them, n represents the number of features after the vector is converted into a one-dimensional vector. Indicates data, Is a printed image The characteristic data of is a high-quality sample image The feature data is normalized to obtain the similarity score , use cross entropy to measure similarity with the image category label, and use the result as the loss function. Use the trained stamp image detection model to calculate the similarity score between the feature vectors of the image to be detected and the template image, determine the similarity between the image to be detected and the sample image, and judge the quality of the image to be detected; Construct a weight-sharing twin network and train it using the training data set to obtain a trained stamp image detection model. In the step of detecting the stamp pattern in the image to be detected by the stamp image detection model, the weight-sharing twin network is reversely updated according to the loss function: , in, The value of is 0 or 1, where 0 indicates the worst quality of the printed image, and 1 indicates the highest quality of the printed image; The method also includes detecting surface defects, cracks and spots on the printed bottle by comparing the pixel intensity of each area and checking for discontinuities in color or texture, and confirming the location of the defects.
2. A printed bottle defect detection system, used to implement the printed bottle defect detection method according to claim 1, characterized in that: The system comprises: The sample collection module is used to collect training samples, build training data sets, and obtain images to be detected; A text image extraction module is used to extract the image region using the maximum stable extreme value region to obtain the text image; The text recognition module is used to evaluate the text contained in the text image using the Soble operator edge detection to determine whether the printed text is clear; The print detection module is used to construct a weight-sharing twin network, train it using a training data set, obtain a trained print image detection model, and detect the print pattern in the image to be detected through the print image detection model.
