Non-supervision-based high-efficiency multi-layer overprinting silk-screen printing defect detection method

Abstract

The invention relates to a non-supervision-based high-efficiency multilayer overprinting silk-screen printing defect detection method. The method comprises the following steps: collecting sample images, dividing the sample images into a training set and a test set, and training a teach network and a student network; inputting an image to be detected into a teach network and a student network at the same time to obtain two feature vectors; abnormal scores of the two feature vectors are calculated, and a thermodynamic diagram reflecting the abnormal degree is generated according to normalized abnormal score values; automatically completing the calculation of the optimal threshold according to the maximum thermodynamic value obtained by the thermodynamic diagram and the NG and OK annotation information of the verification set; and determining a final detection result according to the thermodynamic diagram and a threshold value, and displaying contour information of the abnormal region. The abnormal sample can be efficiently detected and the abnormal position can be positioned only by learning a small number of normal samples; during training, an optimal model is automatically generated and an optimal threshold value is searched, and a detection result is given through a thermodynamic diagram generated by the model and the optimal threshold value, so that the field detection efficiency and the detection precision are greatly improved.

Description

Technical Field

[0001] The invention relates to the technical field of image vision detection, in particular to an unsupervised and efficient multi-layer overprinting screen printing defect detection method. Background Art

[0002] In daily life, logo patterns are printed on the outer packaging of various consumer products. In order to improve the beauty of the patterns and prevent counterfeiting, manufacturers will make complex graphic structures and match them with rich colors, which requires multi-layer overprinting. In addition, some products will add complex means such as hot stamping to further beautify the product packaging. Before the birth of machine vision technology, the quality inspection of these complex patterns was mainly completed manually. With the continuous advancement of technology and the improvement of computing power, the use of machine vision technology to replace manual quality inspection has developed rapidly. In recent years, with the continuous development of artificial intelligence (AI) technology, it has become a trend to use AI to replace human eyes. For screen-printed products with multi-layer overprinting, the patterns are generally complex. In actual scenes, there are insufficient defect samples, but the types of defects are diverse, and the long-tail distribution is obvious. It is difficult for supervised AI detection methods to collect a large number of defect samples, and due to the complex background and subtle defects, they face very serious out-of-distribution (OOD) detection problems.

[0003] At present, traditional comparison algorithms are generally used to detect multi-layer overprinted silk-screen printed products. That is, first select good products that meet the requirements, and then use traditional algorithms to compare the products to be inspected with the good products at the pixel level. For some areas where there are local deviations but meet customer requirements, the edge relaxation strategy is used to ignore the abnormalities in the area, and finally the defective area is found by adjusting the threshold.

[0004] However, the prior art mainly has the following disadvantages:

[0005] 1. At present, the process of multi-layer overprinting and printing products is complicated, and there are processes such as multi-layer printing and hot stamping. The actual product may have color difference, slight deviation and other acceptable conditions. Although the pixel-level comparison method can avoid large over-killing through some post-production strategies, it will also lead to the omission of local minor defects. It is difficult to find an effective strategy to balance over-killing and omission.

[0006] 2. Traditional algorithms require manual adjustment of thresholds, which may have different thresholds for different regions. Manual adjustment of thresholds is blind and cannot automatically optimize, making it difficult to take into account the various possible defects.

[0007] There are also some unsupervised anomaly detection methods, but they can only detect relatively simple screen-printed products and cannot handle minor defects caused by multi-layer overprinting. Summary of the invention

[0008] The technical problem to be solved by the present invention is to provide an unsupervised and efficient multi-layer overprinting screen printing defect detection method, which solves the problem of efficient detection and positioning of various unforeseen defects of multi-layer overprinting screen printing products and the problem of rapid iteration of the detection model.

[0009] The technical solution adopted by the present invention to solve the technical problem is: an unsupervised and efficient multi-layer overprinting screen printing defect detection method, comprising the following steps:

[0010] S1. Collect normal sample images and abnormal sample images, divide them into training sets and test sets, and train the teacher network and student network based on the ST structure network;

[0011] S2, input the image to be detected into the teacher network and the student network at the same time to obtain two feature vectors;

[0012] S3. Calculate the anomaly scores of the two feature vectors, and generate a heat map reflecting the degree of anomaly according to the normalized anomaly score values;

[0013] S4. Automatically calculate the optimal threshold value based on the maximum thermal value obtained from the thermal map and the NG and OK annotation information of the validation set;

[0014] S5. Determine the final detection result based on the heat map and the threshold, and display the contour information of the abnormal area.

[0015] Furthermore, in step S1 of the present invention, normal samples are used in the training process, wherein the teacher network loads a pre-trained model, only output features are given without parameter update, and the student network fits the output of the teacher network.

[0016] Furthermore, in step S3 of the present invention, the difference vector Δ output by the teacher network and the student network is T-S It is expressed as: Among them, μ is the mean vector of each normal sample in the training data set output through each channel of the teacher network, and σ is the standard deviation vector.

[0017] Furthermore, the present invention performs a T-S Each channel element is subjected to quantile calculation, and Δ T-S Divide the channel into K regions, use multithreading to parallelly calculate the 0.999 quantiles of the K regions, and finally take the 0.999 quantiles of the K regions, retaining Δ T-S The difference vector Δ is obtained by taking the elements above 99.9% quantile pct＝0.999 .

[0018] Furthermore, in step S4 described in the present invention, the scoring formula of the maximum thermal value is: score = (v1-u1) / (u2-u1), wherein V1 is the vector difference, u1 is the 90% quantile vector of the difference vector v1, and u2 is the maximum value vector.

[0019] Furthermore, the difference feature tensor after quantile normalization of the present invention is ~Δ T-S =(Δ T-S -p1) / (p2-p1); where Δ T-S It is the difference feature tensor output by the teacher network and the student network of the test set, p1 is the 90% quantile of the validation set, and p2 is the maximum value.

[0020] The beneficial effect of the present invention is that it solves the defects existing in the background technology. The present invention uses an unsupervised deep learning anomaly detection model to efficiently detect abnormal samples and locate abnormal positions by learning only a small number of normal samples; the optimal model is automatically generated and the optimal threshold is searched during the training process, and the detection results are given end-to-end through the heat map and the optimal threshold generated by the model; for the problems that multi-layer overprinting screen printing products change quickly, have a wide variety of defects, are small, and are difficult to foresee, the on-site detection efficiency and detection accuracy can be greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic diagram of the network structure of the unsupervised detection model of the present invention;

[0022] Figure 2 It is a principle diagram of the detection process of the unsupervised detection model of the present invention;

[0023] Figure 3 It is a schematic diagram of over-killing and missed detection under different thresholds of the threshold search algorithm of the present invention. DETAILED DESCRIPTION

[0024] The present invention will now be described in further detail in conjunction with the accompanying drawings and preferred embodiments. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0025] like Figure 1-Figure 3 An efficient unsupervised multi-layer overprinting screen printing defect detection method is shown, which comprises:

[0026] 1. ST structure network: including teacher feature extraction network and student feature extraction network; based on the ST structure network, it includes two networks, teacher and student. The network structure of teacher and student is as follows Figure 1As shown in Figure 2, it is used to input the image to be inspected into two networks at the same time to obtain two feature vectors.

[0027] 2. A heat map generation module that displays abnormal conditions; although the differential output of the teacher and student networks can initially determine abnormal conditions, for multi-layer overprinted screen-printed products, a certain degree of robustness against slight deviations is required during the actual detection process. Figure 1 The two feature vectors output by the network structure in are as follows Figure 2 The process shown calculates the anomaly score of each pixel and generates a heat map reflecting the degree of anomaly based on the normalized anomaly score value.

[0028] 3. Automatic search module for abnormal thresholds: used to automatically calculate the optimal threshold based on the maximum thermal value obtained from the thermal map and the NG and OK annotation information of the validation set.

[0029] 4. Abnormal area output module: used to determine the final detection result based on the heat map and threshold, and display the contour information of the abnormal area.

[0030] Principle description:

[0031] 1. The ST structure network uses normal samples during the training process. The teacher network loads the pre-trained model and only gives the output features without updating the parameters. The student network can fit the output of the teacher. The trained model is consistent with the teacher output when processing normal areas. However, when processing abnormal areas, since the student is not involved in the training process, the features of the abnormal areas will be greatly different from those of the teacher. This allows for a preliminary determination of abnormal areas. On the other hand, considering the inference efficiency factor, the model constructed here also meets the lightweight requirements.

[0032] 2. Figure 1 The teacher and student difference feature vectors obtained by the network structure in may contain some situations that are not caused by defects, such as slight offsets and tiny impurities. In order to further display the main differences, the 90% quantile vector u1 and the maximum value vector u2 of the difference vector V1 are calculated. The final thermal value score is given by formula 1:

[0033] score=(v1-u1) / (u2-u1) (1)

[0034] Finally, the heat map filters out some deviations caused by non-substantial defects, allowing the model to better identify abnormal areas and significantly reduce background over-detection. On the other hand, for multiple samples, the difference values ​​caused by non-abnormalities in some areas may be higher than the difference values ​​caused by abnormalities in another sample. This module avoids the problem of missed detection or over-detection caused by setting the threshold too high or too low, thereby focusing on real defects.

[0035] 3. Initially, according to Figure 2 The obtained heat map data statistically calculates the maximum heat value of each sample in the validation set, and performs a threshold search from the minimum to the maximum heat value with a step size of 0.01 according to the NG and GOOD labels of the samples. The model missed detection number escape_amount and overkill number overkill_amount under each threshold are recorded, and the missed detection sensitivity coefficient ε is set. The optimal threshold is when ε×escape_amount+overkill_amount reaches the minimum.

[0036] As attached Figure 2 As shown in the figure, after the sample image is input into both the teacher and student networks, two feature tensors are obtained. When there are abnormal areas in the sample, the student model trained only with normal samples and the teacher model trained with the pre-trained model will generate difference features, and the main difference features are selected through the post-processing algorithm to ignore the influence of noise, and then normalized to form a heat map of the abnormal area. Finally, the best threshold obtained by the threshold search algorithm is used to form an abnormality judgment and location.

[0037] Steps:

[0038] Training and testing steps:

[0039] 1. Create training and testing data sets

[0040] a. Collect 120-150 normal sample images and a small number of abnormal sample images of the items to be tested;

[0041] b. The collected normal sample images are randomly divided into training set and test set in a 1:1 ratio;

[0042] c. Place the normal samples divided into the training set into the good folder under the train directory, place the normal samples divided into the test set in the previous step into the good folder under the test directory, and place all abnormal samples into the ng folder under the test directory.

[0043] 2. Training the detection model

[0044] a. Load training and test samples, retain 10% of the samples in the training set for verification, and the remaining 90% for training;

[0045] b. Initialize the model parameters, where the teacher network loads the pre-trained model parameters, sets the Adam training optimizer learning rate parameter to 0.0001, and weight_decay to 0.00001; sets the number of training steps to 80000, sets the learning rate scheduler step_size to 76500, and gamma to 0.1.

[0046] c. Calculate the mean vector μ and standard deviation vector σ of each channel output of each normal sample in the training data set through the teacher network.

[0047] d. During the training process, the teacher network does not participate in the training, only the student network is trained, and the difference vector Δ between the output of the teacher network and the student network T-S Expressed as:

[0048]

[0049] In order to retain the main abnormal features and reduce the influence of noise, the quantiles of each channel element of the difference vector Δ are calculated. During the calculation process, Δ is divided into K regions according to the channel dimension, and the 0.999 quantiles of the K regions are calculated in parallel using multi-threading. Finally, the 0.999 quantile of the K regions is taken, and the elements above 99.9% quantile in Δ are retained to obtain the difference vector Δ pct＝0.999 . So when Batchsize is N, the training loss is expressed as:

[0050]

[0051] where u i ∈Δ pct＝0.999 .

[0052] e. During the training process, when the loss value is lower than the threshold η=1, a test set inference is performed every time the iteration step is an integer multiple of 5000. During the inference process, the thermal value corresponding to the 90% quantile p1 and the maximum value p2 of the validation set is first calculated according to formula (1).

[0053] f. Calculate the difference feature tensor Δ between the teacher network and the student network output of the test set according to formula (2) T-S , according to formula (4), calculate the difference feature tensor after quantile normalization in e The corresponding abnormal heat map is generated by bilinear interpolation to the original sample image size.

[0054]

[0055] g. Record the maximum thermal value of each sample in the current test set, and traverse the threshold from 0.05 to 0.95 at intervals of 0.05. Find the optimal threshold parameter based on the good and ng flags of the test set.

[0056] h. Determine sample anomalies and locate abnormal areas based on the abnormal heat map and optimal threshold.

[0057] Therefore, the present invention adopts AI algorithm, which can automatically learn relevant image features by learning a small number of normal samples. The features not only contain the shallow contour information of the samples, but also combine the deep semantic information. When facing slight deviations, the abnormal score of the area can be comprehensively judged by the quantiles of different levels of features and abnormal features, so that the model can better focus on the detection of real defects and ignore acceptable deviations.

[0058] The above description only describes the specific implementation mode of the present invention. Various examples do not limit the essential content of the present invention. After reading the description, ordinary technicians in the relevant technical field can modify or deform the specific implementation modes described above without departing from the essence and scope of the invention.

Claims

1. An unsupervised and efficient multi-layer overprint screen printing defect detection method, characterized in that: The following steps are included: S1. Collect normal sample images and abnormal sample images, divide them into training sets and test sets, and train the teacher network and student network based on the ST structure network; S2, input the image to be detected into the teacher network and the student network at the same time to obtain two feature vectors; S3. Calculate the anomaly scores of the two feature vectors, and generate a heat map reflecting the degree of anomaly according to the normalized anomaly score values; S4. Automatically calculate the optimal threshold value based on the maximum thermal value obtained from the thermal map and the NG and OK annotation information of the validation set; S5. Determine the final detection result based on the heat map and the threshold, and display the contour information of the abnormal area.

2. The unsupervised and efficient multi-layer overprinting screen printing defect detection method according to claim 1, characterized in that: In step S1, normal samples are used in the training process, wherein the teacher network loads the pre-trained model, only output features are given without parameter update, and the student network fits the output of the teacher network.

3. The unsupervised and efficient multi-layer overprinting screen printing defect detection method according to claim 1, characterized in that: In step S3, the difference vector Δ output by the teacher network and the student network is T-S It is expressed as: Among them, μ is the mean vector of each normal sample in the training data set output through each channel of the teacher network, and σ is the standard deviation vector.

4. The unsupervised and efficient multi-layer overprinting screen printing defect detection method according to claim 3, characterized in that: For the difference vector Δ T-S Each channel element is subjected to quantile calculation, and Δ T-S Divide the channel into K regions, use multithreading to parallelly calculate the 0.999 quantiles of the K regions, and finally take the 0.999 quantiles of the K regions, retaining Δ T-S The difference vector Δ is obtained by taking the elements above 99.9% quantile pct＝0.999 .

5. The unsupervised and efficient multi-layer overprinting screen printing defect detection method according to claim 4, characterized in that: In step S4, the scoring formula of the maximum thermal value is: score = (v1-u1) / (u2-u1), where V1 is the vector difference, u1 is the 90% quantile vector of the difference vector v1, and u2 is the maximum value vector.

6. The unsupervised and efficient multi-layer overprinting screen printing defect detection method according to claim 4, characterized in that: The difference feature tensor after quantile normalization is ~Δ T-S =(Δ T-S -p1) / (p2-p1); where Δ T-S It is the difference feature tensor output by the teacher network and the student network of the test set, p1 is the 90% quantile of the validation set, and p2 is the maximum value.

Images