Industrial product surface defect localization method integrating learning and knowledge

Abstract

The present invention discloses a method for locating surface defects of industrial products that integrates learning and knowledge. Surface defect location of industrial products is an important part of industrial product quality inspection, and it is crucial to achieve accurate and efficient defect location. Usually, defective images are also called abnormal samples, and defect-free images are also called normal samples. There are currently many target detection and defect detection methods, but they require a large number of abnormal samples and relatively accurate abnormal area location annotations, which is difficult to achieve in actual production environments. In practical applications, domain experts have summarized valuable rule knowledge in long-term practice, and the rational use of rules can further improve the performance of abnormal location. The present invention proposes a method for locating surface defects of industrial products that integrates learning and knowledge. It can use a small number of weakly labeled abnormal samples and rule knowledge to help the network learn abnormal patterns on the basis of a large number of normal samples, thereby improving the performance of abnormal location.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and relates to a defect localization method, and in particular to an industrial product surface defect localization method based on fusion learning and knowledge. Background Art

[0002] With the rapid development of artificial intelligence and computer vision technologies, computer vision has been widely applied in various production and daily life fields, such as anomaly localization, object detection, defect detection, face recognition, pose estimation, image generation, and more. Among them, anomaly localization, object detection, and defect detection can all accomplish the task of locating defective areas in images. Defective images are often referred to as anomaly samples, while defect-free images are referred to as normal samples. Numerous object detection and defect detection methods currently exist, but they require a large number of anomaly samples and annotated locations of anomaly regions. However, in real industrial scenarios, normal samples are readily available and plentiful, while anomaly samples are often difficult to obtain and few in number. Furthermore, annotating the locations of anomaly regions requires significant manpower. In practical applications, object detection and defect segmentation methods often suffer from a shortage of anomaly samples and label dependency. Anomaly localization methods utilize a large number of normal samples to achieve anomaly localization, alleviating these issues to a certain extent.

[0003] Among anomaly localization methods, the most common is based on image reconstruction. The reconstruction network uses a large number of normal samples to learn normal patterns, enabling it to reconstruct normal regions accurately. The network's reconstruction residuals for normal regions are small. However, anomaly regions differ significantly from normal regions and cannot be reconstructed by the network. The network's reconstruction residuals for anomaly regions are large. Therefore, using the image reconstruction residuals can help locate anomaly regions.

[0004] Classic anomaly localization methods use normal samples to accomplish this task. However, when trained solely on normal samples, localization performance is often unsatisfactory. In many industrial scenarios, a small number of weakly labeled anomaly samples are often available at a low cost. If these small numbers of anomaly samples can be used to help the network learn some anomaly patterns, anomaly localization performance will be improved.

[0005] In real-world applications, domain experts have accumulated valuable knowledge about rules through long-term practice. Experts describe and summarize abnormal regions and develop corresponding rules. These rules provide more information about abnormal regions, helping the network better learn abnormal patterns and further improving anomaly localization performance.

[0006] This paper proposes a method for locating surface defects in industrial products that integrates learning and knowledge. Based on a large number of normal samples, the proposed method uses a small number of weakly labeled defect images and rule-based knowledge to help the network learn abnormal patterns, thereby improving defect localization performance. Summary of the Invention

[0007] The purpose of this invention is to address the shortcomings of existing defect localization methods and propose an industrial product surface defect localization method that integrates learning and knowledge to improve defect localization performance.

[0008] The method for locating surface defects of industrial products that integrates learning and knowledge includes the following steps:

[0009] Step 1: Obtain industrial product surface images and perform preprocessing and data enhancement;

[0010] Step 2: Design an industrial product surface image reconstruction network and its loss function, and use defect-free industrial product surface images to train the network to obtain a pre-trained model;

[0011] Step 3: Summarize the laws of surface defects of industrial products based on the formation principles of defects and actual conditions, and design defect rules based on the laws;

[0012] Step 4: Calculate the reconstructed residual map from the defect image and the pre-trained model. Based on the defect rule, design a threshold selection mechanism to divide the defect area from the reconstructed residual map to obtain a pseudo label.

[0013] Step 5. Based on the principle that the reconstruction error of normal pixels is small and the reconstruction error of defective pixels is large, a loss function is designed to fine-tune the pre-trained model. The network is reconstructed using the pseudo-labels from the previous round. When the network training epoch reaches the preset number, the current round of training is stopped and the pseudo-labels obtained in this round are used for the next round of fine-tuning and reconstructing the network. This is done until the verification set error is less than the preset threshold. The defect localization model is obtained.

[0014] Furthermore, the preprocessing process described in step 1 is to unify the image size and normalize the grayscale value of the image; the data enhancement includes: image flipping, image rotation, and mixup, and mixup is to perform weighted averaging on two random defect-free industrial product surface images to obtain a new normal sample.

[0015] Furthermore, the reconstruction network in step 2 is a convolutional autoencoder, including an encoder and a decoder; the encoder includes a convolutional layer and a LeakyReLU activation layer. The encoder extracts semantic information from the image, then extracts features with representational meaning from the semantic information, and outputs a feature vector. The decoder has a symmetrical structure with the encoder and restores the image based on the feature vector. The loss function is L2 Loss, which calculates the residual between the output image and the original image at each point and sums the residuals of all points. The specific form is:

[0016]

[0017] Among them, x(r,c) and are the grayscale values ​​of the original image and the reconstructed image at point (r, c), h and w are the height and width of the image respectively. The method for training the network is the error back propagation method.

[0018] Furthermore, the defect rule described in step 3 adopts a rule representation method based on fuzzy logic, pre-sets various attributes of the defect, and calculates the rule score of each attribute. The defect rule calculation can describe several attributes. For a rule, the "maximum-multiplication" synthesis operator is used to comprehensively consider the rule scores of each attribute to obtain the rule score under the rule. Assuming there are n rules, n different rule scores rs1, rs2, ..., rs can be obtained. n , and finally the rule score of the region is the disjunction of all rule scores:

[0019] rs=∨ i rs i .

[0020] Furthermore, the threshold selection mechanism described in step 4 is to traverse the candidate thresholds from low to high for each defect sample, calculate the rule score of each segmented area according to the rule, and when the rule score of the segmented area exceeds a certain threshold, it is considered that the area is likely to be a defect area, and set a relative threshold score to measure the quality of the threshold. For all possible defect areas, the defect area determined by the threshold with the highest relative threshold score is the final defect area.

[0021] Furthermore, the relative threshold score is the ratio of the threshold score to the reference threshold score, wherein the threshold score is the product of the rule score and the area of ​​the defect area, and the reference threshold score is the threshold score when the rule score is greater than or equal to 0.8 for the first time.

[0022] Furthermore, a connected region may contain multiple defective regions. As the threshold value increases, different defective regions may be separated, and the optimal threshold value will be determined separately for each separated sub-region.

[0023] Furthermore, the specific form of the loss function in step 5 is:

[0024]

[0025] Among them, x(r,c) is the gray value of the image at point (r,c), h and w are the height and width of the image respectively, N n is the number of normal pixels in the pseudo label, N an is the number of defective pixels in the pseudo label, PL is the pseudo label, c an is a hyperparameter, L N+AN+R (x) is the loss function. It can be seen that L N+AN+R (x) generates positive loss for normal points in pseudo labels and negative loss for abnormal points in pseudo labels. N+AN+R When (x) is used as the loss function during network fine-tuning, the normal points perform gradient ascent on the network, and the abnormal points perform gradient descent on the network.

[0026] Furthermore, the reconstructed residual image is post-processed using a guided filtering method to reduce the influence of noise. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a specific implementation flow chart of the method of the present invention;

[0028] Figure 2 It is a network structure diagram of the convolutional autoencoder in the method of the present invention;

[0029] Figure 3 This is an experimental example of the method of the present invention (a is the original image, b is the reconstructed image, c is the reconstructed residual image, and d is the pseudo label);

[0030] Figure 4 The following are the prediction results of the present invention and the comparison with the benchmark results. DETAILED DESCRIPTION

[0031] In order to make the purpose and advantages of the present invention more clear, the specific implementation methods and steps of the method proposed in the present invention are further described below.

[0032] like Figure 1 As shown in Figure 2, the specific implementation steps of the industrial product surface defect localization method that integrates learning and knowledge are as follows:

[0033] Step 1: Obtain surface images of industrial products such as magnetic tiles or metals, preprocess them, and perform data enhancement using random flipping and mixup methods:

[0034] The dataset for locating surface defects of industrial products is images collected by image acquisition devices in actual industrial scenes. The data used in this paper comes from the magnetic tile dataset, the MVTec AD dataset, and the KolektorSDD2 dataset.

[0035] The obtained raw data is preprocessed: the image size is scaled to 256*256, and the grayscale value of the image is normalized, that is,

[0036]

[0037] Where max(x) and min(x) are the maximum and minimum grayscale values ​​of the image, respectively. To increase the training data and improve the generalization ability of the model, data augmentation was performed on the image. Specific data augmentation methods include image flipping, image rotation, and mixup. The mixup data augmentation method is cited from [Zhang H, Cisse M, Dauphin YN, et al. mixup: Beyond empirical risk minimization [J]. arXiv preprint arXiv: 1710.09412, 2017.]. Mixup takes a weighted average of two random normal samples to obtain a new normal sample, which can greatly increase the amount of normal sample data. The mixup formula is:

[0038]

[0039] where x i and x j are two randomly selected normal sample images, and λ is a hyperparameter in [0,1].

[0040] Step 2: Design an industrial product surface image reconstruction network and its loss function, and use defect-free industrial product surface images to train the network to obtain a pre-trained model:

[0041] The present invention designs a convolutional autoencoder as a reconstruction network. The network design is shown in Table 1, where Conv refers to the convolution layer and ConvT refers to the transposed convolution layer. In the encoder, 10 stacked convolution layers are used. In order to enhance the nonlinear fitting ability of the network, each convolution layer is followed by a LeakyReLU activation layer. As the number of layers increases, the number of convolution kernels in the convolution layer first increases and then decreases, and finally outputs a feature vector of 1*1*200. The encoder first extracts rich semantic information from the image, and then extracts features with representational significance from the semantic information. The structure of the decoder is symmetrical to that of the encoder. It receives a 200-dimensional input vector and finally outputs a 256*256 image. The decoder restores the image based on the representation vector. The network structure is as follows: Figure 2 The fully convolutional neural network used in the present invention can perform global semantic understanding of images, and the training parameters are greatly reduced compared to the fully connected network, which is conducive to better completing the anomaly localization task.

[0042] Table 1 Reconstructed network structure

[0043]

[0044]

[0045] In order to make the convolutional autoencoder have good reconstruction performance of the original image, a suitable loss function is also particularly important. This paper chooses L2 Loss as the reconstruction loss. L2 Loss calculates the residual between the output image and the original image at each point and sums the residuals of all points. The specific form of L2 Loss is:

[0046]

[0047] Among them, x(r,c) and where h and w are the grayscale values ​​of the original image and the reconstructed image at point (r, c), respectively. Using L2 Loss as the loss function, the network will minimize the difference between the original image and the reconstructed image to achieve a good reconstruction effect.

[0048] The present invention uses the error back propagation method based on gradient descent to train the reconstruction network. Specifically, all the parameters to be learned in the model are randomly initialized, and then a small batch of sample data {(x i ,yi),i∈b k}, where b k is the number of samples in the kth batch. Input data {x i}After the convolutional autoencoder forward propagation, the output value of the batch model is obtained According to the output value With the expected output value {yi} can calculate the training error {e i Based on the error backpropagation method, the gradient value of the parameters to be learned in each layer of the autoencoder network can be calculated. After obtaining the gradient value, the gradient optimization method can be used to perform a gradient update on all the parameters to be learned. The Adam optimizer with better working performance is used in this invention. The above training process is repeated until the error on the validation set no longer decreases in several consecutive iterations, which means that the network has converged and the network training is stopped.

[0049] Step 3: Based on the formation principle of defects and actual conditions, summarize the property rules of industrial product surface defects such as size, grayscale, shape, and flatness, and design industrial product surface defect rules based on these rules:

[0050] Rules are the accumulated experience and knowledge gained by experts through long-term production operations. When locating surface defects on industrial products, experts use this experience to identify patterns in the size, grayscale, shape, and flatness of defect areas, such as "defects often have low grayscale and small area," and thus develop defect rules. To ensure consistent rule expression, this paper uses the "if..., then..." structure to describe rules, i.e., "If this area has low grayscale and small area, then it is a defect."

[0051] Natural language often exhibits ambiguity. To better quantify rules, this paper designs a rule representation method based on fuzzy logic. Specifically, each attribute, such as "grayscale" or "area," can be considered a linguistic variable. A linguistic variable is a word or sentence in natural language. Its value is not a number, but a fuzzy set expressed in fuzzy language. Each linguistic variable is represented by a five-tuple, as shown in Table 2:

[0052] Table 2 Language variable quintuple

[0053] symbol significance example x variable name Grayscale T(x) A collection of language variables [Small, medium, large] G grammar rules / U Domain [0,+∞] M Semantic rules (membership functions) /

[0054] Define the membership function of each language value on the domain. For example, for "low grayscale", the following membership function can be defined:

[0055]

[0056] Where g represents the ratio of the grayscale of the region to the surrounding grayscale.

[0057] A rule may describe several attributes. The present invention uses a "maximum-multiplication" synthesis operator to comprehensively consider the descriptions of various attributes in a rule, thereby obtaining the rule score of the region under the rule.

[0058] For example, there is a rule: if the grayscale is low, the area is small, and the aspect ratio is small, then it is a defect. Then the rule score rs(ROI) of the region ROI (x(r, c)) to which the pixel x(r, c) belongs can be calculated:

[0059] rs(ROI)=μ 小 (g)*μ 小 (a)*μ 小 (m)

[0060] Among them, μ 小 (g) is the membership degree of the grayscale relative value g (the ratio of the center grayscale to the surrounding grayscale) to “small grayscale”, μ 小 (a) is the degree of membership of area a to “small area”, μ 小 (m) is the membership degree of aspect ratio m to “small aspect ratio”. Assuming there are n rules, we can get n different rule scores rs1, rs2, ..., rs n ,Finally, the rule score of this region is the disjunction of all rule scores:

[0061] rs=V i rs i

[0062] Step 4: Calculate the reconstructed residual image from the defect image and the pre-trained model. Based on the defect rule, a threshold selection mechanism is designed to divide the defect area from the reconstructed residual image to obtain a pseudo label:

[0063] Based on the pre-trained model, this paper uses defect images and rules to fine-tune the network, further improving network performance. Defect images and rules can help obtain information about the location and shape of defects, thereby helping the network learn defect patterns. The following describes the specific approach.

[0064] After the network is trained with normal samples, the reconstructed residual image obtained by the network output and the original image will produce different values ​​in the normal area and the defect area. In order to obtain the location of the defect area, it is necessary to select a suitable threshold to delineate the defect area. Due to the large differences between different defects, the threshold for each defect area should be different. The selected threshold should make 1) the rule score of the segmented area high; 2) the area of ​​the segmented area retained is large. A high rule score means that the possibility of the segmented area being a defect is high. A large area of ​​the segmented area retained means that fewer defect points are missed. The practice of the present invention is to traverse the candidate threshold thrd from low to high for each defect sample. ± , calculate the rule score of each segmented area according to the rule:

[0065]

[0066] μ res=mean(x res )

[0067] σ res =std(x res )

[0068] thrd + ∈[μ+σ,μ+7σ]

[0069] thrd - ∈[μ-7σ,μ-σ]

[0070] in, is the value of the reconstructed image at point (r, c), x(r, c) is the value of the original image at point (r, c), μ res and σ res Represent the mean and standard deviation of the reconstructed image respectively. When the rule score of the segmented area exceeds a certain threshold, it is considered that the area is likely to be a defect area and is included in the pseudo label. As the threshold continues to increase, the rule score may increase while the segmented area continues to decrease. The present invention designs a relative threshold score (Relative Threshold Score, RTS) to measure the quality of the threshold. In order to avoid the situation where the area is large but the rule score is very low, the present invention sets a benchmark threshold score thrd b (x), which is the threshold score when the rule score is greater than or equal to 0.8 for the first time:

[0071]

[0072] If the rule score of a region is low from beginning to end, then this region should not be considered as a defect region, and the relative threshold score is 0. As the threshold increases, the product of the rule score and the area constitutes the threshold score RS(ROI(x; thrd(x)))Area(ROI(x; thrd(x))) for each threshold. The threshold score is related to the baseline threshold score RS(ROI(x; thrd(x))). b (x)))Area(ROI(x;thrd b The ratio of (x))) is the relative threshold score. The relative threshold score measures the quality of each threshold. The present invention selects the threshold with the highest relative threshold score to demarcate the defect area. In addition, a connected area may contain multiple defect areas. As the threshold increases, different defect areas may be separated. After separation, each sub-area will determine the optimal threshold thrd separately. * (x). The process of selecting the threshold can be expressed by the following formula:

[0073]

[0074]

[0075] Where ROI(x; thrd(x)) is the connected region after pixel x is divided by threshold thrd(x), thrdb(x) is the baseline threshold of pixel x, and RTS(ROI(x; thrd(x))) is the relative threshold score of the region. After determining the optimal threshold, the pseudo label can be obtained:

[0076] A={x|(RTS(thrd * (x))>0)∩(|x res (r,c)|≥|thrd * (x)|)}

[0077] PL=1 A (x)

[0078] Among them, A is the set of all points that meet the pseudo label requirements, 1 A (x) is the indicator function. Pseudo-labels provide information about the location and shape of defective areas. Proper use of this information can help improve network performance.

[0079] Step 5: Design a loss function based on the principle that the reconstruction error of normal pixels is small and the reconstruction error of defective pixels is large, and fine-tune the pre-trained model:

[0080] After obtaining this round of pseudo-labels, the network is fine-tuned using these pseudo-labels. Specifically, the present invention uses gradient descent for normal pixels in the pseudo-labels and gradient ascent for defective pixels. The designed loss function guides the network to have a small reconstruction error for normal pixels and a large reconstruction error for defective pixels. The specific form of the loss function is:

[0081]

[0082] Among them, N n is the number of normal pixels in the pseudo label, N an is the number of defective pixels in the pseudo label, c an is a hyperparameter. When the network training epoch reaches a certain number, the current round of training is stopped.

[0083] The trained network in this round will be used to obtain pseudo labels for the next round. On the one hand, the network is continuously optimized through pseudo labels, and on the other hand, a better network helps to obtain better pseudo labels. The two promote each other and gradually improve network performance.

[0084] Step 6: Post-process the reconstructed residual image using the guided filtering method to smooth the high residual areas caused by noise.

[0085] The present invention uses guided filtering to post-process the reconstructed residual image to improve the prediction results. Guided filtering is cited from [K.He, J.Sun and X.Tang, "Guided Image Filtering," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 6, pp. 1397-1409, June 2013, doi: 10.1109 / TPAMI.2012.213] and is an edge-preserving bilateral filter. Its core formula is:

[0086]

[0087] q i =p i -n i

[0088] Among them, p is the input image, I is the guide image, q is the output image, ω k is the local window number, a k and b k is the parameter to be fitted. In the present invention, the input image is the reconstructed residual image, and the guide image is the original image. At the same time, the guided filter considers that the input image p is obtained by adding undesired noise or texture n to q. k and b k When , the gap between p and q should be kept as small as possible while maintaining the local linear model. Ridge regression with regularization is used:

[0089]

[0090] The final solution is:

[0091]

[0092]

[0093]

[0094] Among them, μ k and For the guidance image I in the window ω k The mean and variance in |ω| is the window ω k The number of pixels in For the input image p in window ω kThe present invention uses the original image and the reconstructed residual map as input for guided filtering. After guided filtering, the final reconstructed residual map is output. Guided filtering can smooth and denoise the background while maintaining the gradient. Using guided filtering to process the reconstructed residual map can reduce the impact of noise, thereby achieving better positioning results and obtaining a better reconstructed residual map.

[0095] Example

[0096] The following describes the experimental process of this invention using a magnetic tile dataset. After acquiring the data, we scaled all images to 256*256 and normalized them to a grayscale range of [0, 1]. We used 825 normal images and 43 defective images as the training set, and the remaining 373 images as the test set. In the training set, defective images accounted for approximately 5% of the total samples.

[0097] After dividing the training set, we perform data augmentation on the normal samples in the training set, including horizontal flipping, vertical flipping, and mixup. The weighting parameter λ of the two samples in mixup is set to 0.5.

[0098] For the reconstruction network, we designed a convolutional autoencoder suitable for processing image data as the basic network architecture. The convolutional autoencoder only contains convolutional layers and LeakyReLU activation layers, and has fewer network parameters.

[0099] During the network pre-training phase, we set the batch size of network training to 50 and the number of training epochs to 200. The learning rate of the Adam optimizer was set to 1e-4, the weight decay rate was set to 2e-5, and the rest of the parameters were set to default.

[0100] During the network fine-tuning phase, we selected the optimal threshold for each region according to the rules and obtained the pseudo-label. When using pseudo-labels to fine-tune the network, we set the parameter c an The network is fine-tuned for 200 epochs in each round, and for a total of 3 rounds.

[0101] After fine-tuning the network, we use guided filtering to post-process the reconstructed residual image. The input image is the reconstructed residual image, the guided image is the original image, the guided filtering window size is 16*16, and the regularization penalty coefficient ∈ is 0.01.

[0102] During the testing phase, we used test data to evaluate network performance and compared it with the benchmark. We used common evaluation metrics in anomaly localization: AUROC and PRO to evaluate network performance.

[0103] We also used the method of the present invention to test and evaluate the performance of the other two datasets, and finally obtained Table 3 and Table 4.

[0104] From the experimental results in Tables 3 and 4, we can see that: (1) the method proposed in the present invention has a significant improvement over the baseline and has relatively good performance; (2) the method proposed in the present invention has obvious improvements over the baseline on different data sets and has strong versatility; (3) the present invention achieves good results by using only a few abnormal samples and weak labels, and has high application value in real industrial scenarios.

[0105] Table 3 AUROC evaluation index

[0106] Dataset Magnetic tile Mvtec AD KolektorSDD2 Benchmark 0.876 0.873 0.852 Method of the present invention 0.942 0.952 0.921

[0107] Table 4 PRO evaluation indicators

[0108] Dataset Magnetic tile Mvtec AD KolektorSDD2 Benchmark 0.841 0.750 0.798 Method of the present invention 0.910 0.880 0.886

Claims

1. A method for locating industrial product surface defects that integrates learning and knowledge, characterized by: It includes the following steps: Step 1: Obtain industrial product surface images and perform preprocessing and data enhancement; Step 2: Design an industrial product surface image reconstruction network and its loss function, and use defect-free industrial product surface images to train the network to obtain a pre-trained model; the reconstruction network is a convolutional autoencoder, including an encoder and a decoder; the encoder includes a convolutional layer and a LeakyReLU activation layer. The encoder extracts semantic information from the image, then extracts features with representational significance from the semantic information, and outputs a feature vector. The decoder has a symmetrical structure with the encoder and restores the image based on the feature vector; Step 3: Summarize the laws of surface defects of industrial products based on the formation principle and actual situation of defects, and design defect rules based on the laws; the defect rules adopt a rule representation method based on fuzzy logic, pre-set various attributes of defects, calculate the rule score of each attribute, calculate the defect rule to describe several attributes, and for a rule, use the "maximum-multiplication" synthesis operator to comprehensively consider the rule scores of each attribute to obtain the rule score under the rule. Assuming there are n rules, n different rule scores rs1, rs2, ..., rs are obtained. n , and finally the rule score of the region is the disjunction of all rule scores: rs=∨ i rs i Step 4: Calculate the reconstructed residual image based on the defect image and the pre-trained model, and design a threshold selection mechanism based on the defect rule to divide the defect area from the reconstructed residual image to obtain a pseudo label; the threshold selection mechanism is to traverse the candidate thresholds from low to high for each defect sample, and calculate the rule score of each segmented area according to the rule. When the rule score of the segmented area exceeds a certain threshold, it is considered that the area is likely to be a defect area, and a relative threshold score is set to measure the quality of the threshold. For all possible defect areas, the defect area determined by the threshold with the highest relative threshold score is the final defect area; the relative threshold score is the ratio of the threshold score to the benchmark threshold score, wherein the threshold score is the product of the rule score and the area of ​​the defect area, and the benchmark threshold score is the threshold score when the rule score is greater than or equal to 0.8 for the first time; Step 5: Based on the principle that the reconstruction error of normal pixels is small and the reconstruction error of defective pixels is large, a loss function is designed to fine-tune the pre-trained model. The network is reconstructed using the pseudo-labels from the previous round. When the network training epoch reaches the preset number, the current round of training is stopped and the pseudo-labels obtained in this round are used for the next round of fine-tuning and reconstructing the network. This is done until the validation set error is less than the preset threshold. A defect location model is obtained, and the surface image of the industrial product to be identified is input into the defect location model to obtain a reconstructed residual image, thereby locating the defect.

2. The method for locating industrial product surface defects by integrating learning and knowledge according to claim 1 is characterized in that: The preprocessing process described in step 1 is to unify the image size and normalize the grayscale value of the image; the data enhancement includes: image flipping, image rotation, and mixup. Mixup is to perform weighted averaging on two random defect-free industrial product surface images to obtain a new normal sample.

3. The method for locating industrial product surface defects by integrating learning and knowledge according to claim 1 is characterized in that: The loss function in step 2 is L2Loss, which calculates the residual between the output image and the original image at each point and sums the residuals of all points. The specific form is: Among them, x(r,c) and are the grayscale values ​​of the original image and the reconstructed image at point (r, c), h and w are the height and width of the image, respectively. The network training method is the error back propagation method.

4. The method for locating industrial product surface defects by integrating learning and knowledge according to claim 1, characterized in that: A connected area may contain multiple defective areas. As the threshold increases, different defective areas may be separated. After separation, the optimal threshold is determined separately for each sub-area.

5. The method for locating industrial product surface defects by integrating learning and knowledge according to claim 1, characterized in that: The specific form of the loss function in step 5 is: Among them, x(r,c) is the grayscale value of the image at point (r,c), l2(x(r,c)) is the L2Loss of the image at point (r,c), h and w are the height and width of the image respectively, N n is the number of normal pixels in the pseudo label, N an is the number of defective pixels in the pseudo label, PL is the pseudo label, c an is a hyperparameter, L N+AN+R (x) is the loss function.

6. The method for locating industrial product surface defects by integrating learning and knowledge according to claim 1, characterized in that: The reconstructed residual image is post-processed using a guided filtering method, where the input image of the guided filtering is the reconstructed residual image and the guided image is the original image.

Images