Complex textile surface defect detection method based on semi-supervision

Abstract

The invention discloses a semi-supervision-based complex textile surface defect detection method, and belongs to the technical field of complex industrial product surface defect detection, and the method comprises the following steps: S1, taking U-Net as a frame, and completing a semantic segmentation task in a training stage by means of a simulated abnormal sample and a double-input image; s2, difference information between the two input images is calculated through an image-level feature difference calculation module PDFM, and spliced image-level features FDi are obtained; s3, performing feature fusion on the spliced image-level features FDi through a multi-scale feature fusion module MSFFA; and S4, finally, a result in the MSFFA module is decoded through an up-sampling module improved by U-Net, and a prediction result is finally obtained. According to the defect detection method for the complex textile surface based on semi-supervision, the defects on the complex textile surface can be accurately positioned, the model architecture is simple, the generated overhead is low, and the defect detection method is suitable for industrial production.

Description

Technical Field

[0001] The invention relates to the technical field of complex industrial product surface defect detection, and in particular to a complex textile surface defect detection method based on semi-supervision. Background Art

[0002] With the rapid development of manufacturing and industrial production, surface defect detection plays a vital role in ensuring product quality and improving production efficiency. In many fields with high precision and high quality requirements, such as electronic products, automobile manufacturing, aerospace, optical glass, etc., surface quality is directly related to the overall performance and market competitiveness of the product. Especially when it comes to some high-end textiles, their surface defects not only affect the appearance, but also directly affect the functionality and service life of the textiles. Textile surface defects mainly include broken yarns, stains, holes, etc. of the fabric. These defects are often difficult to effectively identify through manual inspection, especially in complex fabrics (such as various patterned fabrics), where the manifestations of defects are more diverse and difficult to detect. Traditional surface defect detection methods mainly rely on manual visual inspection or simple machine vision inspection. Manual visual inspection is inefficient and easily affected by human factors; existing image processing-based defect detection methods usually rely on traditional edge detection, texture analysis or color recognition technology. These methods may be effective for simple defect detection, but in the surface of complex patterns, due to the complexity of the pattern itself (such as repeated geometric shapes, non-uniform textures, illumination changes, etc.), existing technologies are difficult to accurately identify tiny defects or artifacts.

[0003] In order to overcome the above problems, in recent years, more and more research has begun to focus on the application of deep learning and artificial intelligence technology in surface defect detection. By using deep learning algorithms such as convolutional neural networks (CNN), it is possible to automatically extract features from complex patterns and then perform high-precision defect recognition. However, these methods still face the following challenges: (1) Data annotation is difficult. High-quality training data sets are the basis of deep learning models. Especially in surface defect detection of complex patterns, defect samples are often scarce and require a lot of manual annotation, which increases the cost of data preparation. (2) Complexity of model training: Deep learning models require a lot of computing resources and time to train, and often require complex model tuning and parameter selection. (3) The problem of distinguishing different defects: For complex patterns, how to accurately distinguish various types of defects and changes in the pattern itself is still an unresolved problem.

[0004] Therefore, there is an urgent need for a new surface defect detection method that can effectively deal with the surface defect problems of complex textiles and has the advantages of high efficiency, accuracy and real-time performance. Summary of the invention

[0005] The purpose of the present invention is to provide a semi-supervised defect detection method for complex textile surfaces, which can accurately locate defects on the surface of complex textiles. The model architecture is simple and the overhead is low, and it is suitable for industrial production.

[0006] To achieve the above object, the present invention provides a method for defect detection of complex textile surfaces based on semi-supervision, comprising the following process:

[0007] S1, using U-Net as the framework, completes the semantic segmentation task with the help of simulated abnormal samples and dual input images during the training phase;

[0008] S2, calculating the difference information between the two input images through the image-level feature difference calculation module PDFM, and obtaining the spliced ​​image-level feature FDi;

[0009] S3, the spliced ​​image-level features FDi are fused through the multi-scale feature fusion module MSFFA;

[0010] S4. The result in the MSFFA module is decoded through the upsampling module improved by U-Net, and finally the prediction result is obtained.

[0011] Preferably, the method for simulating anomalies in S1 is as follows:

[0012] S1.1. First, the structural anomaly is simulated to generate a two-dimensional Perlin noise P, which is then binarized by the threshold T to obtain the mask M. p ;

[0013] S1.2, then the texture anomaly simulation, the mask image M obtained in S1.1 p and the noisy image I n Performing element-wise multiplication yields p Definition of I n In this process, a transparency factor is introduced to balance the fusion of the original image and the noisy image, so that the simulated abnormal pattern is closer to the real abnormality;

[0014] I′ n =σ(M p ⊙I n );

[0015] Where σ is the transparency factor, and the noisy foreground image I' is obtained n ;

[0016] S1.3, the mask image M p Convert to Then in Perform element-wise multiplication on the original image I to obtain image I', and then obtain the simulated abnormal image I according to the following formula A :

[0017]

[0018] Thus, two input images are obtained, one is the input positive sample, and the other is the simulated abnormal image; the pre-trained Resnet18 is used as the encoder to extract the first layer, the middle layer and the last layer of image-level features in turn.

[0019] Preferably, the PDFM module in S2 comprises the following steps:

[0020] S2.1. First, the intermediate layer features of the two input data are passed through the CoordAttention module layer by layer.

[0021] S2.2, then calculating the absolute difference between the feature maps of the lth layer for the output features in S2.1, and obtaining the difference features between the two;

[0022] FD = |f1_ii[l]-f2_ii[l]|;

[0023] Among them, f1_ii[l] is the intermediate layer feature of the positive sample after l processing, f2_ii[l] is the intermediate layer feature of the sample to be detected after l processing, and l is the feature level;

[0024] S2.3, finally, the output of the image to be detected in the attention module is spliced ​​with the difference information calculated in S2.2 to obtain the spliced ​​image-level feature FD i .

[0025] Preferably, the steps of the multi-scale feature fusion module MSFFA in S3 are as follows:

[0026] S3.1. First, FD i After passing through a 3*3 convolution layer respectively, the coordinate attention mechanism CoordAtt is called to generate an attention map to emphasize the important areas in the input feature map; the convolution-processed feature map and the attention map are element-wise multiplied, so that the result of the attention mechanism is applied to the convolution feature map, and finally a convolution sequence is used to further process the feature map weighted by attention;

[0027] S3.2, then upsampling the feature maps of the three feature dimensions obtained in S3.1 and concatenating them with adjacent feature maps to combine the fine-grained low-level features with the high-level semantic information;

[0028] S3.3 Finally, use FD iExtract three spatial attention maps M3, M2, and M1 to strengthen the guessing of the best difference information of the abnormal area;

[0029]

[0030] Among them, i represents the summation range, C3 is The number of channels in C2 is The number of channels in C1 is The number of channels in ; is the upsampled feature map of M2, It is the upsampled feature map of M1.

[0031] Preferably, in S4, L1 loss and Focal loss are used to ensure the similarity of all pixels in the image space, and the true value S of the abnormal area in the simulated image and the simulated predicted value are minimized by the following formula: The L1 loss between l1 and Focal loss L f :

[0032]

[0033] L f =-α t (p t ) γ log(1-p t );

[0034] Among them, when the true value of the pixel in S is 1, p t =p; when the true value of the pixel in S is 0, p t =1-p; α t and γ are hyperparameters that control the degree of weighting;

[0035] Finally, these two constraints are combined into an objective function as follows:

[0036] L tot =λ l1 L l1 +λ f L f ;

[0037] Among them, λ l1 and λ f is a hyperparameter.

[0038] Preferably, during the reasoning process, the abnormal area in the image is positioned from coarse to fine by a registration method, wherein the registration method is as follows:

[0039] Feature detection: First, use the feature detection algorithm to detect the feature points kp1 and kp2 of the two images and their corresponding descriptors des1 and des2;

[0040] Feature matching: Use the feature matching algorithm to perform k nearest neighbor matching to find the best match for each descriptor, and finally get a list of matches containing two nearest neighbor matches;

[0041] Then filter out good matches good_matches, and only when the distance of the first match is less than 0.7 times the distance of the second match, it is considered a good match;

[0042] Calculate the homography matrix: Extract the key point coordinates of the source image and the target image from good_matches. For each pair of corresponding points, construct an equation to describe the transformation from the source image to the target image. Assume that the homography matrix is ​​a 3*3 matrix H, and its form is as follows:

[0043]

[0044] Where H is the homography matrix to be calculated:

[0045]

[0046] Then we get the following form, that is, we get the coordinates from the source image coordinates (x1, y1) to the target image coordinates (x2, y2):

[0047] x2=h 11 x1+h 12 y1+h 13 ;

[0048] y2=h 21 x1+h 22 y1+h 23 ;

[0049] 1 = h 31 x1+h 32 y1+h 33 ;

[0050] Perspective transformation: Perform perspective transformation on the source image according to the calculated homography matrix H to align it with the target image; the transformed image img1Reg will have the same size as the target image, that is, the final registered image is obtained.

[0051] Therefore, the present invention adopts the above-mentioned semi-supervised defect detection method for complex textile surfaces, which can accurately locate the defects on the surface of complex textiles. The model architecture is simple and the overhead generated is low, which is suitable for industrial production.

[0052] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a schematic diagram of the registration process of an embodiment of a method for defect detection on a complex textile surface based on semi-supervision according to the present invention;

[0054] Figure 2 It is a schematic diagram of abnormal simulation of an embodiment of a defect detection method for a complex textile surface based on semi-supervision according to the present invention;

[0055] Figure 3 is a PDFM schematic diagram of an embodiment of a defect detection method for a complex textile surface based on semi-supervision according to the present invention;

[0056] Figure 4 Schematic diagram of MSFFA of an embodiment of a defect detection method for complex textile surface based on semi-supervision according to the present invention;

[0057] Figure 5 It is a schematic diagram of the overall framework of an embodiment of a method for defect detection on a complex textile surface based on semi-supervision according to the present invention;

[0058] Figure 6 It is a schematic diagram of the experimental effect of an embodiment of a defect detection method for complex textile surfaces based on semi-supervision of the present invention. DETAILED DESCRIPTION

[0059] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0060] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.

[0061] Embodiment 1

[0062] The present invention is an improvement based on the MemSeg network. The overall framework is as follows: Figure 5 As shown, the specific process is as follows:

[0063] First, prepare the collected positive sample data set for training, and then prepare a small number of defect samples for testing. Figure 1 As shown in the figure, the training set mainly includes positive sample images and images to be tested. The images to be tested are defects generated by the abnormal simulation module. The pre-trained ResNet-18 is used to extract the first layer, middle layer and last layer image-level features of the two images. Then the PDFM module is used to calculate the difference information of the middle layer features of the two images. The PDFM schematic diagram is shown in Figure 3The MSFFA module is used to fuse these information. The MSFFA schematic diagram is shown in Figure 4 As shown in Figure 3, the improved U-Net module is used to upsample the image to obtain the final prediction result, and the dual loss function is used to continuously optimize the model performance.

[0064] Specifically, if Figure 2 As shown, in the abnormal simulation module, a two-dimensional Perlin noise P is first generated, and then the mask M is obtained by binarization through the threshold T. p , mask image M p and the noisy image I n Performing element-wise multiplication yields p Definition of I n Then the mask image M p Convert to Then in Perform element-wise multiplication on the original image I to obtain p Definition of I n Then the mask image M p Convert to Then in Perform element-wise multiplication on the original image I to obtain image I', and then Then we can get the simulated abnormal image I A .

[0065] After obtaining the simulated image, the dual input images can be sent to the encoder, and the pre-trained ResNet-18 is used to extract the first layer, middle layer and last layer features of the two input images respectively. Then the middle layer features are sent to the PDFM module to calculate the feature difference, and the difference information is spliced ​​with the features of the image to be detected to obtain the spliced ​​image-level feature FD i .

[0066] If you use FD directly i Decoding will lead to feature redundancy, increase the computational scale, and reduce the inference speed. i The feature map is sent to the multi-scale feature fusion module (MSFFA) for feature fusion and spatial attention mechanism processing and generates an attention map to emphasize the important areas in the input feature map. The feature map processed by convolution and the attention map are multiplied element by element, so that the result of the attention mechanism is applied to the convolution feature map, thereby enhancing important features and suppressing unimportant features. After that, the feature map weighted by attention is further processed through a convolution sequence to extract higher-level features. Finally, it is input to the improved U-Net module for upsampling to obtain the final prediction result. In addition, a dual loss function is used to ensure that the model prediction value is as close to the true value as possible, and the model is continuously optimized.

[0067] In the inference stage, the dual input images are first registered to obtain two similar images of the same size. The size of the image to be tested does not have to be 256*256, it can be a larger image, and it will be scaled to meet the model requirements before entering the model. In this way, in actual industrial generation, the amount of image processing can be reduced, and there is no need to crop the larger image to be tested into several 256*256 small images, saving image processing time;

[0068] like Figure 6 As shown, Figure 6 The figure is a schematic diagram of the experimental effect of the present invention. After testing, this registration method can effectively register a variety of complex textile patterns, and it only takes about 1 second to register a source image of 3300*1600 with a target image of 674*674. If the GPU occupancy is low, it can even take less time.

[0069] Therefore, the present invention adopts the above-mentioned semi-supervised defect detection method for complex textile surfaces, which can accurately locate the defects on the surface of complex textiles. The model architecture is simple and the overhead generated is low, which is suitable for industrial production.

[0070] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for defect detection of complex textile surfaces based on semi-supervision, characterized in that: The process includes: S1, using U-Net as the framework, completes the semantic segmentation task with the help of simulated abnormal samples and dual input images during the training phase; S2, calculate the difference information between the two input images through the image-level feature difference calculation module PDFM to obtain the spliced ​​image-level feature FD i ; S3, splicing to obtain image-level features FD i Feature fusion is performed through the multi-scale feature fusion module MSFFA; S4. The result in the MSFFA module is decoded through the upsampling module improved by U-Net, and finally the prediction result is obtained.

2. A method for defect detection of complex textile surfaces based on semi-supervision according to claim 1, characterized in that: The method for simulating anomalies in S1 is as follows: S1.

1. First, the structural anomaly is simulated to generate a two-dimensional Perlin noise P, which is then binarized by the threshold T to obtain the mask M. p ; S1.2, then the texture anomaly simulation, the mask image M obtained in S1.1 p and the noisy image I n Performing element-wise multiplication yields p Definition of I n The region of interest in In this process, a transparency factor is introduced to balance the fusion of the original image and the noisy image, so that the simulated abnormal pattern is closer to the real abnormality; I' n =σ(M p ⊙I n ); Where σ is the transparency factor, and the noisy foreground image I' is obtained n ; S1.3, the mask image M p Convert to Then in Perform element-wise multiplication on the original image I to obtain image I', and then obtain the simulated abnormal image I according to the following formula A : Thus, two input images are obtained, one is the input positive sample, and the other is the simulated abnormal image; the pre-trained Resnet18 is used as the encoder to extract the first layer, the middle layer and the last layer of image-level features in turn.

3. The method for defect detection of complex textile surfaces based on semi-supervision according to claim 1, characterized in that: The PDFM module in S2 includes the following steps: S2.

1. First, the intermediate layer features of the two input data are passed through the CoordAttention module layer by layer. S2.2, then calculating the absolute difference between the feature maps of the lth layer for the output features in S2.1, and obtaining the difference features between the two; FD = |f1_ii[l]-f2_ii[l]|; Among them, f1_ii[l] is the intermediate layer feature of the positive sample after l processing, f2_ii[l] is the intermediate layer feature of the sample to be detected after l processing, and l is the feature level; S2.3, finally, the output of the image to be detected in the attention module is spliced ​​with the difference information calculated in S2.2 to obtain the spliced ​​image-level feature FD i .

4. The method for defect detection of complex textile surfaces based on semi-supervision according to claim 3 is characterized in that: The steps of the multi-scale feature fusion module MSFFA in S3 are as follows: S3.

1. First, FD i After passing through a 3*3 convolution layer respectively, the coordinate attention mechanism CoordAtt is called to generate an attention map to emphasize the important areas in the input feature map; the convolution-processed feature map and the attention map are element-wise multiplied, so that the result of the attention mechanism is applied to the convolution feature map, and finally a convolution sequence is used to further process the feature map weighted by attention; S3.2, then upsampling the feature maps of the three feature dimensions obtained in S3.1 and concatenating them with adjacent feature maps to combine the fine-grained low-level features with the high-level semantic information; S3.3 Finally, use FD i Extract three spatial attention maps M3, M2, and M1 to strengthen the guess of the best difference information of the abnormal area; Among them, i represents the summation range, C3 is The number of channels in C2 is The number of channels in C1 is The number of channels in ; is the upsampled feature map of M2, It is the upsampled feature map of M1.

5. The method for defect detection of complex textile surfaces based on semi-supervision according to claim 1, characterized in that: In S4, L1 loss and Focal loss are used to ensure the similarity of all pixels in the image space, and the true value S of the abnormal area in the simulated image and the simulated predicted value are minimized by the following formula: The L1 loss between l1 and Focal loss L f : L f =-a t (p t ) γ log(1-p t ); Among them, when the true value of the pixel in S is 1, p t =p; when the true value of the pixel in S is 0, p t =1-p; α t and γ are hyperparameters that control the degree of weighting; Finally, these two constraints are combined into an objective function as follows: L tot =λ l1 L l1 +λ f L f ; Among them, λ l1 and λ f is a hyperparameter.

6. The method for defect detection of complex textile surfaces based on semi-supervision according to claim 4, characterized in that: During the inference process, the abnormal area in the image is positioned from coarse to fine by a registration method, wherein the registration method is as follows: Feature detection: First, use the feature detection algorithm to detect the feature points kp1 and kp2 of the two images and their corresponding descriptors des1 and des2; Feature matching: Use the feature matching algorithm to perform k nearest neighbor matching to find the best match for each descriptor, and finally get a list of matches containing two nearest neighbor matches; Then filter out good matches good_matches, and only when the distance of the first match is less than 0.7 times the distance of the second match, it is considered a good match; Calculate the homography matrix: Extract the key point coordinates of the source image and the target image from good_matches. For each pair of corresponding points, construct an equation to describe the transformation from the source image to the target image. Assume that the homography matrix is ​​a 3*3 matrix H, and its form is as follows: Where H is the homography matrix to be calculated: Then we get the following form, that is, we get the coordinates from the source image coordinates (x1, y1) to the target image coordinates (x2, y2): x2=h 11 x1+h 12 y1+h 13 ; y2=h 21 x1+h 22 y1+h 23 ; 1=h 31 x1+h 32 y1+h 33 ; Perspective transformation: Perform perspective transformation on the source image according to the calculated homography matrix H to align it with the target image; the transformed image img1Reg will have the same size as the target image, that is, the final registered image is obtained.

Images