Textile Defect Recognition Method Combining Cross-Domain Migration and Anomaly Detection

Abstract

The present invention relates to a textile defect recognition method that integrates cross-domain migration and anomaly detection. A semi-supervised learning mechanism is introduced to deeply mine the feature distributions of the source domain and the target domain, and domain migration is achieved by means of production confrontation, enabling accurate recognition even when the fabric style changes. Aiming at the small-scale surface defects of fabrics, an unsupervised anomaly detection method based on multi-level feature matching is designed. By using the knowledge distillation and multi-level feature matching mechanisms, the model is made more targeted, and fabric defects can be accurately detected without annotation, realizing low-cost and high-precision recognition of fabrics.

Description

Technical Field

[0001] The present invention relates to the fields of deep learning, domain adaptation, anomaly detection, and textile defect recognition, and particularly to a textile defect recognition method integrating cross-domain transfer and anomaly detection. Background Art

[0002] Textiles are indispensable materials in both people's daily lives and industrial fields, and their demand is increasing day by day. However, during the production process, due to objective factors such as weaving thread materials, machine failures, manual operation errors, and production environments, various defects will form on the surface of textiles, making quality inspection a top priority. In industrial sites, the manufacturing of fabrics is linked together, and earlier detection of defects can reduce losses to a greater extent. Given the many drawbacks of traditional detection methods, the automatic classification of fabric defects has become an industry trend.

[0003] With the development of hardware devices, deep learning methods have been widely applied to fabric defect recognition in recent years, promoting related research and applications in this industry, and many defect detection schemes based on deep convolutional neural networks have emerged. It can automatically extract features from fabric images, and various general supervised deep learning algorithms such as Faster R-CNN, SSD, and YOLO have been widely studied and applied in this field. However, existing fabric defects have the characteristic of varying scales. For fabric defects of different sizes and scales, the difficulty of detection is different. If existing deep learning methods are directly used for mixed-scale detection, not only is the requirement for the algorithm high, but it is also difficult to achieve an ideal recognition effect in terms of detection efficiency and accuracy, and it cannot meet the real-time production requirements.

[0004] Moreover, the above-mentioned supervised deep learning models have the characteristic of data hunger. During the model training process, a large amount of defect data needs to be fed, while the acquisition of defect data in the actual production process of textiles is much more difficult than normal data. In addition, the models trained by these deep learning methods are fixed. When the style of textiles changes, the old models will no longer be applicable to the new style. If the migration of textile styles is to be completed, data must be collected again, the new and old styles must be integrated, and the model must be retrained, and the collection and annotation of data are time-consuming and inefficient.

[0005] Therefore, how to achieve high-precision recognition of fabric defects for problems such as the small amount of textile defect data, high annotation costs, varying defect scales, and diverse defect styles is an urgent problem to be solved by the present invention. Summary of the Invention

[0006] The object of the present invention is to solve the problem of how to complete fabric defect recognition in the production process of textiles with the lowest cost and the highest efficiency. In the actual production process of textiles, only fabric defects need to be identified and normal fabrics are retained. Since large-scale defect features are more obvious and easier to be recognized by the model, according to this idea, a textile defect recognition method integrating cross-domain migration and anomaly detection is provided, which can achieve high-precision recognition of multiple scales and various styles on the premise of only annotating a small amount of fabric data. First, the image is sent into the morphological defect recognition network to identify whether there are large-scale defects, and then it is sent into the surface defect detection network to detect small-scale defects. If it is first confirmed that there are large-scale defects, the small-scale defect detection can be skipped and the result can be directly output.

[0007] To achieve the above object, the technical solution of the present invention is: a textile defect recognition method integrating cross-domain migration and anomaly detection, comprising the following steps:

[0008] Step 1, preprocessing of fabric image data

[0009] Collect fabric images by machine as the original data set, process the initial data in the original data set, select a fixed style as the source domain, and integrate other styles as the target domain;

[0010] Step 2, establishing a fabric data image library

[0011] Divide the image set into normal images, large-scale morphological defect images and small-scale surface defect images, and annotate part of the data in the morphological defect images, and retain the marked information;

[0012] Step 3, recognition of morphological defects of fabrics

[0013] For large-scale morphological defects in fabrics, an improved semi-supervised cross-domain neural network SSDANN is proposed, which consists of a feature extractor, a label predictor and a domain discriminator; the feature extractor realizes the mapping between different style data through the source domain and the labeled target domain; the domain discriminator realizes the migration of data from the source domain to the target domain through unsupervised learning.

[0014] During the training process of SSDANN, the source domain and target domain data are respectively input into the feature extractor to extract the deep features of the fabric, and then the feature information is sent into two branches: the label predictor G and the domain discriminator D; the source domain data is subjected to supervised learning and the label predictor is optimized; the source domain data and the unlabeled target domain data are input into the domain discriminator for unsupervised learning to confuse the feature information between the source domain and the target domain.

[0015] Step 4, detection of surface defects of fabrics

[0016] Aiming at the small-scale surface defects in fabrics, an anomaly detection method that does not require defect data for training is proposed. Firstly, in view of the domain differences between general fields and fabric fields, the knowledge distillation technology is used to enhance the deep model's ability to express fabric image features. Secondly, a multi-level feature extraction method is designed to construct a normal data feature memory unit, and to generate and store the multi-level implicit features of normal fabric samples. Finally, a multi-level feature matching mechanism based on the memory unit is proposed, and the anomaly score of the sample to be tested is calculated through the nearest neighbor retrieval algorithm to judge the fabric state.

[0017] Compared with the prior art, the present invention has the following beneficial effects: for large-scale morphological defects of fabrics, the present invention proposes a multi-style fabric defect recognition network, introduces a semi-supervised learning mechanism, deeply mines the feature distribution of the source domain and the target domain, and uses the production confrontation method to achieve domain migration, so that the fabric style can still be accurately recognized when it changes. For small-scale surface defects of fabrics, an unsupervised anomaly detection method based on multi-level feature matching is designed, and the knowledge distillation and multi-level feature matching mechanism are used to make the model more targeted, and fabric defects can be accurately detected without labeling, so as to achieve low-cost and high-precision recognition of fabrics. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flowchart of multi-scale defect recognition for textiles according to the present invention.

[0019] Figure 2 This is the SSDANN network structure diagram

[0020] Figure 3 Schematic diagram of the knowledge distillation process.

[0021] Figure 4 Schematic diagram of the anomaly determination framework based on multi-level feature matching DETAILED DESCRIPTION

[0022] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings.

[0023] like Figure 1 In this example, a textile defect recognition method is proposed based on the combination of cross-domain research and anomaly detection. According to the different defect scales, a morphological defect recognition network and a surface defect recognition network are designed respectively.

[0024] The cross-domain neural network module based on large-scale defects divides the data into source domain and target domain. The source domain is a fixed style, and the target domain is an integration of other styles. This module can learn the data features between the two domains. The domain discriminator brings the potential connections between fabrics of different styles closer by generating adversarial methods, and the label predictor directly identifies textile defects.

[0025] Based on the multi-level feature matching network module for small-scale surface defects, when the output result of the cross-domain network is normal, it is then sent to this anomaly detection network for discrimination. The introduction of knowledge distillation and multi-level feature matching can enhance the feature extraction ability for small-scale defects, improve the accuracy, and finally judge the fabric defect state according to the relationship between the anomaly score output by the network and the threshold.

[0026] Figure 2 The SSDANN network structure diagram designed by the present invention for morphological defects in fabrics. The feature extractor consists of multiple convolutional and pooling layers, and its function is to map the data to a specific feature space, enabling the label predictor to distinguish fabric categories and the domain discriminator to confuse whether the fabric comes from the source domain or the target domain. Using sigm as the activation function among them, its output is:

[0027] G f (x; W, b) = sigm(Wx + b) (1)

[0028] In the above formula, x is the source domain and target domain samples, W is the weight factor, and b is the bias term. The labeled data in the source domain is sent into the feature extractor, and then the output part is sent into the label predictor. The label predictor will classify the source domain data and try to classify the correct label as much as possible. The output of the label predictor is the Softmax activation function, regarding x s as an example in the source domain, and y s as the label corresponding to x s , the output of the label predictor can be expressed as:

[0029] G p (G f (x s )); Z, k) = softmax(ZG f (x s )) + k) (2)

[0030] In the above formula, G f (x s ) represents the input of the label predictor and is also the output of the feature extractor. Z is the weight factor, and k is the bias term. Only the feature extractor and the label predictor are used in the model testing and actual use stages, and its loss function can be expressed as:

[0031]

[0032] In the above formula, G p (G f (x s )) is the output of the label predictor, and y s is the source domain sample x sThe corresponding label. The feature extractor and the label classifier form a feedforward neural network. For the fixed style of the source domain textiles, there is the following optimization objective function:

[0033]

[0034] In the above formula, n represents the number of source domain samples, represents the label prediction loss of the i-th sample, λ represents a regularization term parameter, and R(W,b) represents a regularizer, which plays a role in preventing the label prediction network from overfitting. The domain classifier is used to perform domain discrimination on the overall input data to determine whether the data comes from the source domain or the target domain, and can be understood as a binary classifier. As shown by the arrows in the figure, this is an unsupervised learning process. At this stage, only the unlabeled source domain data and target domain data are input to the feature extractor. After processing, it is input to the domain classifier for domain discrimination. The domain classifier uses the sigmoid activation function, and its output is expressed as:

[0035] G d (G f (x m );O,p)=sigm(O T G f (x m )+p) (5)

[0036] In the above formula, x m represents the integrated data of the source domain and the target domain, and O and p represent the weight and the bias term respectively. The domain classifier uses binary cross-entropy as the loss function, and its output is expressed as:

[0037]

[0038] In the above formula, d i is 0 or 1, representing the domain label of the i-th sample. The feature extractor needs to interfere with the domain discriminator as much as possible, preferably making it unable to distinguish whether d i comes from the source domain or the target domain, while the domain classifier tries its best to distinguish the source of the features. Through such adversarial training, a similar characteristic distribution can be achieved between the source domain data and the target domain data, sharing domain-invariant features. To simplify the training process, a gradient reversal layer (GRL) is introduced between the feature extractor and the domain classifier. In the forward propagation, GRL is regarded as an identity transformation, and in the backward propagation, the gradient is reversed, multiplied by a -α and passed to the previous layer. The forward and backward propagations of the gradient reversal layer are expressed as follows:

[0039]

[0040] Where \(I\) is an identity matrix. During the backpropagation process, the gradient of the domain classification loss is automatically inverted before backpropagating to the parameters of the feature extractor, implementing an adversarial loss similar to that of a GAN. The objective optimization function of the domain discriminator is:

[0041]

[0042] Textiles of different styles have varying style differences. It is difficult for the original network to extract features from textile data with very different styles, and the classification effect is only mediocre, making it difficult to meet the requirements of industrial production. Therefore, the SSDANN network proposed in this invention adds a semi-supervised learning mechanism, and only a small number of samples in the target domain need to be labeled for training. The target domain data \((x pt , y pt ) with annotation information is input into the feature extractor to extract features, and the output is as follows:

[0043] G f (x pt ; W, b) = ReLU(Wx pt +b) (9)

[0044] Then, similar to the source domain data above, the output of the feature extractor is input into the label predictor, and the output result is:

[0045] G p (G f (x pt ) ; O, p) = softmax(OG f (x pt )+p) (10)

[0046] The label classification loss of the labeled target domain data is:

[0047]

[0048] The calculation method of the loss function for the labeled target domain data is the same as (Equation 3). The network uses the gradient descent method for backpropagation to achieve iterative update of network parameters. In SSDANN, due to the existence of the semi-supervised learning mechanism, the features of the target domain data can be extracted to better distinguish textile defects of different styles. In the test phase, the target domain test data x tt is input into the network for testing, passing through the feature extractor and the label predictor in sequence, and finally being classified in the label predictor. The output obtained is as follows:

[0049] y t = G p (G f (x tt ) ; O, p) = softmax(OG f(x tt ) + p)(12)

[0050] In the above formula, y t represents a one-dimensional vector, and each element value represents the probability of a textile defect category. The index of the maximum value is used as the final output, that is, the label of the corresponding category.

[0051] Figure 3 Figure is a schematic diagram of the knowledge distillation algorithm in the present invention. The deep convolutional neural network can extract high-quality visual features of the input image. However, there is a large gap between textile images and images in the public domain. Directly introducing deep learning models in the public domain will result in performance degradation. To address the above domain difference problem, a teacher-student framework is adopted in this paper to transfer the knowledge learned by the CNN in the public dataset to the fabric image domain, enabling the deep learning model to have stronger pertinence in the textile field while retaining the original high-quality image feature extraction ability.

[0052] This network mainly includes two CNN models with the same structure, and the training samples are normal fabric images. The public domain model serves as the teacher T, responsible for imparting knowledge, and the network weight parameters of the model are fixed; the textile domain model serves as the student S, absorbing and learning knowledge, and updating the network weight parameters according to the loss value during the training process. Suppose and respectively represent the feature maps of the l-th layer output by the training sample I passing through the teacher model and the student model. The total loss function of the training can be expressed by the following formula:

[0053]

[0054] Among them, vec(x) represents performing a vectorization operation on the feature map to convert the feature matrix into a one-dimensional feature vector, and the student model is updated using the stochastic gradient descent algorithm during the training process.

[0055] Figure 4 Figure Figure 4 is a schematic diagram of the anomaly determination framework based on multi-level feature matching designed by the present invention for small-scale surface defects. In the training stage, taking normal fabric images as the input, the hidden feature maps of each layer of the deep learning model are extracted to construct a multi-level feature memory unit based on normal data. In the testing stage, for a certain test sample, after the same multi-level feature extraction operation, the multi-level features of the test data are obtained, and multi-level feature matching is performed with the constructed memory unit to calculate the anomaly score of the current sample. If this score exceeds a certain threshold, it is determined that the fabric has surface defects.

[0056] Suppose Φ l (I i ) ∈ R (w×h×c)Denote the feature map of the \(l\)-th layer after the \(i\)-th sample passes through the deep learning model. \(w\), \(h\), and \(c\) represent the width, height, and number of channels respectively. The expression of the memory unit is as follows:

[0057]

[0058] Among them, \(P\) represents global average pooling, which stretches the input of any size into a one-dimensional vector. \(N\) represents the total number of normal samples. The number of levels used in this article is 4, and \(l\in\{1,2,3,4\}\) corresponds to four different downsampling scale feature levels in the classic residual network ResNet.

[0059] Suppose the feature of the \(l\)-th level of the \(i\)-th normal sample in the memory unit \(M\) is The feature of the \(l\)-th level of the sample to be tested is denoted as \(f\) l , then the initial anomaly score at the \(l\)-th layer can be expressed by the following formula:

[0060]

[0061] Among them, The physical meaning of is the minimum Euclidean distance between the test sample feature and the corresponding feature in the memory unit. However, there are relatively outlier specific sample features in the normal data itself. Using only the memory feature with the closest distance as the judgment basis is difficult to prove the anomaly of the test sample itself. Therefore, the present invention comprehensively considers multiple neighboring features and designs an anomaly score correction module to improve the confidence of the anomaly score. The anomaly score correction factor can be expressed as:

[0062]

[0063] Among them, \(N\) k represents \(k\)-nearest neighbor retrieval, and selects the \(k\) nearest neighbor memory features from the test feature \(f\) l . The final anomaly score \(s\) can be expressed as:

[0064]

[0065] When the anomaly score exceeds a certain threshold, it can be determined that the sample is abnormal, that is, there are small-scale surface defects in the textile image.

[0066] The above is the preferred embodiment of the present invention. All changes made according to the technical solution of the present invention, when the functions and effects generated do not exceed the scope of the technical solution of the present invention, shall fall within the protection scope of the present invention.

Claims

1. A textile defect recognition method integrating cross - domain migration and anomaly detection, characterized in that, It includes the following steps: Step 1, preprocessing of fabric image data Collect fabric images through a machine as the original data set, process the initial data in the original data set, select a fixed style as the source domain, and integrate other styles as the target domain; Step 2, establishing a fabric data image library Divide the image set into normal images, large-scale morphological defect images, and small-scale surface defect images, and label some data in the morphological defect images, and retain the marked information; Step 3, recognition of large-scale morphological defects of fabrics For the large-scale morphological defects in fabrics, an improved semi-supervised cross-domain neural network SSDANN is proposed, which consists of a feature extractor, a label predictor, and a domain discriminator; the feature extractor realizes the mapping between different style data through the source domain and the labeled target domain; the domain discriminator realizes the migration of data from the source domain to the target domain through unsupervised learning; During the training process of SSDANN, the source domain and target domain data are respectively input into the feature extractor to extract the deep features of the fabric, and then the feature information is sent to two branches, namely the label predictor G and the domain discriminator D; The source domain data is used for supervised learning to optimize the label predictor; the source domain data and the unlabeled target domain data are input into the domain discriminator for unsupervised learning to confuse the feature information between the source domain and the target domain; Step 4, detection of surface defects of fabrics For the small-scale surface defects in fabrics, an anomaly detection method that does not require defect data to participate in training is proposed: First, to address the domain difference problem between the general domain and the fabric domain, knowledge distillation technology is used to improve the ability of the deep model to express fabric image features; Secondly, a multi-level feature extraction method is designed to construct a normal data feature memory unit to generate and store the multi-level hidden features of normal fabric samples; finally, a multi-level feature matching mechanism based on the memory unit is proposed, and the anomaly score of the test sample is calculated through the nearest neighbor retrieval algorithm to judge the fabric state; The anomaly detection method that does not require defect data for training constructs a multi-level feature matching network based on small-scale surface defects, including two CNN models with the same structure, and the training samples are normal fabric images; the public domain model serves as the teacher T, responsible for imparting knowledge, and the network weight parameters of the model are fixed; the fabric domain model serves as the student S, absorbs and learns knowledge, and updates the network weight parameters according to the loss value during the training process; assume and respectively represent the feature maps of the l-th layer output by the training sample I passing through the teacher T and the student S, and the total loss function of the training is expressed by the following formula: Among them, vec(x) represents performing a vectorization operation on the feature map to convert the feature matrix into a 1D feature vector, and the student model is updated using the stochastic gradient descent algorithm during the training process; In the training stage, taking normal fabric images as input, the hidden feature maps of each layer of the multi-level feature matching network are extracted to construct a multi-level feature memory unit based on normal data; in the test stage, for a certain test sample, the hidden feature maps of each layer of the multi-level feature matching network of the test sample are extracted and multi-level feature matching is performed with the multi-level feature memory unit based on normal data to calculate the anomaly score of the current sample. If this score exceeds a predetermined threshold, it is determined that the fabric has surface defects; Suppose Φ l (I i ) ∈ R (w×h×c) represents the l-th layer feature map of the i-th sample after passing through the multi-level feature matching network, where w, h, and c represent the width, height, and number of channels respectively; the expression of the multi-level feature memory unit based on normal data is as follows: Among them, P represents global average pooling, which stretches the input of any size into a 1D vector, N represents the total number of normal samples, and l∈{1,2,3,4} corresponds to four different downsampling scale feature levels in the classic residual network ResNet; Suppose that the l-level feature of the i-th normal sample in the multi-level feature memory unit M based on normal data is The l-level feature of the sample to be tested is denoted as f l , then the initial anomaly score at layer l is represented by the following formula: Among them, The physical meaning is the minimum Euclidean distance between the test sample features and the corresponding features in the multi-level feature memory unit based on normal data; considering multiple neighboring features comprehensively, an anomaly score correction module is designed to improve the confidence of the anomaly score; the anomaly score correction factor is expressed as: where N k represents k-nearest neighbor retrieval, and selects the k nearest neighbor memory features to the test feature f l ; the final anomaly score s is expressed as: When the anomaly score exceeds the predetermined threshold, it can be judged that the sample is abnormal, that is, the fabric image has small-scale surface defects.

2. The textile defect recognition method integrating cross-domain migration and anomaly detection according to claim 1, wherein The specific composition of the SSDANN is as follows: The feature extractor consists of multiple convolutional and pooling layers, which are used to map data to a specific feature space, enabling the label predictor to distinguish fabric categories and the domain discriminator to confuse whether the fabric comes from the source domain or the target domain. Using sigm as the activation function among them, its output is: G f (x; W, b) = sigm(Wx + b) (1) Where x is the source domain and target domain samples, W is the weight factor, and b is the bias term; the labeled data of the source domain is sent into the feature extractor, and then the output part is sent into the label predictor; the label predictor classifies the source domain data and classifies the correct label; the output of the label predictor is the Softmax activation function, taking x s as an example of the source domain, and y s as the label corresponding to x s The output of the label predictor is expressed as: G p (G f (x s )); Z,k) = softmax(ZG f (x s ) + k) (2) where G f (x s ) represents the input of the label predictor and is also the output of the feature extractor, Z is the weight factor, and k is the bias term; the loss function is expressed as: where G p (G f (x s )) is the output of the label predictor, and y s is the label corresponding to the source domain sample x s ; the feature extractor and the label classifier form a feed-forward neural network; for the fixed style of the source domain fabric, there is the following optimization objective function: where n represents the number of source domain samples, and L i p (W, b, Z, k) represents the label prediction loss of the i-th sample, λ represents a regularization term parameter, and R(W, b) represents a regularizer that serves to prevent overfitting of the label prediction network; The domain classifier is used to perform domain discrimination on the overall input data to determine whether the data comes from the source domain or the target domain. This is an unsupervised learning process. At this stage, only unlabeled source domain data and target domain data are input into the feature extractor. After processing, they are input into the domain classifier for domain discrimination. The domain classifier uses the sigmoid activation function, and its output is expressed as: G d (G f (x m );O,p)=sigm(O T G f (x m )+p) (5) where x m represents the integrated data of the source domain and the target domain, and O and p represent the weight and the bias term respectively; the domain classifier takes binary cross-entropy as the loss function, and its output is expressed as: where d i is 0 or 1, representing the domain label of the i-th sample; To simplify the training process, a gradient reversal layer GRL is introduced between the feature extractor and the domain classifier. In the forward propagation, GRL is regarded as an identity transformation, while in the backward propagation, it realizes gradient inversion, multiplies by a -α and passes it to the previous layer. The forward and backward propagations of the gradient reversal layer are shown as follows: In the formula, I is an identity matrix. During the backward propagation process, the gradient of the domain classification loss is automatically inverted before being propagated backward to the parameters of the feature extractor, realizing the adversarial loss of a GAN. The objective optimization function of the domain discriminator is: Input the labeled target domain data (x pt , y pt ) into the feature extractor to extract features, and the output is as follows: G f (x pt ; W, b) = ReLU(Wx pt + b) (9) The output of the feature extractor is input into the label predictor, and the output result is: G p (G f (x pt );O,p)=softmax(OG f (x pt )+p) (10) The label classification loss of the labeled target domain data is: The calculation method of the loss function for the labeled target domain data is the same as formula (3). The network uses the gradient descent method for backward propagation to realize the iterative update of network parameters. During the test phase, the target domain test data x tt is input into the SSDANN for testing. It passes through the feature extractor and the label predictor successively, and finally classification is performed in the label predictor. The obtained output is as follows: y t = G p (G f (x tt )); O,p) = softmax(OG f (x tt ) + p)(12) where y t represents a one-dimensional vector, and the value of each element represents the probability of the fabric defect category. The index of the maximum value is used as the final output, that is, the label of the corresponding category.

Images