A causal machine learning method for visual inspection of defects on printed fabrics

Abstract

The application discloses a kind of deep learning methods for printing fabric defect visual inspection, it is characterized in that, by building visual acquisition equipment in printing fabric defect detection end, collect positive sample data and respectively construct defect data set and printing data set of printing fabric image;Arbitrarily selected convolutional neural network is trained using printing data set, the parameters of model are optimized by minimizing loss function, realize the learning of printing fabric background pattern feature, and save the final parameters of model;Arbitrarily selected convolutional neural network is trained using defect data set, the feature difference of multiple scales is calculated in the training process of model and finally integrated to give loss function, the parameters of model are optimized using weighted loss function, realize the accurate learning and detection of defect feature.The application can realize the accurate identification and detection of printing fabric defect with background pattern visual interference.

Description

TECHNICAL FIELD

[0001] The present application relates to a causal machine learning method for visual detection of defects of printed fabrics, and belongs to the technical fields of computer vision, machine learning, product quality detection, etc. BACKGROUND

[0002] Printed fabrics are important textile products with high profit value, which can be widely used in the production and manufacturing of clothing textiles, decorative textiles, and industrial textiles as basic raw materials. The production and manufacturing of printed fabrics need to go through complicated production and manufacturing processes such as fabric design, printing, and product post-processing. In the above production process, equipment failure or human operation errors can cause defects on the surface of the fabric, resulting in a significant decrease in product price and causing unnecessary economic losses to textile enterprises. However, due to the fact that the surface of the printed fabric is covered with background patterns with diverse color distribution and morphological characteristics, which have similar graphical representations as fabric defects, visual confusion occurs during the detection process. Therefore, how to avoid the strong interference of the background patterns and achieve autonomous and accurate detection of the defects of the printed fabrics is a key problem to be solved.

[0003] With the development of computer vision technology, many scholars have begun to study the detection method of textile defects. The current visual detection technology can be divided into traditional image processing based method, traditional machine learning based method and big data based detection method. The traditional image processing method differentiates the representation of defects in vision by enhancing the texture features of fabric images, and realizes the identification of the defect area. However, this method has the defects of low detection efficiency, simple application scene and single detection type, and cannot cope with complex detection tasks with multiple defect representations. With the wide application of artificial intelligence technology, the defect detection method based on machine learning is proposed. By fusing artificial neural network and support vector machine and other machine learning technologies on the basis of traditional image processing method, the learning, identification and detection of defect features are realized. However, due to the factors such as sparse data samples and simple network structure, the timeliness, accuracy and generalization of the detection method cannot meet the actual detection demand. In order to overcome the difficulties of the above two methods in data, model and algorithm, the defect detection method based on the second generation of data-driven artificial intelligence technology is widely studied. By designing and constructing deep neural network to sample and reduce the dimension of a large number of fabric image positive and negative data samples, and then establishing the "feature-label" data correlation, the accurate learning and efficient detection of defects are realized. However, for the visual confusion of printed fabric defect visual detection, the deep neural network is easy to produce sample selection deviation at pixel level and learn false data correlation in the establishment of feature and label data correlation, that is, the detection model cannot really distinguish defects and background patterns, which affects the credibility and interpretability of detection. Some scholars try to solve this problem from the perspective of visual attention, but this strategy does not break through the essential difficulty of the data-driven detection model that cannot cope with the printed fabric defect detection, that is, starting from the machine learning paradigm, guiding the training and optimization process of the detection model, realizing the accurate identification and detection of defects.

[0004] Currently, some scholars begin to combine causality science on the basis of machine learning, so as to solve the problem of decision confusion or error of visual model. Causality science is a frontier of statistical information science, which can analyze the causal relationship of key variables in specific tasks, so as to further realize the utilization of causal relationship. However, due to the great difference between the background patterns of various printed fabrics, the general causal machine learning method cannot realize the accurate learning and detection of printed fabric defects. SUMMARY

[0005] The technical problem to be solved by the present application is that the existing big data based fabric defect visual automatic detection technology cannot avoid the visual interference of printed fabric background patterns, and cannot realize the accurate detection of defects.

[0006] In order to solve the above technical problems, the technical scheme of the present application is to provide a causal machine learning method for printed fabric defect visual detection, characterized by comprising the following steps:

[0007] Step 1, performing causal reasoning on the printed fabric defect visual detection task, constructing a structural causal model, and analyzing to obtain a causal intervention strategy P(L|do(D)) that can accurately identify defects:

[0008] P(L|do(D))=P(L|B)P(L|D,B)

[0009] In the formula, P, D, B and L represent pixel matrix, defect feature, pattern feature and defective fabric label respectively;

[0010] Step 2, building a visual acquisition device at the printed fabric defect detection end, collecting positive and negative sample data of printed fabric images and respectively constructing a defect data set D containing normal and defective printed fabric images and a printed data set D' containing only normal printed fabric images:

[0011] D={(x1,y1),...,(x n ,y n )}

[0012] D'={(x'1,y'1),...,(x' n ,y' n )}

[0013] In the formula, x n and x' n represent the nth image of different data sets, y n and y' n represent the label of the nth image of different data sets; do(D) represents the causal intervention on the confounding factor of the defect feature; P(L|B) is the first part of the causal intervention strategy, indicating that only background pattern features exist in the feature space of the detection model; P(L|D,B) is the second part of the causal intervention strategy, indicating that the detection model predicts the label of the defective fabric under the condition that the defect feature and the background pattern feature exist in the feature space at the same time;

[0014] Step 3, training an arbitrarily selected convolutional neural network F' with k downsampling operations using the printed data set D', optimizing the parameters θ' of the convolutional neural network F' by minimizing the loss function, realizing the learning of the background pattern features of the printed fabric, and saving the final parameters θ' of the convolutional neural network F', thereby completing the first part of the causal intervention strategy;

[0015] Step 4, training an arbitrarily selected convolutional neural network F with k down-sampling operations using the defect dataset D, the convolutional neural network F being consistent with the structure of the convolutional neural network F', the calculation of the feature difference at k scales in the training process of the convolutional neural network F realizing multi-scale feature causal intervention, and finally integrating the causal relationship weight to construct a loss function, using the weighted loss function to optimize the parameters θ of the convolutional neural network F, realizing accurate learning and detection of defect features, and saving the final parameters of the convolutional neural network F, thereby completing the second part of the causal intervention strategy.

[0016] Preferably, in step 1, the calculation method of the causal intervention strategy is a backdoor criterion.

[0017] Preferably, the step 1 comprises the following steps:

[0018] Step 101, analyzing the structural causal model M constructed for the printed fabric defect detection task c :

[0019] M c = {P, D, B, L}

[0020] Wherein, P, D, B and L represent pixel matrix, defect feature, pattern feature and defect fabric label respectively;

[0021] Step 102, analyzing the structural causal model, and obtaining the causal intervention strategy for accurately detecting defects through backdoor adjustment of causal reasoning: P(L|do(D)) = P(L|B)P(L|D,B).

[0022] Preferably, the step 3 comprises the following steps:

[0023] Step 301, in order to complete P(B) of the causal intervention strategy, i.e. to obtain the background pattern feature, training an arbitrarily selected convolutional neural network F' with k down-sampling operations using the printed data set D':

[0024] F' = {f'1,..., f' k}

[0025] In the formula, f' k represents the output of the kth sampling stage of the convolutional neural network F';

[0026] Step 302, setting the training round number E' of the convolutional neural network F' and the input data batch size B'; for any batch of data in any round of training, using the loss function L' to calculate the loss between the predicted value and the true value:

[0027]

[0028] Step 303, after each round of training, the parameters θ' of the convolutional neural network F' are optimized:

[0029]

[0030] In the formula, α represents the learning rate, and (θ')' represents the optimized parameters of the convolutional neural network F';

[0031] Step 304, save the final parameters θ' of the convolutional neural network F' after E' rounds of training.

[0032] Preferably, the step 4 comprises the following steps:

[0033] Step 401, in order to complete the P(L|D, B) of the causal intervention strategy, that is, to collect the background pattern features and defect features and to predict the labels of the data images, train an arbitrary convolutional neural network F with k downsampling operations using the defect data set D, and the structure of the convolutional neural network F is consistent with that of the convolutional neural network F':

[0034] F = {f1,..., fk} k}

[0035] In the formula, fk represents the output of the kth sampling stage of the convolutional neural network F; k

[0036] Step 402, set the number of training rounds E of the convolutional neural network F and the input data batch size B; for any batch of data in any round of training, calculate the feature difference S of the corresponding k dimensions of the convolutional neural network F' and the convolutional neural network F in the form of feature difference respectively k , realize the causal intervention of k-dimensional feature level:

[0037] S k = Φ aavg_p (f k -f' θ'k )

[0038] In the formula, Φ aavg_p (·) represents an adaptive mean pooling operation, and f' θ'k represents the convolutional neural network F' that has been trained in step 304, at this time it does not perform label prediction of the input image, but only performs end-to-end feature extraction.

[0039] Step 403, integrate the k-dimensional feature difference S k to calculate the causal relationship weight S c :

[0040]

[0041] In the formula, ω​i represents the integrated parameter of the i-th dimension;

[0042] Step 404, according to the causal relationship weight S c The loss function L of the convolutional neural network F is constructed, and the loss of the predicted value and the true value is calculated:

[0043]

[0044] Step 405, after each round of training, the parameters θ of the convolutional neural network F are optimized:

[0045]

[0046] In the formula, (θ)' represents the optimized parameters of the convolutional neural network F;

[0047] Step 406, save the final parameters θ of the convolutional neural network F after E rounds of training, and the training is completed.

[0048] The key to accurate detection of defects of printed fabrics is to guide the detection model to learn the feature difference between the background patterns of printed fabrics and defects, so that the deep neural network accurately constructs the data association between features and labels. The application makes full use of the feature extraction capability of deep neural network for high-dimensional image, realizes causal intervention through feature difference, thereby maximizing the difference between background patterns and defects, guiding the detection model to accurately capture the causal relationship between defect features and defect fabric labels to realize defect recognition and detection.

[0049] Compared with the prior art, the application has the following technical effects:

[0050] (1) The application starts from the essence of learning and optimization of the current data-driven defect detection method, and uses causal reasoning to guide the deep neural network to overcome the visual confusion problem of printed fabric defect detection

[0051] (2) The application realizes causal intervention through feature difference to help the model accurately learn defect features;

[0052] (3) The application considers the feature distribution difference of different background patterns, introduces feature difference of different scales to realize defect detection of printed fabrics with different background patterns, thereby enhancing the generalization of the detection model;

[0053] (4) The application uses the causal relationship weighted loss function containing feature difference information to guide the detection model to accurately learn the defect through multi-network collaborative training. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1 The flow chart of the implementation process of the causal machine learning method for printed fabric defect visual detection provided by the application. DETAILED DESCRIPTION

[0055] The application will be further described below in connection with specific embodiments. It should be understood that these embodiments are only used to illustrate the application and not used to limit the scope of the application. Furthermore, it should be understood that after reading the content of the application, those skilled in the art can make various modifications or changes to the application, and these equivalent forms also fall within the scope defined by the appended claims.

[0056] As shown in the accompanying drawings, Figure 1 a causal machine learning method for visual detection of defects of printed fabrics disclosed by the application comprises the following steps:

[0057] Step 1): performing causal inference on the visual detection task of defects of printed fabrics, constructing a structural causal model, and analyzing to obtain a causal intervention strategy that can accurately identify defects, further comprising the following steps:

[0058] Step 1.1): analyzing the structural causal model M constructed for the defect detection task of printed fabrics c :

[0059] M c ={P,D,B,L} (1)

[0060] Wherein, P, D, B and L represent pixel matrix, defect feature, pattern feature and defect fabric label respectively.

[0061] Step 1.2): analyzing the structural causal model, and obtaining a causal intervention strategy that can accurately detect defects through backdoor adjustment of causal inference:

[0062] P(L|do(D))=P(L|B)P(L|D,B) (2)

[0063] In the formula, do(D) represents the causal intervention on the confounding factor (i.e. background pattern feature) of the defect feature, P(L|do(D)) represents the causal intervention strategy, i.e. predicting the defect fabric label when the causal intervention is performed on the background pattern feature, P(L|B) is the first part of the causal intervention strategy, indicating that only the background pattern feature exists in the feature space of the detection model, and P(L|D,B) is the second part of the causal intervention strategy, indicating that the detection model predicts the defect fabric label when the defect feature and the background pattern feature exist in the feature space at the same time.

[0064] Step 2): First, a visual acquisition device is set up at the inspection end of the printed fabric to collect images of both normal and defective fabrics for actual engineering purposes. Simultaneously, positive and negative sample image data of printed fabrics with various background patterns are integrated and collected, and a defect dataset D containing images of both normal and defective printed fabrics and a printed fabric dataset D' containing only images of normal printed fabrics are constructed.

[0065] D = {(x1,y1),...,(x n ,y n )} (3)

[0066] D'={(x'1,y'1),...,(x' n ,y' n (4)

[0067] In the formula, x n and x' n Let y represent the nth image from different datasets. n and y' n These represent the labels of the nth image in different datasets.

[0068] Step 3): To complete the causal intervention strategy P(B), i.e., the acquisition of background pattern features, an arbitrarily selected convolutional neural network is trained using the printed dataset. The model parameters are optimized by minimizing the loss function to learn the background pattern features of the printed fabric, and the final parameters of the model are saved. This further includes the following steps:

[0069] Step 3.1): This invention utilizes a convolutional neural network as the basic network for the defect detection model. Two networks with identical structures are constructed, and the model is trained using the defect dataset D and the print dataset D', respectively.

[0070] First, train any convolutional neural network F' with k downsampling operations using the print dataset D':

[0071] F' = {f'1,...,f' k} (5)

[0072] In the formula, f' k This represents the output of the k-th sampling stage of the convolutional neural network F'.

[0073] Step 3.2): Set the number of training epochs E' and the batch size B' of the input data for the convolutional neural network F'. For any batch of data in any training epoch, calculate the loss between the predicted value and the true value using the loss function L':

[0074]

[0075] Step 3.3): After each round of training, the parameters θ' of the convolutional neural network F' are optimized:

[0076]

[0077] wherein a represents a learning rate, and (θ')' represents the optimized parameters of the convolutional neural network F'.

[0078] Step 3.4): The final parameters θ' of the convolutional neural network F' after the E'th round of training are saved, denoted as F' θ' .

[0079] Step 4): In order to complete the P(L|D, B) of the causal intervention strategy, i.e., to collect the background pattern features and defect features and to predict the labels of the data images, an arbitrarily selected convolutional neural network is trained using the defect data set, the feature difference at multiple scales is calculated during the training process of the model to realize multi-scale feature causal intervention, and finally the causal relationship weight is integrated to construct a loss function. The parameters of the model are optimized using the weighted loss function to realize accurate learning and detection of defect features, and the final parameters of the model are saved, further comprising the following steps:

[0080] Step 4.1): An arbitrarily selected convolutional neural network F with k downsampling operations is trained using the defect data set D, and the structure of the convolutional neural network F is consistent with that of the convolutional neural network F':

[0081] F = {f1,..., fk} (8) k

[0082] wherein fk represents the output of the k'th sampling stage of the convolutional neural network F. k

[0083] Step 4.2): The number of training rounds E of the convolutional neural network F and the input data batch size B are set. For any one batch of data in any one round of training, the feature difference S k of the k dimensions corresponding to the convolutional neural network F' and the convolutional neural network F is calculated in the form of feature difference, realizing causal intervention at the k-dimensional feature level:

[0084] S k = Φ aavg_p (f k -f' θ'k ) (9)

[0085] wherein Φ aavg_p (·) represents an adaptive mean pooling operation, and f' θ'k represents the convolutional neural network F' that has been trained in step 304, which does not perform label prediction of the input image at this time, but only performs end-to-end feature extraction. ​​

[0086] Step 4.3): Integrating the feature difference S of k dimensions k Calculate the causal relationship weight S c :

[0087]

[0088] In the formula, ω i represents the integrated parameter of the i-th dimension.

[0089] Step 4.4): According to the causal relationship weight S c Construct the loss function L of the convolutional neural network F, and calculate the loss between the predicted value and the true value:

[0090]

[0091] Step 4.5): After each round of training, optimize the parameters θ of the convolutional neural network F:

[0092]

[0093] In the formula, (θ)' represents the optimized parameters of the convolutional neural network F.

[0094] Step 4.6): Save the final parameters θ of the convolutional neural network F after E rounds of training, denoted as F θ , and the training is completed.

Claims

1. A causal machine learning method for visual inspection of defects on printed fabric, characterized in that, Comprising the following steps: Step 1, the visual inspection task of printing fabric defect causal reasoning, build structure causal model, analysis can accurately identify the causal intervention strategy P(L|do(D)) of defect: P(L|do(D))=P(L|B)P(L|D,B) In the formula: P, D, B and L represent pixel matrix, defect feature, pattern feature and defective fabric label respectively; Step 2, through the printing fabric defect detection end to build visual acquisition equipment, collect positive and negative sample data of printing fabric image and construct defect data set D containing normal and defective printing fabric image and printing data set D' containing only normal printing fabric image: D = {(x1,y1),...,(x n ,y n )} D' = {(x'1, y'1),..., (x'N, y'N)} n n )}​ where x n and x' n represent the n-th image of different data sets, y n and y' n represent the label of the n-th image of different data sets; do(D) represents the causal intervention on the confounding factor of the observable variable of defect features; P(L|B) is the first part of the causal intervention strategy, which represents the case that only background pattern features exist in the feature space of the detection model; P(L|D,B) is the second part of the causal intervention strategy, which represents the case that the detection model predicts the label of the defective fabric when the defect features and the background pattern features exist simultaneously in the feature space. Step 3, using printing data set D' to train any selected convolutional neural network F' with k downsampling operations, optimizing the parameters θ' of convolutional neural network F' by minimizing the loss function, realizing the learning of printing fabric background pattern feature, and saving the final parameters θ' of convolutional neural network F', so as to complete the first part of the causal intervention strategy; Step 4, using defect data set D to train any selected convolutional neural network F with k downsampling operations, the structure of convolutional neural network F is consistent with that of convolutional neural network F', and the feature difference of k scales is calculated in the training process of convolutional neural network F to realize multi-scale feature causal intervention, and finally the loss function is constructed by integrating the causal relationship weight, and the parameters θ of convolutional neural network F are optimized by using the weighted loss function, realizing the accurate learning and detection of defect feature, and saving the final parameters of convolutional neural network F, so as to complete the second part of the causal intervention strategy, comprising the following steps: Step 401, in order to complete the P(L|D,B) of causal intervention strategy, that is, to collect the background pattern feature and defect feature and predict the label of data image, use defect data set D to train any convolutional neural network F with k downsampling operations, and the structure of convolutional neural network F is consistent with that of convolutional neural network F': F = {f1,...,f k} In the formula, f k represents the output of the kth sampling stage of the convolutional neural network F; Step 402, set the training round number E of the convolutional neural network F, the input data batch quantity B; for any one batch of data in any one round of training, the feature difference is calculated respectively in the form of the feature difference S corresponding to the k dimensions of the convolutional neural network F' and the convolutional neural network F k , realize the causal intervention for the k-dimensional feature level: S k = Φ aavg_p (f k -f' θ'k ) In the formula, Φ aavg_p (·) represents an adaptive mean pooling operation, f θ'k represents the convolutional neural network F' that has been trained in step 304, at this time it does not perform label prediction of the input image, only end-to-end feature extraction; Step 403, integrating the feature difference S of k dimensions k The causal relationship weight S is calculated c : where ω i represents the integration parameter of the i-th dimension; Step 404, according to the causal relationship weight S c The loss function L of the convolutional neural network F is constructed, and the loss of the predicted value and the true value is calculated: Step 405, after each round of training, the parameters θ of convolutional neural network F are optimized: In the formula, (θ)' represents the optimized parameters of convolutional neural network F; Step 406, save the final parameters θ of convolutional neural network F after E rounds of training, and the training is completed.

2. The causal machine learning method for visual inspection of print defects on printed fabric as claimed in claim 1 wherein, In step 1, the calculation method of the causal intervention strategy is backdoor criterion.

3. The cause-effect machine learning method for visual inspection of print defects on a printed fabric as claimed in claim 1 wherein, The step 1 comprises the following steps: Step 101, analyze the structure causal model M constructed for the print fabric defect detection task c : M c = {P, D, B, L} Wherein, P, D, B and L represent pixel matrix, defect feature, pattern feature and defective fabric label respectively; Step 102, analyze the structure causal model, and obtain the causal intervention strategy P(L|do(D)) that can accurately detect defects through backdoor adjustment of causal reasoning: P(L|do(D))=P(L|B)P(L|D,B).

4. The causal machine learning method for visual inspection of print defects on printed fabric as claimed in claim 1 wherein, The step 3 comprises the following steps: Step 301, in order to complete the P(B) of causal intervention strategy, that is, to acquire the background pattern feature, use printing data set D' to train any convolutional neural network F' with k downsampling operations: F' = {f'1,...,f' k} In the formula, f' k represents the output of the kth sampling stage of the convolutional neural network F'; Step 302, set the training round E' of the convolutional neural network F', the input data batch amount B'; for any one batch of data in any one round of training, calculate the loss of the predicted value and the true value by using the loss function L': Step 303, after each round of training, optimize the parameters θ' of the convolutional neural network F': In the formula, α represents the learning rate, and (θ')' represents the optimized parameters of the convolutional neural network F'; Step 304, save the final parameters θ' of the convolutional neural network F' after E' rounds of training.

Images