Cross-domain textile quality inspection method for parameter evolution under modulation of conditional diffusion model

Abstract

The invention relates to a cross-domain textile quality inspection method for parameter evolution under modulation of a conditional diffusion model. The method comprises the following steps: step 1, collecting textile defect picture data; step 2, dividing fields and marking flaw data; step 3, building a textile defect identification system; 4, initializing components of the textile defect identification system; 5, carrying out model training by a textile defect identification system. The cross-domain textile quality inspection method based on parameter evolution under conditional diffusion model modulation has the advantages that the cross-domain textile quality inspection method based on parameter evolution under conditional diffusion model modulation has universality and can be used for cross-domain textile defect recognition tasks under different fabric production conditions or pattern and texture; according to the method, the situation that the source field continuously arrives is simulated in the model training process, the textile defect recognition system can reuse the model parameter value of the previous stage as the optimization starting point in the incremental learning process, the training time required by model convergence is saved, and meanwhile, manual labeling for target field data is not needed.

Description

Technical Field

[0001] The invention belongs to the field of industrial textile defect quality inspection, and specifically relates to a cross-domain textile quality inspection method of parameter evolution under conditional diffusion model modulation. Background Art

[0002] In the field of textile manufacturing, the stability of textile quality is crucial to ensure production efficiency and product quality. However, in the process of fabric production, various defects (such as holes, dirty yarn, broken warp, broken weft, etc.) will inevitably occur due to machine failure or yarn problems. These defects will lead to product degradation, customer complaints and other problems. In order to reduce quality problems and production losses, it is necessary to quickly identify and classify textile defects based on existing production status data so that appropriate corrective measures can be taken in a timely manner.

[0003] At present, the textile industry still mainly relies on manual visual recognition to identify fabric defects, which requires a lot of manpower and has poor real-time performance. In addition, manual processing is prone to subjective interference. The worker's qualifications, experience, emotional state, etc. will affect the accuracy of textile defect identification, and often lead to missed detection and false detection. In order to reduce labor costs and improve quality inspection efficiency, some textile companies have begun to try to introduce deep learning technology to realize the automatic identification of textile defects.

[0004] At present, there have been some studies on the automatic identification of textile defects. For example, the patent document of Chinese invention patent application number CN201711352882.8 discloses a textile defect identification method based on Gabor filter and RBF support vector machine, which includes the following steps: collecting textile images and preprocessing; generating Gabor filter groups and filtering textile images; selectively fusing the filter image groups; binarizing the fused images to make the defect areas visible; generating feature vectors for the defect areas; and classifying the feature vectors using the pre-trained RBF kernel function support vector machine classifier. The present invention uses Gabor filters to analyze images to ensure that texture information of each scale and angle can be covered, and uses RB support vector machines to analyze defect categories.

[0005] For example, the patent document of Chinese invention patent application No. CN202011399329.1 discloses a method for classifying fabric image defects, including the steps: S1, cutting the fabric image to be detected into M×M pixels, and converting the fabric image to be detected into a grayscale image; S2, extracting features of the grayscale image obtained in step S1 by a fusion method of grayscale co-occurrence matrix and gradient direction histogram to obtain a data set; S3, using a linear discriminant analysis method to perform feature selection on the data set obtained after feature extraction in step S2 to generate a new data set; S4, inputting the new data set obtained in step S3 into a support vector machine for testing to obtain the classification result of the fabric image to be detected.

[0006] However, most of the above methods are limited to a certain type of fabric and fabric production environment. Once the color and texture of the fabric or the light intensity of the production environment changes, resulting in a shift in the distribution of test data, these models trained based on traditional deep learning algorithms will find it difficult to accurately identify the type of fabric defects based on these data. In addition, these methods often consider static and unchanging fabric production conditions, and the models also use non-incremental training methods. Whenever new textile defect training data is collected after a period of iteration of the fabric production line, these methods need to train the model from scratch, and cannot reuse the previously learned knowledge of fabric defect characteristics, resulting in low efficiency in model optimization and upgrading.

[0007] In general, there are two main problems with the current textile quality inspection methods based on traditional deep learning technology: 1) The model has poor migration capabilities and is difficult to adapt to a variety of fabrics and changing workshop production environments; 2) The model optimization and upgrade efficiency is low, and it is difficult to efficiently use the newly added training data to improve the model performance. Therefore, the study of cross-domain defect classification models with strong generalization and sustainable learning capabilities is of great practical significance for coping with complex and changing workshop production environments and cross-fabric migration. Based on the above problems existing in the prior art, the present invention proposes a cross-domain textile quality inspection method with parameter evolution under conditional diffusion model modulation. Summary of the invention

[0008] The purpose of the present invention is to provide a cross-domain textile quality inspection method based on parameter evolution under conditional diffusion model modulation in order to solve the problems existing in the background technology.

[0009] The cross-domain textile quality inspection method of parameter evolution under the modulation of the conditional diffusion model comprises:

[0010] Step 1: Textile defect image data collection;

[0011] Step 2: domain division and defective data labeling;

[0012] Step 3, building a textile defect recognition system;

[0013] Step 4, initializing the textile defect recognition system components;

[0014] Step 5, the textile defect recognition system performs model training;

[0015] Step 6: The textile defect recognition system performs defect recognition and classification on the textile images in the target area.

[0016] Furthermore, in step 1, the textile defect image data is collected: the edge side visual sensor installed on the shuttle loom collects textile defect images corresponding to various defect categories under multiple different fabric production conditions and fabric patterns and textures.

[0017] Furthermore, the field division and defect data labeling described in step 2: the textile defect data division module divides the collected textile defect images into different fields according to the production conditions and pattern texture of the fabric, wherein each field represents a data distribution of the textile defect data, and the divided fields will be used in the subsequent model training phase and testing phase of the textile defect recognition system.

[0018] Furthermore, in step 2, the division into different fields includes: the number of fields used for training is T s , the number of fields used for testing is T t , where T s +T t =T; then T s domains are used as source domains for training, and the remaining T t domains are used as target domains for testing. Afterwards, for each textile defect image in the source domain, its defect category is labeled. The constructed domains satisfy the same label space but different data distribution.

[0019] Furthermore, the step 3 of building a textile defect recognition system includes: a textile defect feature extractor learning module builds a defect feature extractor, a textile defect classifier learning module builds a defect classifier and a reference point queue, a conditional diffusion model training module builds a conditional diffusion model, and a diffusion model training sample collection module builds an anchor point queue and a prototype queue.

[0020] Furthermore, in step 4, the textile defect recognition system components are initialized: the textile defect feature extractor learning module initializes the parameters of the defect feature extractor, the textile defect classifier learning module initializes the parameters of the defect classifier and initializes the reference point queue to an empty first-in-first-out queue, the conditional diffusion model training module initializes the parameters of the conditional diffusion model, and the diffusion model training sample collection module initializes the anchor point queue and the prototype queue to empty first-in-first-out queues.

[0021] Further, in step 5, the textile defect recognition system performs model training: in the scenario where the source domain continues to arrive, the model training is performed based on the method of parameter diffusion generation of the textile defect classifier, which specifically includes the following steps:

[0022] Step 5.1, queue reset and parameter state inheritance: The diffusion model training sample collection module initializes the anchor queue and prototype queue as empty first-in-first-out queues. When the tth source domain arrives, if t>1, the textile defect feature extractor learning module, the textile defect classifier learning module and the conditional diffusion model training module respectively set the parameters of the defect feature extractor, the parameters of the defect classifier and the parameters of the conditional diffusion model to the parameters in the previous source domain. Status after the training;

[0023] Step 5.2, the textile defect feature extractor learning module learns from the current source domain Randomly sample B samples from: in is the i-th sampled image from the t-th source domain, and It's a picture The defect category label of

[0024] Step 5.3, the textile defect feature extractor learning module takes the randomly sampled B image data Input the defect feature extractor to obtain the depth feature map, perform global average pooling on the depth feature map, and obtain the B image data The corresponding depth feature vector

[0025] Step 5.4, the textile defect classifier learning module converts the deep feature vector As the input of the defect classifier, get the current defect classifier for this B picture data The defect category probability prediction vector

[0026] Step 5.5, the textile defect classifier learning module predicts the vector based on the defect category probability and the true defect category label Calculating cross entropy loss

[0027] Step 5.6, if the reference point queue is not empty, the textile defect feature extractor learning module calculates the consistency loss based on the defect classifier parameters of the historical source domain stored in the reference point queue Otherwise, go to step 5.7;

[0028] Step 5.7, the textile defect recognition system calculates the overall loss The stochastic gradient descent algorithm is used for back propagation to update the parameters of the defect feature extractor and the defect classifier to minimize the overall loss and improve the performance of textile defect recognition.

[0029] Step 5.8: If the training warm-up phase on the current source domain has ended, the diffusion model training sample collection module calculates and updates the current source domain The prototype matrix μ t , and the prototype matrix μ t and the defect classifier parameters W updated in step 5.7 t Store them in the prototype queue and anchor queue respectively; otherwise, go to step 5.9;

[0030] Step 5.9, if the anchor queue is full, the conditional diffusion model training module calculates the noise estimation loss of the conditional diffusion model

[0031] Step 5.10, the conditional diffusion model training module back-propagates the gradient through the stochastic gradient descent algorithm to update the parameters of the conditional diffusion model to minimize the noise estimation loss; otherwise, proceed to step 5.11;

[0032] Step 5.11, repeat the above steps 5.2-5.10 until the model in the textile defect recognition system converges or reaches the maximum number of training steps;

[0033] Step 5.12, the textile defect classifier learning module will be in the current source domain The defect classifier parameters after training are stored in the reference point queue;

[0034] Step 5.13, if t <T s (T s is the number of source fields), the textile defect recognition system loads the next source field and jumps to step 5.1, otherwise jumps to step 5.14;

[0035] In step 5.14, the textile defect recognition system returns the reference point queue and the defect feature extractor parameters and conditional diffusion model parameters after the last source domain training is completed.

[0036] Furthermore, in step 5.3, the depth feature vector obtained by the global average pooling operation on the depth feature map output by the defect feature extractor is It has the form of formula (1):

[0037]

[0038] Among them, d f Represents the dimension of the deep feature vector.

[0039] Furthermore, in step 5.4, the defect category probability prediction vector It has the form of formula (2):

[0040]

[0041] Where C is the total number of defect categories, is the defect class probability prediction vector The value at the kth position of means the probability that the sample belongs to the kth type of defect.

[0042] Furthermore, in step 5.5, the cross entropy loss The formula is shown in formula (3):

[0043]

[0044] Among them, N t Is the source domain The total number of training samples, W t is the defect classifier in the source domain The parameters during training, It is a sample The deep feature vector obtained after being processed by the feature extractor, It is a sample The defect category label, softmax(·) represents the normalized exponential function, which is used to present the multi-classification results in the form of probability, and CrossEntropy(·) represents the cross entropy function.

[0045] Furthermore, in step 5.6, the consistency loss The formulas are shown in equations (4) to (5):

[0046]

[0047] Among them, |Q r | is the reference point queue Q r Length, is the reference point queue Q r Historical source fields saved in Parameters of the defect classifier after training. KL(·,·) is the Kullback-Leibler divergence function, and sg(·) is the gradient clipping operation to stabilize the training process.

[0048] Furthermore, in step 5.7, the overall loss The formula is shown in formula (6):

[0049]

[0050] in, is the cross entropy loss on the t-th source domain, is the consistency loss on the t-th source domain, and λ is a hyperparameter that weighs the loss contribution.

[0051] Furthermore, in step 5.8, the prototype matrix μ t The formulas are shown in equations (7) to (8):

[0052]

[0053] n t :=n t +B......(8),

[0054] Among them, n t Records the number of training samples from the end of the training warm-up phase to the current training iteration number when training on the t-th source domain; μ t [k] is the prototype matrix μ t The kth row of means the central eigenvector of the kth type of defect.

[0055] Furthermore, in step 5.9, the noise estimation loss The formulas are shown in equations (9) to (12):

[0056]

[0057] Among them, Q a is the anchor queue, It is the anchor queue Q a The mth saved in the source field Defect classifier parameters during training, It is the prototype queue Q p The mth saved in the source field The prototype matrix during training, is an isotropic standard multivariate Gaussian distribution, ∈ is the sampling distribution random noise, s is the number of noise adding steps, ε θ is a conditional diffusion model with parameter θ, β s' is the variance used in the s'th step during the noise addition process of the conditional diffusion model, and ⊕ represents the concatenation operation.

[0058] Further, in step 6, the textile defect recognition system performs defect recognition and classification on the textile images in the target area, specifically including:

[0059] Step 6.1, Textile Defect Recognition System Record Test Area For the current target areas;

[0060] Step 6.2, estimate the test area Prototype matrix: The domain-customized textile defect classifier parameter generation module estimates the prototype matrix μ based on the input image of the test domain using the deep feature vector processed by the defect feature extractor. test ;

[0061] Step 6.3, construct the defect classifier parameters for the test domain: The domain-customized textile defect classifier parameter generation module generates multiple sets of defect classifier parameters based on the reference point queue, conditional diffusion model and prototype matrix of the test domain, and integrates all defect classifier parameters through the average function to obtain the final defect classifier parameters for the test domain. Defect classifier parameters

[0062] Step 6.4, defect category prediction in the test domain: The domain-customized textile defect classifier parameter generation module uses the defect feature extractor and defect classifier parameters after training. For test areas The defect category of the image is predicted, and the defect category with the largest classification confidence in the defect category probability prediction vector is selected as the final classification result;

[0063] In step 6.5, the textile defect recognition system loads the next target area until all target areas have been tested.

[0064] Furthermore, in step 6.2, the prototype matrix μ test The expressions are shown in equations (13) to (14):

[0065]

[0066] Among them, N test It is the test area The total number of test samples, It is the test area The deep feature vector of the i-th sample.

[0067] Furthermore, in step 6.3, the classifier parameters The formula is shown in formula (15):

[0068]

[0069] in, Based on historical source fields Defect classifier parameters and test fields The prototype matrix μ test , construction conditions The j-th defect classifier parameter residual generated by the conditional diffusion model, M gIndicates the number of times the defect classifier parameters are generated under each condition.

[0070] Compared with the prior art, the present invention has the following advantages:

[0071] 1. The method described in the present invention is not a defect recognition method for a specific fabric production condition or a specific fabric pattern and texture. The cross-domain textile quality inspection method of parameter evolution under the modulation of the conditional diffusion model proposed in the present invention is universal and general, and can be used for cross-domain textile defect recognition tasks under different fabric production conditions or patterns and textures;

[0072] 2. The training process of the method described in the present invention simulates the scenario where the source domain continues to arrive. During the incremental learning process, the textile defect recognition system does not need to start training from scratch, and can reuse the model parameter values ​​of the previous stage as the starting point for optimization, which greatly saves the training time required for model convergence. At the same time, there is no need to manually label the target domain data, reducing labor costs.

[0073] 3. The method described in the present invention utilizes the powerful distribution modeling and generation capabilities of the conditional diffusion model to achieve field-customized defect classifier parameter generation and provide more accurate and robust prediction of textile defect categories. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 It is a module relationship diagram of the cross-domain textile quality inspection method of parameter evolution under the modulation of the conditional diffusion model in an embodiment of the present invention;

[0075] Figure 2 It is a flow chart of a cross-domain textile quality inspection method of parameter evolution under the modulation of a conditional diffusion model in an embodiment of the present invention;

[0076] Figure 3 It is a schematic diagram of each module of the cross-domain textile quality inspection method of parameter evolution under the modulation of the conditional diffusion model according to an embodiment of the present invention; DETAILED DESCRIPTION

[0077] In order to make the purpose, technical solution and advantages of the method described in the present invention clearer, the specific embodiments of the method described in the present invention are further described in detail and supplemented with explanations in conjunction with formulas and drawings.

[0078] Example

[0079] like Figure 1-2 As shown, the cross-domain textile quality inspection method of parameter evolution under the modulation of the conditional diffusion model includes:

[0080] Step 1: Data collection of textile defect images: The edge-side visual sensor installed on the shuttle loom collects images of holes, stains, broken warps, and broken wefts of plain and twill textiles under different light intensities;

[0081] Step 2: Domain division and defect data annotation: The textile defect data division module divides the collected images into different domains according to the light intensity, plain weave, and twill of the fabric production conditions. The number of domains used for training is T. s , the number of fields used for testing is T t , where T s +T t =T; then T s domains are used as source domains for training, and the remaining T t domains are used as target domains for testing. Afterwards, for each textile defect image in the source domain, its defect category is annotated. The total number of defect categories is recorded as C. The constructed domains must have the same label space but different data distribution.

[0082] Step 3, building a textile defect recognition system: the textile defect feature extractor learning module builds a defect feature extractor, the textile defect classifier learning module builds a defect classifier and a reference point queue, the conditional diffusion model training module builds a conditional diffusion model, and the diffusion model training sample collection module builds an anchor point queue and a prototype queue;

[0083] Step 4, initialization of textile defect recognition system components: the textile defect feature extractor learning module initializes the parameters of the defect feature extractor, the textile defect classifier learning module initializes the parameters of the defect classifier and initializes the reference point queue to an empty first-in-first-out queue, the conditional diffusion model training module initializes the parameters of the conditional diffusion model, and the diffusion model training sample collection module initializes the anchor point queue and the prototype queue to empty first-in-first-out queues;

[0084] Step 5: Model training of the textile defect recognition system: In the scenario where the source domain continues to arrive, model training is performed based on the method of parameter diffusion generation of the textile defect classifier;

[0085] Step 6: The textile defect recognition system performs defect recognition and classification on the textile images in the target area.

[0086] As an optional step specific embodiment, in step 2, each domain only contains textile defect data of one fabric production condition or one pattern and texture. At the same time, as time goes by, source domains continue to arrive to simulate the continuous emergence of new training data in the textile production scenario as production iterations occur. Therefore, the textile defect recognition system can use the new source domains arriving at the current moment to further optimize and improve the generalization of the textile defect recognition system in unseen target domains. At the same time, in order to reduce storage costs, the textile defect recognition system can only access the current source domain. data, while historical source fields The data is not accessible, specifically, for the source field Express it as t∈{1,2,…,T s},in Is the source domain The i-th 224×224 textile defect image in It is a sample The defect category label, C is the total number of defect categories, N t Is the source domain The total number of samples for a specified target domain Express it as N test The target area The total number of test samples, The target area The i-th test sample is also a 224×224 textile defect image.

[0087] As an optional step specific embodiment, in step 3, a defect feature extractor E with a parameter of ψ is constructed based on a dense convolutional network with good feature extraction capability in the visual field. ψ ; Defect classifier H W It consists of a linear layer with a C-dimensional output and a parameter W; the conditional diffusion model ε θ Constructed as a U-shaped network with parameter θ; reference point queue Q r Constructed as a first-in-first-out queue with a maximum length of L; the anchor queue Q a Constructed as a first-in-first-out queue with a maximum length of M; prototype queue Q p Constructed as a first-in-first-out queue with a maximum length of M.

[0088] As an optional step specific embodiment, in step 4, in order to accelerate the model convergence of the textile defect recognition system, the textile defect feature extractor learning module uses the parameters of the pre-trained dense convolutional network to extract the defect feature extractor E ψ The parameters are initialized to ψ 0 The textile defect classifier learning module uses random initialization to initialize the defect classifier H W The parameter is W 0 , and at the same time the reference point queue Q r Initialized to an empty queue, the conditional diffusion model training module also uses random initialization to initialize the conditional diffusion model ε θ The parameter is θ 0 , the diffusion model training sample collection module will be anchor queue Q a and the prototype queue Q pBoth are initialized to empty queues.

[0089] As an optional step specific embodiment, step 5 includes:

[0090] Step 5.1, queue reset and parameter state inheritance: The diffusion model training sample collection module sets the anchor queue Q a and the prototype queue Q p Reset to an empty queue: When the tth source domain arrives, if t==1, the textile defect feature extractor learning module, the textile defect classifier learning module and the conditional diffusion model training module set the parameter ψ of the defect feature extractor at the current time t to 1 , the parameters of the defect classifier W 1 and the parameters θ of the conditional diffusion model 1 are set to the corresponding parameters initialized in step 3, namely ψ 1 ←ψ 0 ,W 1 ←W 0 ,θ 1 ←θ 0 ; Otherwise, the parameter ψ of the defect feature extractor at the current time t is t , the parameters of the defect classifier W t and the parameters θ of the conditional diffusion model t Set to the previous source field The parameter state after the training is completed, that is, ψ t ←ψ t-1 ,W t ←W t-1 ,θ t ←θ t-1 , which allows the model knowledge learned in the previous training process to be reused without completely retraining, reducing the training time required for model convergence and making it more efficient;

[0091] Step 5.2, the textile defect feature extractor learning module learns from the current source domain Randomly sample B samples from: in is the i-th sampled image from the t-th source domain, and It's a picture The defect category label of

[0092] Step 5.3, the textile defect feature extractor learning module takes the randomly sampled B image data Input current defect feature extractor Get the depth feature map, perform global average pooling on the depth feature map, and get the B image data The corresponding depth feature vector

[0093] Step 5.4, the textile defect classifier learning module converts the deep feature vector As the current defect classifier Input, get the current defect classifier for this B picture data The defect category probability prediction vector

[0094] Step 5.5, the textile defect classifier learning module predicts the vector based on the defect category probability and the true defect category label Calculating cross entropy loss

[0095] Step 5.6, if the reference point queue Q r If it is not empty, the textile defect feature extractor learning module is based on the parameters W of the current defect classifier. t and the reference point queue Q r Defect classifier parameters for historical source domains saved in Calculating consistency loss Otherwise, go to step 5.7;

[0096] Step 5.7, the textile defect recognition system calculates the overall loss And use the stochastic gradient descent algorithm for back propagation to update the parameters ψ of the current defect feature extractor t and the parameters W of the current defect classifier t , to minimize the overall loss Achieve improved performance in textile defect identification;

[0097] Step 5.8, if the training warm-up phase on the current source domain has ended, calculate and update the current source domain The prototype matrix μ t , and the prototype matrix μ t and the defect classifier parameters W updated in step 5.7 t Store them in the prototype queue Q respectively p and anchor queue Q a Middle: Q p ←μ t , Q a ←W t Otherwise, go to step 5.9. In the specific implementation, for the training warm-up phase, set the current source domain The maximum number of training iterations on is I t , then the current source domain The warm-up phase of training is from the beginning of training to the first ttraining iterations, where ρ∈(0,1) is a hyperparameter that controls the duration of the warm-up phase of training;

[0098] Step 5.9, if the anchor queue Q a If it is full, the conditional diffusion model training module randomly samples a noise step s from the uniform distribution U(1,2,…,S) and randomly samples a noise ∈ from the isotropic standard multivariate Gaussian distribution to calculate the current conditional diffusion model The noise estimation loss Where S is the maximum number of noise addition steps;

[0099] Step 5.10: The conditional diffusion model training module back-propagates the gradient through the stochastic gradient descent algorithm to update the parameters θ of the current conditional diffusion model. t , to minimize the noise estimation loss Otherwise, go to step 5.11;

[0100] Step 5.11, repeat the above steps 5.2-5.10 until the model in the textile defect recognition system converges or reaches the maximum number of training iterations I t ;

[0101] Step 5.12, in the current source field Defect classifier parameters after training Store in reference point queue Q r :

[0102] Step 5.13, if t <T s (T s is the number of source fields), the textile defect recognition system loads the next source field and jumps to step 5.1, otherwise jumps to step 5.14;

[0103] Step 5.14, the textile defect recognition system returns to the reference point queue Q r and the defect feature extractor parameters after the last source domain training and conditional diffusion model parameters

[0104] As an optional step specific embodiment, step 6 includes:

[0105] Step 6.1, Textile Defect Recognition System Record Test Area The current target areas include N test The target area The total number of test samples, The target area The i-th test sample is also a 224×224 textile defect image;

[0106] Step 6.2, estimate the test area Prototype matrix: The domain-customized textile defect classifier parameter generation module will test the domain All images of The input parameters are The defect feature extractor uses the deep feature vector processed by the defect feature extractor. and the reference point queue Q r The defect classifier parameters saved in are used to estimate the prototype matrix μ test ;

[0107] Step 6.3, construct the defect classifier parameters for the test domain: The domain-customized textile defect classifier parameter generation module is based on the reference point queue Q r , parameters are The conditional diffusion model and the prototype matrix μ of the test domain test , randomly sample M from a standard multivariate Gaussian distribution with isotropic properties g noise, using the conditional diffusion model The inverse denoising process generates multiple sets of defect classifier parameters, and integrates all defect classifier parameters through the average function to obtain the final defect classifier used in the test field. Defect classifier parameters

[0108] Step 6.4, defect category prediction in the test domain: The domain-customized textile defect classifier parameter generation module uses parameters Defect feature extractor and the parameters are Defect classifier For test areas The defect category of the image is predicted, and the defect category with the largest classification confidence in the defect category probability prediction vector is selected as the final classification result; specifically, for each test sample Its category defect prediction Obtained by the following formula:

[0109] Defect Class Prediction:

[0110] Defect class probability prediction vector:

[0111] Deep feature vector:

[0112] Among them, softmax(·) represents the normalized exponential function, which is used to present the results of multi-classification in the form of probability; is the defect class probability prediction vector The value at the kth position of means the probability that the sample belongs to the kth type of defect; It is a sample Defect Feature Extractor The deep feature vector obtained after processing;

[0113] Step 6.5, the textile defect recognition system loads the next target area until all target areas have been tested, such as Figure 3 As shown, the method of the present invention includes the following six modules:

[0114] The textile defect data division module divides the collected textile defect images into different fields according to the production conditions and patterns and textures of the fabrics, where each field represents a data distribution of textile defect data. The divided fields will be used in the subsequent model training and testing stages of the textile defect recognition system;

[0115] Textile defect feature extractor learning module: A domain-shared feature space is learned using prediction consistency constraints to eliminate data distribution shifts caused by changes in fabric production conditions (such as light intensity, etc.) and fabric color and texture changes; at the same time, through the restriction of domain-invariant feature representation, the distribution pattern changes at the data level are reflected in the defect classifier parameters as much as possible, so as to model the cross-domain pattern transformation of the defect classifier parameters. Specifically, the prediction consistency constraint is expressed as follows:

[0116]

[0117] in, is the prediction consistency loss when training on the t-th source domain, |Q r | is the reference point queue Q r The length, N t Is the source domain The total number of training samples, C is the total number of defect categories, KL(·,·) is the Kullback-Leibler divergence function, and softmax(·) represents the normalized exponential function, which is used to present the results of multi-classification in the form of probability. is the reference point queue Q r Historical source fields saved in The defect classifier parameters after training, It is a sample The deep feature vector after being processed by the defect feature extractor. sg(·) is the gradient truncation operation to stabilize the training process.

[0118] The textile defect classifier training module uses the cross entropy loss of the textile defect classification task to train a unique defect classifier for each field during the continuous arrival of the source field, and models the pattern of defect classifier parameters in each field. At the same time, since the data of the historical source field can no longer be accessed in the continuous learning scenario, the present invention needs to maintain a global reference point queue Q r , the defect classifier parameters after training on each source domain are saved for use in the subsequent conditional diffusion model training module and the field-customized textile defect classifier parameter generation module. Specifically, the cross entropy loss expression is as follows:

[0119]

[0120] in, is the cross entropy loss when training on the t-th source domain, N t Is the source domain The total number of training samples, W t is the defect classifier in the source domain The parameters during training, It is a sample The deep feature vector obtained after processing by the defect feature extractor, It is a sample The defect category label of , C is the total number of defect categories, softmax(·) represents the normalized exponential function, which is used to present the results of multi-classification in the form of probability, and CrossEntropy(·) represents the cross entropy function;

[0121] In the training process of each source field, the diffusion model training sample collection module uses an anchor queue Q a and a prototype queue Q p To collect information about defect classifier parameters and prototype matrix during training, and provide training samples for subsequent conditional diffusion models;

[0122] The conditional diffusion model training module is based on the defect classifier parameters in the historical source field and the prototype matrix information in the current source field. The conditional diffusion model is trained to learn how to change the defect classifier parameters in the historical source field to the defect classifier parameters in the current source field according to the prototype matrix (i.e., the class center distribution information in the current source field), so that it has the ability to migrate the defect classifier parameters. Specifically, the noise estimation loss that the conditional diffusion model training module needs to minimize during the training process is as follows:

[0123]

[0124] In the above formula:

[0125]

[0126] in, is the noise estimation loss after the t-th source field arrives, is the reference point queue Q r Historical source fields saved in The defect classifier parameters after training, It is the anchor queue Q a The mth saved in the source field The defect classifier parameters during training, ∈, are sampled from a standard multivariate Gaussian distribution with isotropic properties. random noise, s is the number of noise adding steps, ε θ is a conditional diffusion model with parameter θ, It is the prototype queue Q p The mth saved in the source field The prototype matrix during training, is an isotropic standard multivariate Gaussian distribution, β s' is the variance used in the s'th step of the conditional diffusion model noise addition process, ⊕ represents the splicing operation, C is the total number of defect categories, d f Represents the dimension of the deep feature vector;

[0127] Domain-customized textile defect classifier parameter generation module based on reference point queue Q r The defect classifier parameters of the historical source domain saved in the source domain are generated by injecting the information of the target domain into the conditional diffusion model through modulation conditions, so as to realize cross-domain textile defect recognition. Specifically, the domain-customized textile defect classifier parameter generation module first estimates the specified target domain using the following formula: The prototype matrix μ test :

[0128]

[0129] In the above formula:

[0130]

[0131] Among them, μ test [k] is the prototype matrix μ test The kth row of N means the central eigenvector of the kth type of defect; test It is the test area The total number of test samples; is the defect class probability prediction vector The value at the kth position; It is the test area is the deep feature vector of the ith sample; C is the total number of defect categories; |Qr | is the reference point queue Q r Length, is the reference point queue Q r Historical source fields saved in The defect classifier parameters after training; softmax(·) represents the normalized exponential function, which is used to present the results of multi-classification in the form of probability;

[0132] After getting the prototype matrix μ test Afterwards, the domain-customized textile defect classifier parameter generation module further constructs the target domain Defect classifier parameters This process is achieved through the following formula:

[0133]

[0134] in, Based on historical source fields Defect classifier parameters and target areas The prototype matrix μ test , construction conditions The j-th defect classifier parameter residual generated by the conditional diffusion model; M g represents the number of times the defect classifier parameters are generated under each condition; C is the total number of defect categories; d f Represents the dimension of the deep feature vector;

[0135] Finally, for the target area For each test sample of Defect feature extractor and parameters Defect classifier For a given textile defect image, the defect category is predicted, and the defect category with the largest classification confidence in the defect category probability prediction vector is selected as the final classification result. It is T s Parameters of the defect feature extractor after training in the source domain.

[0136] The present invention is not limited by the above-mentioned embodiments. The above-mentioned embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the concept and scope of the present invention, the present invention may have various changes and improvements. These changes and improvements shall all fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims.

Claims

1. A cross-domain textile quality inspection method based on parameter evolution under conditional diffusion model modulation, comprising: Step 1: Textile defect image data collection; Step 2, field division and defective data labeling; Step 3, building a textile defect recognition system; Step 4, initializing the textile defect recognition system components; Step 5, the textile defect recognition system performs model training; Step 6: The textile defect recognition system performs defect recognition and classification on the textile images in the target area.

2. According to the cross-domain textile quality inspection method of parameter evolution under conditional diffusion model modulation as claimed in claim 1, the textile defect image data collection in step 1 is: The edge-side visual sensor installed on the shuttle loom collects textile defect images corresponding to various defect categories under multiple different fabric production conditions and fabric patterns and textures.

3. According to the cross-domain textile quality inspection method with parameter evolution under the modulation of the conditional diffusion model as described in claim 1, the field division and defect data labeling described in step 2: the textile defect data division module divides the collected textile defect images into different fields according to the production conditions and pattern texture of the fabric, wherein each field represents a data distribution of the textile defect data, and the divided fields will be used in the subsequent model training stage and testing stage of the textile defect recognition system.

4. According to the cross-domain textile quality inspection method of parameter evolution under conditional diffusion model modulation as claimed in claim 1, in step 2, the division into different fields includes: The number of domains used for training is T s , the number of fields used for testing is T t , where T s +T t =T; then T s domains are used as source domains for training, and the remaining T t domains are used as target domains for testing. Afterwards, for each textile defect image in the source domain, its defect category is labeled. The constructed domains satisfy the same label space but different data distribution.

5. According to the cross-domain textile quality inspection method of parameter evolution under conditional diffusion model modulation according to claim 1, the step 3 of building a textile defect recognition system comprises: The textile defect feature extractor learning module builds the defect feature extractor, the textile defect classifier learning module builds the defect classifier and the reference point queue, the conditional diffusion model training module builds the conditional diffusion model, and the diffusion model training sample collection module builds the anchor point queue and the prototype queue.

6. According to the cross-domain textile quality inspection method of parameter evolution under conditional diffusion model modulation as claimed in claim 1, the textile defect recognition system component is initialized in step 4: The textile defect feature extractor learning module initializes the parameters of the defect feature extractor, the textile defect classifier learning module initializes the parameters of the defect classifier and initializes the reference point queue to an empty first-in-first-out queue, the conditional diffusion model training module initializes the parameters of the conditional diffusion model, and the diffusion model training sample collection module initializes the anchor point queue and the prototype queue to empty first-in-first-out queues.

7. According to the cross-domain textile quality inspection method of parameter evolution under conditional diffusion model modulation as claimed in claim 1, the textile defect recognition system in step 5 performs model training: In the scenario where the source domain continues to arrive, model training is performed based on the method of parameter diffusion generation of textile defect classifier, including the following steps: Step 5.1, queue reset and parameter state inheritance: The diffusion model training sample collection module initializes the anchor queue and prototype queue as empty first-in-first-out queues. When the tth source domain arrives, if t>1, the textile defect feature extractor learning module, the textile defect classifier learning module and the conditional diffusion model training module respectively set the parameters of the defect feature extractor, the parameters of the defect classifier and the parameters of the conditional diffusion model to the parameters in the previous source domain. Status after the training; Step 5.2, the textile defect feature extractor learning module learns from the current source domain Randomly sample B samples from: in is the i-th sampled image from the t-th source domain, and It's a picture The defect category label of Step 5.3, the textile defect feature extractor learning module takes the randomly sampled B image data Input the defect feature extractor to obtain the depth feature map, perform global average pooling on the depth feature map, and obtain the B image data The corresponding depth feature vector Step 5.4, the textile defect classifier learning module converts the deep feature vector As the input of the defect classifier, get the current defect classifier for this B picture data The defect category probability prediction vector Step 5.5, the textile defect classifier learning module predicts the vector based on the defect category probability and the true defect category label Calculating cross entropy loss Step 5.6, if the reference point queue is not empty, the textile defect feature extractor learning module calculates the consistency loss based on the defect classifier parameters of the historical source domain stored in the reference point queue Otherwise, go to step 5.7; Step 5.7, the textile defect recognition system calculates the overall loss The stochastic gradient descent algorithm is used for back propagation to update the parameters of the defect feature extractor and the defect classifier to minimize the overall loss and improve the performance of textile defect recognition. Step 5.8: If the training warm-up phase on the current source domain has ended, the diffusion model training sample collection module calculates and updates the current source domain The prototype matrix μ t , and the prototype matrix μ t and the defect classifier parameters W updated in step 5.7 t Store them in the prototype queue and anchor queue respectively; otherwise, go to step 5.9; Step 5.9, if the anchor queue is full, the conditional diffusion model training module calculates the noise estimation loss of the conditional diffusion model Step 5.10, the conditional diffusion model training module back-propagates the gradient through the stochastic gradient descent algorithm to update the parameters of the conditional diffusion model to minimize the noise estimation loss; otherwise, proceed to step 5.11; Step 5.11, repeat the above steps 5.2-5.10 until the model in the textile defect recognition system converges or reaches the maximum number of training steps; Step 5.12, the textile defect classifier learning module will be in the current source domain The defect classifier parameters after training are stored in the reference point queue; Step 5.13, if t <T s , T s is the number of source fields, the textile defect recognition system loads the next source field and jumps to step 5.1, otherwise jumps to step 5.14; In step 5.14, the textile defect recognition system returns the reference point queue and the defect feature extractor parameters and conditional diffusion model parameters after the last source domain training is completed.

8. According to the cross-domain textile quality inspection method with parameter evolution under the conditional diffusion model modulation of claim 7, in step 5.3, the depth feature vector obtained by the global average pooling operation of the depth feature map output by the defect feature extractor It has the form of formula (1): in, d f Represents the dimension of the deep feature vector.

9. According to the cross-domain textile quality inspection method with parameter evolution under the conditional diffusion model modulation of claim 7, in step 5.4, the defect category probability prediction vector It has the form of formula (2): in, C is the total number of defect categories, is the defect category probability prediction vector The value at the kth position of means the probability that the sample belongs to the kth type of defect.

10. According to the cross-domain textile quality inspection method of parameter evolution under conditional diffusion model modulation as claimed in claim 7, in step 5.5, the cross entropy loss The formula is shown in formula (3): in, N t Is the source field The total number of training samples, W t is the defect classifier in the source domain The parameters during training, It is a sample The deep feature vector obtained after being processed by the feature extractor, It is a sample The defect category label, softmax(·) represents the normalized exponential function, which is used to present the multi-classification results in the form of probability, and CrossEntropy(·) represents the cross entropy function; In step 5.6, the consistency loss The formulas are shown in equations (4) to (5): Among them, |Q r | is the reference point queue Q r Length, is the reference point queue Q r Historical source fields saved in The defect classifier parameters after training. KL(·,·) is the Kullback-Leibler divergence function, and sg(·) is the gradient clipping operation to stabilize the training process. In step 5.7, the overall loss The formula is shown in formula (6): in, is the cross entropy loss on the t-th source domain, is the consistency loss on the t-th source domain, and λ is a hyperparameter that weighs the loss contribution; In step 5.8, the prototype matrix μ t The formulas are shown in equations (7) to (8): n t :=n t +B......(8), Among them, n t Records the number of training samples from the end of the training warm-up phase to the current training iteration number when training on the t-th source domain; μ t [k] is the prototype matrix μ t The kth row of means the central eigenvector of the kth type of defect; In step 5.9, the noise estimation loss The formulas are shown in equations (9) to (12): Among them, Q a is the anchor queue, It is the anchor queue Q a The mth saved in the source field Defect classifier parameters during training, is the prototype queue Q p The mth saved in the source field The prototype matrix during training, is an isotropic standard multivariate Gaussian distribution, ∈ is the sampling distribution random noise, s is the number of noise adding steps, ε θ is a conditional diffusion model with parameter θ, β s' is the variance used in the s'th step during the noise addition process of the conditional diffusion model, and ⊕ represents the concatenation operation; Step 6: The textile defect recognition system performs defect recognition and classification on the textile images in the target area, specifically including: Step 6.1, Textile Defect Recognition System Record Test Area For the current target areas; Step 6.2, estimate the test area Prototype matrix: The domain-customized textile defect classifier parameter generation module estimates the prototype matrix μ based on the input image of the test domain using the deep feature vector processed by the defect feature extractor. test ; Step 6.3, construct the defect classifier parameters for the test domain: The domain-customized textile defect classifier parameter generation module generates multiple sets of defect classifier parameters based on the reference point queue, conditional diffusion model and prototype matrix of the test domain, and integrates all defect classifier parameters through the average function to obtain the final defect classifier parameters for the test domain. Defect classifier parameters Step 6.4, defect category prediction in the test domain: The domain-customized textile defect classifier parameter generation module uses the defect feature extractor and defect classifier parameters after training. For test areas The defect category of the image is predicted, and the defect category with the largest classification confidence in the defect category probability prediction vector is selected as the final classification result; Step 6.5, the textile defect recognition system loads the next target area until all target areas have been tested; In addition, in step 6.2, the prototype matrix μ test The expressions are shown in equations (13) to (14): Among them, N test It is the test area The total number of test samples, It is the test area The depth feature vector of the i-th sample; In step 6.3, the classifier parameters The formula is shown in formula (15): in, Based on historical source fields Defect classifier parameters and test fields The prototype matrix μ test , construction conditions The j-th defect classifier parameter residual generated by the conditional diffusion model, M g Indicates the number of times the defect classifier parameters are generated under each condition.

Images