Defect detection method based on plain cloth quality inspection experience prompt fine adjustment

Abstract

The invention relates to a defect detection method based on fine adjustment of plain cloth quality inspection experience prompts. The detection method comprises the following steps: step 1, collecting industrial cloth data; 2, cloth data preprocessing and pixel-level labeling are carried out; 3, constructing an industrial cloth defect detection knowledge base set; 4, constructing an industrial cloth defect detection model; step 5, plain cloth defect quality detection based on the industrial cloth defect detection model; 6, training and evaluating an industrial cloth defect detection model; and step 7, using the industrial cloth defect detection method. The detection method disclosed by the invention has the advantages that the industrial cloth defect detection model included in the detection method realizes the detection quality with higher resolution by adopting a dense semantic segmentation technology; pixel-level semantic segmentation is used, so that more accurate position information is provided for detected defects; and by setting and constructing the industrial cloth defect detection model, the labeling cost and the calculation cost are remarkably reduced.

Description

Technical Field

[0001] The invention relates to the technical field of industrial textiles, and in particular to a defect detection method based on fine-tuning of plain cloth quality inspection experience prompts. Background Art

[0002] In contemporary society, fabrics are essential materials in people's daily lives, and their quality directly affects people's usage experience and living standards. In actual production, the three fabric structures of plain, twill and satin each have their own importance and uses. First, plain fabric has the simplest structure and is widely used in daily clothing (such as shirts, bed sheets, tablecloths, etc.) and decorative fabrics; second, twill fabrics are very important in occasions that require wear-resistant and thick fabrics due to their strength and durability; finally, satin fabrics are very important in the high-end market and specific occasions due to their luxurious appearance and feel. From an economic perspective, automated fabric defect detection technology not only improves product quality, reduces losses in the production process, and reduces labor costs, but also promotes innovation and progress in the fabric manufacturing industry. At the social level, by ensuring product quality, safeguarding consumer rights, and reducing environmental impact, fabric defect detection not only improves people's quality of life, but also lays a solid foundation for sustainable development and the popularization of science and technology. However, with the continuous expansion of industrial production scale, fabric production lines need to operate efficiently, which puts higher standards on strict supervision of fabric quality. Traditional manual inspection methods are subjective and inefficient in identifying fabric defects and can no longer meet the efficiency requirements of modern industrial production.

[0003] Industrial fabric defect detection using image processing technology has obvious advantages over traditional manual detection. First, this method can quickly process large-scale image data, significantly improving the speed of detection. Second, using computer vision technology, defects can be automatically identified through algorithms, reducing the impact of human factors on the detection results, thereby improving the accuracy and reliability of detection. In addition, the automated detection system operates around the clock, is not limited by manual working hours and fatigue, and better adapts to the needs of industrial production. However, vision-based automatic detection methods also face some challenges. The quality of industrial fabric images may be unstable due to factors such as material, color, resolution, environmental noise, and sensor differences on the production line. Different types of defects may have different characteristics, which requires the development of diversified feature extraction and classification algorithms. At the same time, for different production lines or different types of industrial fabrics, the generalization ability of the detection model also needs to be rigorously tested and verified. Therefore, when designing an industrial fabric defect detection system for a new production line, these factors must be taken into account and measures must be taken to address them to ensure the robustness and adaptability of the algorithm. In this way, the system can operate stably under various conditions and provide an efficient and accurate detection solution for modern industrial manufacturing.

[0004] At present, there have been some studies on cloth defect detection:

[0005] For example, Chinese invention patent application publication number CN114723705A discloses a method for detecting cloth defects based on image processing. This invention method includes: collecting cloth images, dividing the cloth images into multiple window images, obtaining frequency histograms of the window images, performing similarity calculation on the frequency histograms of every two window images to obtain similarity values, obtaining the average similarity value of the frequency similarity values ​​between each window image and other window images, drawing a similarity curve based on the average similarity value, obtaining the maximum value of the similarity value based on the similarity curve, determining a standard window image based on the maximum value of the similarity value, calculating the periodic conformity of the cloth texture of the position defect window image based on the standard window image, determining the defective pixel point through the periodic conformity, thereby effectively avoiding the interference of the cloth's own texture, accurately locating the defect position, and improving the accuracy of the cloth defect repair position. It has strong practicality and is worthy of promotion.

[0006] For another example, Chinese invention patent application publication number CN115457553A discloses a cloth defect detection method based on deep learning, including: enhancing the robustness of the algorithm by data enhancement of the cloth defect data set, utilizing a multi-layer feature extraction algorithm, and integrating the underlying position feature information with the high-level classification feature information through a horizontally connected top-down architecture, thereby improving the detection effect of cloth defects of different scales, adding a deformable convolution algorithm on this basis, adding a two-dimensional offset to the regular sampling position in the convolution, so that the sampling grid after the offset can be deformed, thereby solving the problem of weak detection capability of irregular-shaped cloth defects, and at the same time, using RoiAlign and Cascade Rcnn convolutional neural network to fuse, by continuously enhancing the IOU threshold, enhancing the adaptability of the defect detection algorithm in complex pattern backgrounds, reducing the erroneous detection of false positive targets, and enhancing the detection accuracy of cloth defects.

[0007] Another example is the Chinese invention patent application publication number CN116342495A, which relates to the field of image data processing technology, and specifically to a cloth defect detection method based on image processing, which includes: obtaining a surface grayscale image of the cloth to be detected, and segmenting the surface grayscale image; obtaining a set of grayscale value sequences in each preset direction in a set of preset directions for a target sub-region; determining a grayscale change abnormality index corresponding to the grayscale value sequence; determining a neighborhood difference index corresponding to the first grayscale value sequence in two adjacent grayscale value sequences; determining a grayscale continuity index in a preset direction for the target sub-region; determining an abnormal direction corresponding to the target sub-region; performing abnormality degree analysis on the target sub-region; and performing loose warp defect detection on the cloth to be detected. This invention improves the efficiency of loose warp defect detection on cloth by performing image processing on the surface grayscale image, and is applied to defect detection on cloth.

[0008] The shortcomings of the above invention patent application are that it mainly focuses on a single production line and requires a large amount of labeled data for supervised re-fine-tuning of the deep network model. It does not fully consider the diversity of the plain cloth production environment in real industrial production lines and the complexity of training deep network models from scratch. Since plain cloth involves multiple production lines and different processes, data collection and labeling become extremely difficult, which limits the scope of application of traditional supervised full-scale fine-tuning methods. In this context, this patent application introduces a defect detection method based on plain cloth quality inspection experience prompt fine-tuning, which makes up for the shortcomings of traditional methods.

[0009] First, traditional methods rely on large-scale labeled data, while data collection in cross-production environments faces greater challenges, including differences in data distribution between different production lines and subjectivity in labeling. This makes the construction of training data for traditional supervised full-scale fine-tuning methods cumbersome and costly. Secondly, the prompt-based fine-tuning method can learn prompts from a single or very few plain-weave labeled data, and can adapt to new production line tasks by updating a small number of prompt parameters, reducing the reliance on large-scale labeled data and the computational cost of full-scale fine-tuning. In addition, the introduction of plain-weave fabric quality inspection experience prompts enables prompt learning to adapt to new plain-weave fabric data more accurately and robustly, thereby improving the quality of defect detection. Therefore, the defect detection method based on plain-weave fabric quality inspection experience prompt fine-tuning makes up for the shortcomings of the traditional supervised full-scale fine-tuning method in the diverse and complex plain-weave fabric production environment, and provides a more flexible and efficient solution for the further development of the field of industrial fabric defect detection. Summary of the invention

[0010] The purpose of the present invention is to provide a defect detection method based on plain cloth quality inspection experience prompt fine-tuning, so as to solve the problems faced by the cloth defect detection method in the above-mentioned prior art, namely, the need for a large amount of labeled data for supervised full-scale fine-tuning of the deep network model, resulting in high training costs and computing resources, as well as poor generalization performance on the new production line.

[0011] The detection method of the present invention comprises the following specific steps:

[0012] Step 1, industrial fabric data collection: collect the raw data of the weaving process in different actual production environments captured by the camera installed at the edge of the water jet knitting machine;

[0013] Step 2: cloth data preprocessing and pixel-level annotation: decode the original cloth data to obtain image data in JPEG format, and then annotate the defective cloth images at the pixel level;

[0014] Step 3: Build an industrial cloth defect detection knowledge base, classify the preprocessed cloth image data by defect type, including broken warp, broken weft, hole, stained yarn, and loose edge, and store the labeled defect data and feature information in the label library to form a standardized data set;

[0015] Step 4, constructing an industrial cloth defect detection model, including constructing a plain cloth quality inspection experience module, a prompt fine-tuning network module, a black box cloth defect detection network module, and a prediction deviation correction network module;

[0016] Step 5, quality detection of plain cloth defects based on the industrial cloth defect detection model;

[0017] Step 6: Industrial fabric defect detection model training and evaluation;

[0018] Step 7, use of industrial fabric defect detection method.

[0019] Furthermore, step 4 of constructing a plain cloth quality inspection experience module, a prompt fine-tuning network module, a black box cloth defect detection network module, and a prediction deviation correction network module includes the following steps:

[0020] Step 4.1: Build a plain fabric quality inspection experience module, obtain typical defect characteristics and quality standards of plain fabric from the industrial fabric defect detection knowledge base, and form quality inspection rules and standards through historical data and expert experience. These experiences and standards will serve as initial prompts for model fine-tuning. 0 ;

[0021] Step 4.2, construct a prompt fine-tuning network module, which consists of multiple groups of prompt generation sub-networks and random projection matrices, specifically including M prompt generation sub-network modules. The specific process of each prompt generation sub-module is as follows:

[0022] From the same multidimensional Gaussian distribution Generate a noise z through the sampling algorithm:

[0023]

[0024] In the above formula (1), i represents the sampling number, Δ is the number of groups, and μ (t) represents the mean of the distribution, ∈ (t) ≥0 represents the number of steps, σ (t) represents the covariance of the distribution;

[0025] Use a random matrix A for matrix multiplication to increase the dimension of the noise z to the same dimension as the input, and get M expert prompts p 1 ,p 2 ,…,p m …,p M , as shown in formula (2):

[0026] p m =Az+p 0 ……(2),

[0027] In the above formula (2), p 0 ∈R d is the initial prompt embedding feature, is a multidimensional Gaussian distribution The features obtained by performing covariance matrix adaptive evolution strategy on is the random projection matrix;

[0028] Step 4.3: Construct a black-box fabric defect detection network module, which includes an encoder with a dilated convolution (θ e )-Decoder(θ d ) structure, multi-scale pyramid pooling network, and fully connected conditional random field network module;

[0029] Step 4.4, construct a network prediction bias correction model, including a residual network with three-layer perceptron learning prediction;

[0030] Step 4.5, initialization of each component of the industrial cloth defect detection model, including the encoder θ e and decoder θ d Initialize with the model parameters of self-supervised pre-training on the ImageNet dataset.

[0031] Furthermore, the quality detection of plain cloth defects based on the industrial cloth defect detection model described in step 5 comprises the following steps:

[0032] Step 5.1, mark the plain fabric image D from the current new production line n Randomly sample a plain cloth data X n . Where D n The expression of is shown in formula (3):

[0033]

[0034] In the above formula (3), D n Indicates the data of new pattern cloth production line, is the image data of the plain cloth production line, N n is the total number of data points of the fabric production line, (i) represents the i-th fabric data point, The height of the i-th picture of the plain cloth production line is H and the length is W. Each plain cloth picture has 3 channels. Indicates tabby picture Pixel-by-pixel annotation;

[0035] Step 5.2, input the plain picture data X n Merge with the generated M prompts to get prompt input X o , as shown in formula (4):

[0036]

[0037] In the above formula (4), p m represents the mth expert prompt embedding feature;

[0038] Step 5.3, Fusion prompt input X o Sent into a black box network model, specifically, the encoder θ of the encoder-decoder structure with dilated convolution e Then we get the deep feature representation F o , as shown in formula (5):

[0039]

[0040] In the above formula (5), θ e represents an encoder with dilated convolution, represents the feature map of the i-th layer generated by the encoder for the new plain cloth production line data, k represents the number of layers of the encoder output feature map, and the predicted logical value output value of the new plain cloth data is shown in the following formula (6):

[0041]

[0042] In the above formula (6), Lo It means that the feature map passes through the decoder θ d The predicted logical value obtained later is represents the feature map of the i-th layer generated by the encoder for the new plain cloth production line data, k represents the number of layers of the encoder output feature map, and then passes through the Softmax function to obtain the plain cloth sample pixel-level prediction output P o As shown in the following formula (7):

[0043] P o =Softmax(L o )=Softmax(θ d (θ e (X o )))……(7),

[0044] In the above formula (7), Softmax(·) represents the activation function commonly used in the model to convert logical values ​​into predictions, and L o It means that the feature map passes through the decoder θ d The predicted logical value obtained after e represents the encoder with dilated convolution, Softmax(·) is as follows (8):

[0045]

[0046] In the above formula (8), C represents the total number of categories, L o,i Indicates logical value L o The i-th element of ;

[0047] Step 5.4, network prediction bias correction, pixel-level prediction output P of the original plain cloth sample n After a three-layer perceptron learns the predicted residual, the pixel-level residual prediction output P is obtained. r , as shown in formula (9)

[0048]

[0049] In the above formula (9), Represents a three-layer perceptron residual learning network module. The residual learning module is regarded as multiple addition operations. The parameters of different layers can be optimized separately. For a three-layer perceptron network, the parameters of the first and third layers are first optimized as shown in the following formula (10):

[0050]

[0051]

[0052] In the above formula (10), f(·) represents the i-th layer network, θ i is the parameter of the i-th layer network, xi is the input of the i-th layer network.

[0053] Furthermore, the industrial cloth defect detection model training and evaluation described in step 6 includes the following steps:

[0054] Step 6.1, randomly sample a labeled new plain fabric production line sample X n Input: Fine-tuning industrial cloth defect detection model based on mixed prompts, sample X n After the expert prompt module is prompted to enter X o , X o Then send it to the encoder θ of the encoder-decoder structure with hole convolution e Then we get the deep feature representation F o , these deep features are input into the decoder θ d In the above example, the predicted logical value output L of the sample is obtained. o , and then pass the Softmax function to get the pixel-level prediction output P of the sample o Finally, a three-layer perceptron is used to obtain the pixel-level residual prediction output P r ;

[0055] Step 6.2, use pixel-level residual prediction output P r and the original pixel-level prediction output P o Calculating the defect detection loss function They are respectively as follows (11) and (12):

[0056]

[0057] In the above formulas (11) and (12), CrossEntropy(·) represents the cross entropy loss function, KL(·) represents the relative entropy loss function, and P o Indicates plain fabric picture X o The original network pixel-level prediction output, P r Represents the predicted output P o The pixel-level residual prediction output of

[0058] Step 6.3, calculate the complete loss As shown in the following formula (13):

[0059]

[0060] In the above formula (13), w represents the weight of the loss function, which is generally taken as 1.0. The stochastic gradient descent algorithm is used to back-propagate the parameters of the three-layer perceptron to update the parameters and minimize the loss function, thereby improving the prediction performance of the new production line model.

[0061] In step 6.4, the loss is calculated by repeatedly requesting the interface of the black box model as a guide for the sampling algorithm, and the parameters of multiple prompts are updated to improve the generalization of the new production line model, as shown in the following formula (14).

[0062]

[0063] In the above formula (14), CrossEntropy(·) represents the cross entropy loss function, Y n Represents sample X n Labels, given that the gradient of the black box model is unknown, in order to solve the above optimization problems, a gradient-free optimization method is adopted, namely the covariance matrix adaptive evolution strategy. The covariance matrix adaptive evolution strategy is a parameterized search distribution model using a multivariate normal distribution. In each iteration process, the covariance matrix adaptive evolution strategy generates a new query solution population by sampling from the multivariate normal distribution.

[0064] Step 6.5, repeat the above steps alternately to train the model until the maximum number of iterations is reached or the training converges;

[0065] Step 6.6, use the validation set to evaluate the model. If the model performance requirements are not met, repeat steps 6.1 to 6.5 to gradually improve the model performance;

[0066] Furthermore, the use of the industrial fabric defect detection method described in step 7 includes: inputting the newly collected plain fabric production line data into the trained industrial fabric defect detection model for defect detection, and selecting the prediction result with the highest classification confidence in the model output result as the final detection result.

[0067] Compared with the existing similar methods, the detection method of the present invention has the following advantages:

[0068] 1. The industrial fabric defect detection model included in the detection method of the present invention achieves higher resolution detection quality by adopting dense semantic segmentation technology. Unlike traditional methods that mainly focus on target detection, the detection method of the present invention uses pixel-level semantic segmentation to provide more accurate location information for detected defects, rather than just approximate areas. The detection method of the present invention enables the model to identify defects of different sizes and shapes in the image, significantly improves the detection accuracy of various defect types, and effectively reduces false detections and missed detections.

[0069] 2. The detection method of the present invention significantly reduces the labeling cost and computing cost by building an industrial cloth defect detection model. Through the prompt fine-tuning mechanism, only a small number of prompt parameters need to be updated to adapt to new data, reducing the computing resources required to retrain the entire neural network, thereby quickly improving the generalization ability of the model and ensuring that the fine-tuned defect detection model can meet actual needs in a variety of new cloth production lines.

[0070] 3. The detection method of the present invention is highly adaptable and flexible, can quickly adapt to changes in different production lines and fabric types, can process images with different lighting conditions, background noise and fabric textures, and ensure that reliable detection results can be provided in a variety of production environments. This high adaptability and flexibility makes the detection method of the present invention more practical and competitive in actual industrial applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 The figure is a schematic diagram of the process of the detection method of the present invention.

[0072] Figure 2 It is a schematic diagram of a module for fine-tuning the experience prompt for plain fabric quality inspection in the detection method of the present invention.

[0073] Figure 3 It is a schematic diagram of the framework of the detection method of the present invention. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical solution and advantages of the method of the present invention clearer, the specific embodiments of the detection method of the present invention are further described in detail below in conjunction with the accompanying drawings and related calculation formulas.

[0075] like Figure 1 As shown, the detection method comprises:

[0076] Step 1: Industrial fabric data collection. The original data captured by the camera installed at the edge of the water jet knitting machine during the weaving process under different actual production environments are collected. The data collected from different production lines are affected by many factors such as fabric material, color difference, image resolution, environmental noise, sensor difference, etc. in the production environment, and there is a data distribution deviation. The types of grey fabrics actually produced include commonly used plain, twill, satin and other textured grey fabrics, as well as broken warp, broken weft, holes, stained yarn, loose edges and other defects. The industrial fabric data collected from different production lines need to be annotated, and the annotated industrial fabric defect detection image data is recorded as is the image data of the plain cloth production line, N n is the total number of data points of the fabric production line, (i) represents the i-th fabric data point, The height of the i-th picture of the plain cloth production line is 288 and the length is 320. Each plain cloth picture has 3 channels. Indicates tabby picture Pixel-by-pixel annotation;

[0077] Step 2: cloth data preprocessing and pixel-level annotation: decode the original cloth data to obtain image data in JPEG format, and then annotate the defective cloth images at the pixel level;

[0078] Step 3: construct an industrial cloth defect detection knowledge base, classify the preprocessed cloth image data by defect type, store the labeled defect data and feature information into a label library, and form a standardized data set;

[0079] Step 4: construct an industrial cloth defect detection model, which specifically includes a plain cloth quality inspection experience module, a prompt fine-tuning network module, a black box cloth defect detection network module, and a prediction deviation correction network module. The prompt fine-tuning network module includes M prompt generation submodules, such as Figure 2 As shown, since the prompt is added on the basis of the pre-trained model, the powerful expression ability of the pre-trained model can be reused, reducing the dependence on a large amount of labeled data; at the same time, by introducing multiple task-specific expert prompts, the domain differences between the pre-trained model and the downstream tasks can be reduced, thereby improving the generalization ability of the model. The detection method of the present invention learns M global prompts at the input level in a black box optimization manner. Because the low-dimensional subspace can be as efficient as the fine-tuning of all model parameters, the detection method of the present invention further reduces the search space of the covariance matrix adaptive evolution strategy. Specifically, each prompt generation submodule is first composed of the same multidimensional Gaussian distribution The evolutionary algorithm searches for a noise z in the low-dimensional subspace, and then uses a random matrix A for matrix multiplication to increase the dimension of the noise z to the same dimension as the input, and finally obtains M expert prompts p m , input data X n Merge with multiple expert prompts to get the fused prompt input X o It is expressed as: Calculate the loss by requesting the black box model interface multiple times Given that the gradient of the black box model is unknown, in order to solve the above optimization problem, the covariance matrix adaptive evolution strategy is adopted. In each iteration, the covariance matrix adaptive evolution strategy generates a new query solution population by sampling from the multivariate normal distribution. i represents the sampling number, Δ is the number of groups, μ (t) represents the mean of the distribution, ∈ (t) ≥0 represents the number of steps, σ (t)represents the covariance of the distribution, and the parameter μ (t) ,∈ (t) , σ (t) Each iteration is updated to minimize the loss of the sampling strategy;

[0080] Based on the black box cloth defect detection network module, the detection method of the present invention is based on an encoder-decoder structure with hole convolution, wherein the encoder part uses hole convolution for downsampling, and the decoder part is used for upsampling to restore the resolution, which helps to retain more spatial details while maintaining context information. In order to better handle the spatial consistency of the segmentation results, the segmentation results are further post-processed in combination with a fully connected conditional random field, which helps to smooth the segmentation boundaries and make the results more accurate. The encoder-decoder structure model with hole convolution pre-trained on ImageNet is used as the encoder θ to extract features. e and a decoder θ that extracts dense predictions d The detection method of the present invention collects multiple features generated in the middle of the network in: f i represents the feature map extracted from the i-th layer, C i Indicates the number of channels of the feature map of the i-th layer, H i represents the height of the feature map of the i-th layer, W i Represents the width of the feature map of the i-th layer. The decoder takes the features of these contextual information of different scales as input and uses the multi-scale pyramid pooling module to fuse all features to obtain a more accurate dense feature representation. This multi-scale feature is finally output to the defect detection result through the classifier;

[0081] Step 5, based on the fine-tuning of the defect detection method based on the experience prompt of plain fabric quality inspection, this step realizes the rapid self-adaptation of a small amount of labeled data on the new production line, such as Figure 3 As shown, the new plain fabric production line has marked data Input to encoder θ with dilated convolution e In the above example, we obtain multi-scale features Then, after the multi-scale pyramid pooling decoder θ d Get the logical value output of the pixel-level prediction of the sample: After post-processing with the fully connected conditional random field and the Softmax function, the sample phase pixel-level prediction output can be obtained: Finally, the original prediction output Through a three-layer perceptron network Plus the original prediction output Get the corrected prediction output Correction output and the original prediction output The defect detection loss functions are calculated as follows: as well as The final complete loss function w represents the weight of the loss function on the actively marked samples of the new production line, which is usually set to 1.0. The stochastic gradient descent algorithm is used to Perform back propagation to update model parameters and minimize the loss function to improve the generalization and performance of the new production line;

[0082] Step 6, industrial fabric defect detection model training and evaluation, by repeating steps 3 and 4 until the maximum number of iterations is reached or the model performance meets the requirements. Based on the usage of the industrial fabric defect model, the fabric image data collected in real time on the new production line is input into the trained industrial fabric defect detection model for defect detection, and the prediction category with the largest classification confidence in the model output result is selected as the final detection result.

[0083] Step 7, using the industrial fabric defect detection method, includes: inputting the newly collected plain fabric production line data into the trained industrial fabric defect detection model for defect detection, and selecting the prediction result with the largest classification confidence in the model output result as the final detection result.

[0084] The present invention is not limited by the above-mentioned embodiments. The above-mentioned embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which shall all fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims.

Claims

1. A defect detection method based on fine-tuning of plain fabric quality inspection experience prompts, the detection method comprising the steps of: Step 1: Industrial fabric data collection, collecting raw data from the weaving process in different actual production environments captured by a camera installed at the edge of the water jet knitting machine; Step 2: cloth data preprocessing and pixel-level annotation: decode the original cloth data to obtain image data in JPEG format, and then annotate the defective cloth images at the pixel level; Step 3: Build an industrial cloth defect detection knowledge base, classify the preprocessed cloth image data by defect type, including broken warp, broken weft, hole, stained yarn, and loose edge, and store the labeled defect data and feature information in the label library to form a standardized data set; Step 4, constructing an industrial cloth defect detection model, including constructing a plain cloth quality inspection experience module, a prompt fine-tuning network module, a black box cloth defect detection network module, and a prediction deviation correction network module; Step 5, quality detection of plain cloth defects based on the industrial cloth defect detection model; Step 6: Industrial fabric defect detection model training and evaluation; Step 7, use of industrial fabric defect detection method.

2. According to the detection method of claim 1, step 4 of constructing a plain cloth quality inspection experience module, a prompt fine-tuning network module, a black box cloth defect detection network module, and a prediction deviation correction network module comprises the following steps: Step 4.1, build a plain fabric quality inspection experience module, obtain the typical defect characteristics and quality standards of plain fabric from the industrial fabric defect detection knowledge base, and form quality inspection rules and standards through historical data and expert experience. These experiences and standards will serve as the initial prompt p0 for model fine-tuning; Step 4.2, construct a prompt fine-tuning network module, which consists of multiple groups of prompt generation sub-networks and random projection matrices, specifically including M prompt generation sub-network modules. The specific process of each prompt generation sub-module is as follows: From the same multidimensional Gaussian distribution Generate a noise z through the sampling algorithm: In the above formula (1), i represents the sampling number, Δ is the number of groups, and μ (t) represents the mean of the distribution, ∈ (t) ≥0 represents the number of steps, σ (t) represents the covariance of the distribution; Use a random matrix A for matrix multiplication to increase the noise z dimension to the same dimension as the input, and get M expert prompts p1, p2, ..., p m …,p M , as shown in formula (2): p m =Az+p0……(2), In the above formula (2), p0∈R d is the initial prompt embedding feature, is a multidimensional Gaussian distribution The features obtained by performing covariance matrix adaptive evolution strategy on is the random projection matrix; Step 4.3: Construct a black-box fabric defect detection network module, which includes an encoder with a dilated convolution (θ e )-Decoder(θ d ) structure, multi-scale pyramid pooling network, and fully connected conditional random field network module; Step 4.4, construct a network prediction bias correction model, including a residual network with three-layer perceptron learning prediction; Step 4.5, initialization of each component of the industrial cloth defect detection model, including the encoder θ e and decoder θ d Initialize with the model parameters of self-supervised pre-training on the ImageNet dataset.

3. According to the detection method of claim 1, the quality detection of plain cloth defects based on the industrial cloth defect detection model in step 5 comprises the following steps: Step 5.1, mark the plain fabric image D from the current new production line n Randomly sample a plain cloth data X n . Where D n The expression of is shown in formula (3): In the above formula (3), D n Indicates the data of new pattern cloth production line, is the image data of the plain cloth production line, N n is the total number of data points of the fabric production line, (i) represents the i-th fabric data point, The height of the i-th picture of the plain cloth production line is H and the length is W. Each plain cloth picture has 3 channels. Indicates tabby picture Pixel-by-pixel annotation; Step 5.2, input the plain picture data X n Merge with the generated M prompts to get prompt input X o , as shown in formula (4): In the above formula (4), p m represents the mth expert prompt embedding feature; Step 5.3, Fusion prompt input X o Sent into a black box network model, specifically, the encoder θ of the encoder-decoder structure with dilated convolution e Then we get the deep feature representation F o , as shown in formula (5): In the above formula (5), θ e represents an encoder with dilated convolution, represents the feature map of the i-th layer generated by the encoder for the new plain cloth production line data, k represents the number of layers of the encoder output feature map, and the predicted logical value output value of the new plain cloth data is shown in the following formula (6): In the above formula (6), L o It means that the feature map passes through the decoder θ d The predicted logical value obtained later is represents the feature map of the i-th layer generated by the encoder for the new plain cloth production line data, k represents the number of layers of the encoder output feature map, and then passes through the Softmax function to obtain the plain cloth sample pixel-level prediction output P o As shown in the following formula (7): P o =Softmax(L o )=Softmax(θ d (θ e (X o )))……(7), In the above formula (7), Softmax(·) represents the activation function commonly used in the model to convert logical values ​​into predictions, and L o It means that the feature map passes through the decoder θ d The predicted logical value obtained after e represents the encoder with dilated convolution, Softmax(·) is as follows (8): In the above formula (8), C represents the total number of categories, L o,i Indicates logical value L o The i-th element of ; Step 5.4, network prediction bias correction, pixel-level prediction output P of the original plain cloth sample n After a three-layer perceptron learns the predicted residual, the pixel-level residual prediction output P is obtained. r , as shown in formula (9) In the above formula (9), Represents a three-layer perceptron residual learning network module. The residual learning module is regarded as multiple addition operations. The parameters of different layers can be optimized separately. For a three-layer perceptron network, the parameters of the first and third layers are optimized first, which is expressed as the following formula (10): In the above formula (10), f(·) represents the i-th layer network, θ i is the parameter of the i-th layer network, x i is the input of the i-th layer network.

4. According to the detection method of claim 1, the industrial cloth defect detection model training and evaluation in step 6 comprises the following steps: Step 6.1, randomly sample a labeled new plain fabric production line sample X n Input: Fine-tuning industrial cloth defect detection model based on mixed prompts, sample X n After the expert prompt module is prompted to enter X o , X o Then send it to the encoder θ of the encoder-decoder structure with hole convolution e Then we get the deep feature representation F o , these deep features are input into the decoder θ d In the above example, the predicted logical value output L of the sample is obtained. o , and then pass the Softmax function to get the pixel-level prediction output P of the sample o Finally, a three-layer perceptron is used to obtain the pixel-level residual prediction output P r ; Step 6.2, use pixel-level residual prediction output P r and the original pixel-level prediction output P o Calculating the defect detection loss function They are respectively as follows (11) and (12): In the above formulas (11) and (12), CrossEntropy(·) represents the cross entropy loss function, Kl(·) represents the relative entropy loss function, and P o Indicates plain fabric picture X o The original network pixel-level prediction output, P r Represents the predicted output P o The pixel-level residual prediction output of Step 6.3, calculate the complete loss As shown in the following formula (13): In the above formula (13), w represents the weight of the loss function, which is generally taken as 1.

0. The stochastic gradient descent algorithm is used to back-propagate the parameters of the three-layer perceptron to update the parameters and minimize the loss function, thereby improving the prediction performance of the new production line model. In step 6.4, the loss is calculated by repeatedly requesting the interface of the black box model as a guide for the sampling algorithm, and the parameters of multiple prompts are updated to improve the generalization of the new production line model, as shown in the following formula (14). In the above formula (14), CrossEntropy(·) represents the cross entropy loss function, Y n Represents sample X n Labels, given that the gradient of the black box model is unknown, in order to solve the above optimization problems, a gradient-free optimization method is adopted, namely the covariance matrix adaptive evolution strategy. The covariance matrix adaptive evolution strategy is a parameterized search distribution model using a multivariate normal distribution. In each iteration process, the covariance matrix adaptive evolution strategy generates a new query solution population by sampling from the multivariate normal distribution. Step 6.5, repeat the above steps alternately to train the model until the maximum number of iterations is reached or the training converges; In step 6.6, use the validation set to evaluate the model. If the model performance requirements are not met, repeat steps 6.1 to 6.5 to gradually improve the model performance.

5. According to the detection method of claim 1, the use of the industrial cloth defect detection method in step 7 comprises: The newly collected plain fabric production line data is input into the trained industrial fabric defect detection model for defect detection, and the prediction result with the highest classification confidence in the model output is selected as the final detection result.

Images