A defect detection method based on plain cloth quality inspection experience prompt fine tuning

Abstract

The present application relates to a kind of based on plain cloth quality inspection experience prompt micro-adjustment defect detection method, the detection method includes: step 1, industrial cloth data collection is recorded;Step 2, cloth data pre-processing and pixel-level annotation;Step 3, construct industrial cloth defect detection knowledge base set;Step 4, construct industrial cloth defect detection model;Step 5, based on the quality detection of plain cloth defect of industrial cloth defect detection model;Step 6, industrial cloth defect detection model training and evaluation;Step 7, the use of industrial cloth defect detection method.The detection method has the superior effect that the detection method includes the industrial cloth defect detection model by using dense semantic segmentation technology, higher resolution detection quality is realized;Using pixel-level semantic segmentation, more accurate position information is provided for the detected defect;By setting the construction of industrial cloth defect detection model, significantly reduce the annotation cost and calculation cost.

Description

Technical Field

[0001] This invention relates to the field of industrial textile technology, and specifically to a defect detection method based on fine-tuning of plain weave fabric quality inspection experience prompts. Background Technology

[0002] In contemporary society, fabric is an indispensable material in people's daily lives, and its quality directly affects their user experience and standard of living. In actual production, plain weave, twill weave, and satin weave each have their own importance and uses. First, plain weave fabric has the simplest structure and is widely used in everyday clothing (such as shirts, bed sheets, tablecloths, etc.) as well as decorative fabrics. Second, twill weave fabric, due to its strength and durability, is very important in situations requiring wear-resistant and thick fabrics. Finally, satin weave fabric, due to its luxurious appearance and feel, is very important in the high-end market and for specific occasions. From an economic perspective, automated fabric defect detection technology not only improves product quality, reduces losses in the production process, and lowers labor costs, but also promotes innovation and progress in the fabric manufacturing industry. At the social level, by ensuring product quality, protecting consumer rights, and reducing environmental impact, fabric defect detection not only improves people's quality of life but also lays a solid foundation for sustainable development and the popularization of technology. However, with the continuous expansion of industrial production scale, fabric production lines need to operate efficiently, which places higher standards on the strict supervision of fabric quality. Traditional manual inspection methods are subjective and inefficient in identifying fabric defects, and can no longer meet the efficiency requirements of modern industrial production.

[0003] Compared to traditional manual inspection, image processing technology offers significant advantages for industrial fabric defect detection. Firstly, it can rapidly process large-scale image data, significantly increasing inspection speed. Secondly, utilizing computer vision technology, defects can be automatically identified through algorithms, reducing the impact of human factors on inspection results and thus improving accuracy and reliability. Furthermore, automated inspection systems operate 24 / 7, unaffected by human working hours and fatigue, better adapting to the needs of industrial production. However, vision-based automated inspection methods also face several challenges. The quality of industrial fabric images can be unstable due to factors such as material, color, resolution, environmental noise, and sensor differences on the production line. Different types of defects may have different characteristics, necessitating the development of diverse feature extraction and classification algorithms. Simultaneously, the generalization ability of the detection model needs rigorous testing and validation for different production lines or different types of industrial fabrics. Therefore, when designing an industrial fabric defect detection system for a new production line, these factors must be considered, and measures must be taken to address them, ensuring the robustness and adaptability of the algorithm. This ensures stable operation of the system under various conditions, providing an efficient and accurate inspection solution for modern industrial manufacturing.

[0004] Some research has already been done on fabric defect detection:

[0005] For example, Chinese invention patent application publication number CN114723705A discloses a method for detecting fabric defects based on image processing. This invention method includes: acquiring a fabric image, segmenting the fabric image into multiple window images, obtaining a frequency histogram of the window images, calculating the similarity between the frequency histograms of every two window images to obtain a similarity value, obtaining the average similarity value between the frequency similarity values ​​of each window image and other window images, plotting a similarity curve based on the average similarity value, obtaining the maximum value of the similarity value based on the similarity curve, determining a standard window image based on the maximum value of the similarity value, calculating the periodicity matching rate of the fabric texture of the defect window image based on the standard window image, and determining the defect pixel point through the periodicity matching rate. This effectively avoids interference from the fabric's own texture, accurately locates the defect position, improves the accuracy of fabric defect repair position, has strong practicality, and is worth promoting.

[0006] For example, Chinese invention patent application publication number CN115457553A discloses a deep learning-based fabric defect detection method, which includes: enhancing the robustness of the algorithm by augmenting the fabric defect dataset; using a multi-layer feature extraction algorithm to fuse low-level positional feature information with high-level classification feature information through a horizontally connected top-down architecture, thereby improving the detection effect of fabric defects at different scales; adding a deformable convolution algorithm to the regular sampling positions in the convolution, allowing the sampling grid after the offset to deform, thus solving the problem of weak detection capability for irregularly shaped fabric defects; and using RoiAlign and Cascade R-CNN convolutional neural networks to enhance the adaptability of the defect detection algorithm to complex pattern backgrounds by continuously increasing the IOU threshold, reducing false positive detections and improving the detection accuracy of fabric defects.

[0007] For example, Chinese invention patent application publication number CN116342495A relates to the field of image data processing technology, specifically to a fabric defect detection method based on image processing. This method includes: acquiring a grayscale image of the surface of the fabric to be detected; segmenting the grayscale image; acquiring a set of grayscale value sequences for each preset direction in a preset direction set for the target sub-region; determining anomaly indicators for grayscale changes corresponding to the grayscale value sequences; determining a neighborhood difference indicator corresponding to the first grayscale value sequence in two adjacent grayscale value sequences; determining a grayscale continuity indicator for the target sub-region in the preset direction; determining the abnormal direction corresponding to the target sub-region; performing anomaly degree analysis processing on the target sub-region; and detecting warp looseness defects in the fabric to be detected. This invention improves the efficiency of detecting warp looseness defects in fabric by processing the surface grayscale image and is applicable to fabric defect detection.

[0008] The shortcomings of the aforementioned patent applications lie in their primary focus on a single production line and the need for extensive labeled data for supervised full-scale retuning of the deep network model. They fail to adequately consider the diversity of plain weave fabric production environments in real industrial production lines and the complexity of training deep network models from scratch. Because plain weave fabric involves multiple production lines and different processes, data collection and labeling become exceptionally difficult, thus limiting the application scope of traditional supervised full-scale retuning methods. Against this backdrop, this patent application introduces a defect detection method based on plain weave fabric quality inspection experience-based prompts for retuning, overcoming the shortcomings of traditional methods.

[0009] First, traditional methods rely on large-scale labeled data, while data collection across production environments faces greater challenges, including differences in data distribution between different production lines and the subjectivity of labeling. This makes the construction of training data for traditional supervised full-scale fine-tuning methods cumbersome and costly. Second, cue-based fine-tuning methods can learn cues from single or very few plain weave labeled data sets, and can adapt to new production line tasks by updating only a small number of cue parameters, reducing the dependence on large-scale labeled data and the computational cost of full-scale fine-tuning. Furthermore, introducing plain weave fabric quality inspection experience cues allows the cue learning to adapt to new plain weave fabric data more accurately and robustly, improving defect detection quality. Therefore, the defect detection method based on plain weave fabric quality inspection experience cues compensates for the shortcomings of traditional supervised full-scale fine-tuning methods in the diverse and complex plain weave fabric production environments, providing a more flexible and efficient solution for the further development of industrial fabric defect detection. Summary of the Invention

[0010] The purpose of this invention is to provide a defect detection method based on fine-tuning of plain weave fabric quality inspection experience prompts, in order to solve the problems faced by the existing fabric defect detection methods, which require a large amount of labeled data for supervised full-scale fine-tuning of deep network models, resulting in high training costs and computational resources, as well as poor generalization performance on new production lines.

[0011] The detection method of the present invention includes the following specific steps:

[0012] Step 1, Industrial Fabric Data Collection: Collect raw data captured by cameras installed at the edge of the water jet knitting machine during the weaving process under different actual production environments;

[0013] Step 2, Fabric data preprocessing and pixel-level annotation: Decode the original fabric data to obtain JPEG format image data, and then perform pixel-level annotation on the defective fabric images.

[0014] Step 3: Construct an industrial fabric defect detection knowledge base. Classify the pre-processed fabric image data according to defect types, including warp breaks, weft breaks, holes, stained yarns, and frayed edges. Store the labeled defect data and feature information in a label library to form a standardized dataset.

[0015] Step 4: Construct an industrial fabric defect detection model, including a plain weave fabric quality inspection experience module, a prompt fine-tuning network module, a black box fabric defect detection network module, and a prediction bias correction network module.

[0016] Step 5: Quality inspection of plain weave fabric defects based on industrial fabric defect detection model;

[0017] Step 6: Training and evaluation of the industrial fabric defect detection model;

[0018] Step 7, Application of industrial fabric defect detection methods.

[0019] Furthermore, step 4, which involves constructing the plain weave fabric quality inspection experience module, the prompt fine-tuning network module, the black box fabric defect detection network module, and the prediction deviation correction network module, includes the following steps:

[0020] Step 4.1: Construct a plain weave fabric quality inspection experience module. Obtain typical defect characteristics and quality standards of plain weave fabric from the industrial fabric defect detection knowledge base. Form quality inspection rules and standards through historical data and expert experience. These experiences and standards will serve as the initial prompt p0 for model fine-tuning.

[0021] Step 4.2: Construct the prompt fine-tuning network module, which consists of multiple prompt generation sub-networks and a random projection matrix. Specifically, it includes M prompt generation sub-network modules. The specific process of each prompt generation sub-module is as follows:

[0022] From the same multidimensional Gaussian distribution A noise z is generated using a sampling algorithm:

[0023]

[0024] In equation (1) above, i represents the sampling sequence number, Δ is the population size, and μ (t) This represents the mean of the distribution, ∈ (t) ≥0 represents the number of steps, σ (t) This represents the covariance of the distribution;

[0025] Using a random matrix A, matrix multiplication is performed to increase the dimension of the noise z to the same dimension as the input, resulting in M ​​expert cues p1, p2, ..., p m …,p M As shown in equation (2):

[0026] p m =Az+p0……(2),

[0027] In equation (2) above, p0∈R d These are the initial prompt embedding features. It is a multidimensional Gaussian distribution The features obtained in the covariance matrix adaptive evolution strategy are used. It is a random projection matrix;

[0028] Step 4.3: Construct a black-box fabric defect detection network module, specifically including an encoder with dilated convolution (θ). e )-decoder(θ d Structure, multi-scale pyramid pooling network, and fully connected conditional random field network modules;

[0029] Step 4.4: Construct a network prediction bias correction model, including a three-layer perceptron learning prediction residual network;

[0030] Step 4.5: Initialize the components of the industrial fabric defect detection model. Initialization includes initializing the encoder θ. e and decoder θ d Initialize using the parameters of a self-supervised pre-trained model on the ImageNet dataset.

[0031] Furthermore, the quality inspection of plain weave fabric defects based on the industrial fabric defect detection model described in step 5 includes the following steps:

[0032] Step 5.1, from the current new production line plain weave fabric labeled image D n A random sample of plain weave fabric data X n Dn The representation is shown in equation (3):

[0033]

[0034] In equation (3) above, D n This indicates data from the new fabric production line. These are image data of a plain weave fabric production line, N n This represents the total number of data points from the woven fabric production line, where (i) represents the i-th woven fabric data point. This indicates that the i-th image in the plain weave production line has a height of H and a length of W, and each plain weave image has 3 channels. Plain weave image Pixel-by-pixel annotation;

[0035] Step 5.2, input plain texture image data X n The input X is obtained by fusing the generated M prompts. o As shown in equation (4):

[0036]

[0037] In equation (4) above, p m This represents the embedding feature of the m-th expert prompt;

[0038] Step 5.3, the prompt to merge is to input X. o A black-box network model is fed in; specifically, it passes through an encoder-decoder structure with dilated convolutions. e The deep feature representation F is then obtained. o As shown in equation (5):

[0039]

[0040] In equation (5) above, θ e This indicates an encoder with dilated convolution. Let k represent the feature map of the i-th layer generated by the encoder from the new plain weave fabric production line data, and k represent the layer number of the encoder output feature map. The predicted logical value output of the new plain weave fabric data is shown in the following formula (6):

[0041]

[0042] In equation (6) above, L o This indicates that the feature map has been decoded by the decoder θ d The predicted logical value obtained afterwards The i-th layer feature map generated by the encoder from the data of the new plain weave fabric production line, where k represents the layer number of the encoder output feature map, is then passed through the Softmax function to obtain the pixel-level prediction output P of the plain weave fabric sample. oAs shown in equation (7):

[0043] P o =Softmax(L o ) = Softmax(θ d (θ e (X o )))……(7),

[0044] In equation (7) above, Softmax(·) represents the activation function commonly used in the model to convert logistic values ​​into predictions, L o This indicates that the feature map has been decoded by the decoder θ d The resulting predicted logic value, θ e The encoder with dilated convolution is Softmax(·) as shown in equation (8):

[0045]

[0046] In equation (8) above, C represents the total number of categories, and L o,i Represents the logical value L o The i-th element;

[0047] Step 5.4, bias correction of network prediction, pixel-level prediction output P of the original plain weave fabric sample. n After learning the predicted residuals using a three-layer perceptron, pixel-level residual prediction output P is obtained. r As shown in equation (9)

[0048]

[0049] In equation (9) above, Let represent the residual learning network module of a three-layer perceptron. The residual learning module is regarded as multiple summation operations. The parameters of different layers can be optimized separately. For a three-layer perceptron network, the parameters of the first and third layers are optimized first, as expressed in the following formula (10):

[0050]

[0051]

[0052] In equation (10) above, f(·) represents the i-th layer of the network, θ i These are the parameters of the i-th layer of the network, x i It is the input of the i-th layer of the network.

[0053] Furthermore, the training and evaluation of the industrial fabric defect detection model described in step 6 includes the following steps:

[0054] Step 6.1: Randomly sample a labeled new plain weave fabric production line sample X. nInput is a fine-tuned industrial fabric defect detection model based on a hybrid prompt, sample X n The expert prompt module prompts for input X. o X o Then it is fed into the encoder θ with a dilated convolution encoder-decoder structure. e The deep feature representation F is then obtained. o These deep features are input into the decoder θ d In the process, the predicted logical value output L of the sample is obtained. o Then, the pixel-level prediction output P of the sample is obtained through the Softmax function. o Finally, a three-layer perceptron is used to obtain the pixel-level residual prediction output P. r ;

[0055] Step 6.2, use pixel-level residual prediction to output P r And the original pixel-level prediction output P o Calculate the defect detection loss function The following equations (11) and (12) are respectively:

[0056]

[0057] In equations (11) and (12) above, CrossEntropy(·) represents the cross-entropy loss function, KL(·) represents the relative entropy loss function, and P o Image X representing plain weave fabric o The original network pixel-level prediction output, P r Indicates the predicted output P o Pixel-level residual prediction output;

[0058] Step 6.3, calculate the complete loss. As shown in equation (13):

[0059]

[0060] In the above equation (13), w represents the weight of the loss function, which is generally taken as 1.0. The stochastic gradient descent algorithm is used to backpropagate the loss function to update the parameters of the three-layer perceptron and minimize the loss function to improve the prediction performance of the new production line model.

[0061] Step 6.4: Calculate the loss by repeatedly requesting the interface of the black box model as a guide for the sampling algorithm, and update the parameters of multiple prompts to improve the generalization of the new production line model, as shown in Equation (14).

[0062]

[0063] In equation (14) above, CrossEntropy(·) represents the cross-entropy loss function, Y n Indicates sample X n Given the unknown gradient of the black-box model, in order to solve the problems of the above optimization, a gradient-free optimization method is adopted, namely the covariance matrix adaptive evolution strategy. The covariance matrix adaptive evolution strategy is a parameterized search distribution model that uses a multivariate normal distribution. In each iteration, the covariance matrix adaptive evolution strategy generates a new group of query solutions by sampling from the multivariate normal distribution.

[0064] Step 6.5: Repeat the above steps alternately to train the model until the maximum number of iterations is reached or the training converges.

[0065] Step 6.6: Evaluate the model using the validation set. If the model performance requirements are not met, repeat steps 6.1 to 6.5 to gradually improve the model performance.

[0066] Furthermore, the use of the industrial fabric defect detection method described in step 7 includes: inputting the newly collected data from the new plain weave fabric production line into the trained industrial fabric defect detection model for defect detection, and selecting the prediction result with the highest classification confidence in the model output as the final detection result.

[0067] Compared with existing similar methods, the detection method described in this invention has the following advantages:

[0068] 1. The industrial fabric defect detection model included in the detection method of the present invention achieves higher resolution detection quality by employing dense semantic segmentation technology. Unlike traditional methods that mainly focus on target detection, the detection method of the present invention uses pixel-level semantic segmentation to provide more accurate location information for detected defects, rather than just a general area. The detection method of the present invention enables the model to identify defects of different sizes and shapes in the image, significantly improving the detection accuracy for various defect types and effectively reducing false detections and missed detections.

[0069] 2. The detection method described in this invention significantly reduces annotation and computation costs by constructing an industrial fabric defect detection model. Through a prompt fine-tuning mechanism, it can adapt to new data by updating only a small number of prompt parameters, reducing the computational resources required to retrain the entire neural network. This rapidly improves the model's generalization ability and ensures that the fine-tuned defect detection model can meet the actual needs in various new fabric production lines.

[0070] 3. The detection method described in this invention has high adaptability and flexibility, and can quickly adapt to changes in different production lines and fabric types. It can process images with different lighting conditions, background noise, and fabric textures, ensuring reliable detection results in diverse production environments. This high adaptability and flexibility makes the detection method described in this invention more practical and competitive in actual industrial applications. Attached Figure Description

[0071] Figure 1 This is a schematic flowchart of the detection method described in this invention.

[0072] Figure 2 This is a schematic diagram of the plain weave fabric quality inspection experience prompt fine-tuning module of the detection method described in this invention.

[0073] Figure 3 This is a schematic diagram of the detection method described in this invention. Detailed Implementation

[0074] To make the objectives, technical solutions, and advantages of the method described in this invention clearer, the specific embodiments of the detection method described in this invention will be further described in detail below with reference to the accompanying drawings and relevant calculation formulas.

[0075] like Figure 1 As shown, the detection method includes:

[0076] Step 1: Industrial Fabric Data Collection. This involves collecting raw data captured by cameras installed at the edge of the water-jet knitting machine during the weaving process under different actual production environments. Data collected from different production lines is affected by various factors such as fabric material, color differences, image resolution, environmental noise, and sensor differences, resulting in data distribution discrepancies. The actual produced fabric types include commonly used plain weave, twill, and satin weaves, as well as defects such as broken warp and weft threads, holes, stained yarns, and frayed edges. The collected industrial fabric data from different production lines needs to be labeled. The labeled industrial fabric defect detection image data is denoted as... These are image data of a plain weave fabric production line, N n This represents the total number of data points from the woven fabric production line, where (i) represents the i-th woven fabric data point. This indicates that the i-th image in the plain weave production line has a height of 288 and a length of 320, and each plain weave image has 3 channels. Plain weave image Pixel-by-pixel annotation;

[0077] Step 2, Fabric data preprocessing and pixel-level annotation: Decode the original fabric data to obtain JPEG format image data, and then perform pixel-level annotation on the defective fabric images.

[0078] Step 3: Construct an industrial fabric defect detection knowledge base. Classify the preprocessed fabric image data according to defect type, and store the labeled defect data and feature information into a label library to form a standardized dataset.

[0079] Step 4: Construct an industrial fabric defect detection model, specifically including a plain weave fabric quality inspection experience module, a prompt fine-tuning network module, a black-box fabric defect detection network module, and a prediction bias correction network module. The prompt fine-tuning network module includes M prompt generation sub-modules, such as... Figure 2 As shown, since the prompts are added on top of the pre-trained model, the powerful expressive capabilities of the pre-trained model can be reused, reducing the dependence on a large amount of labeled data. Simultaneously, by introducing multiple task-specific expert prompts, the domain difference between the pre-trained model and the downstream task can be reduced, thereby improving the model's generalization ability. The detection method of this invention learns M global prompts at the input level using a black-box optimization approach. Because low-dimensional subspaces are as efficient as fine-tuning the full parameters of the model, the detection method of this invention further reduces the search space of the covariance matrix adaptive evolution strategy. Specifically, each prompt generation submodule is first generated by the same multidimensional Gaussian distribution. An evolutionary algorithm is used to search for a noise z in a low-dimensional subspace. Then, a matrix multiplication is performed using a random matrix A to increase the dimensionality of the noise z to match that of the input, ultimately yielding M expert cue symbols p. m Input data X n The input prompt X is obtained by merging multiple expert prompts. o Represented as: Calculate the loss by making multiple requests to the black-box model's interface. Given the unknown gradient of the black-box model, to solve the above optimization problem, an adaptive evolution strategy based on the covariance matrix is ​​adopted. In each iteration, this strategy generates a new group of query solutions by sampling from a multivariate normal distribution. i represents the sampling sequence number, Δ is the population size, and μ (t) Represents the mean of the distribution, ∈ (t) ≥0 represents the number of steps, σ (t) It represents the covariance of the distribution, with parameter μ. (t) ,∈ (t) , σ (t) Each iteration updates the sampling strategy to minimize its loss.

[0080] Based on a black-box fabric defect detection network module, the detection method of this invention utilizes an encoder-decoder structure with dilated convolutions. The encoder part uses dilated convolutions for downsampling, while the decoder part uses upsampling to restore resolution. This helps retain more spatial details while maintaining contextual information. To better handle the spatial consistency of the segmentation results, a fully connected conditional random field (CRF) is further incorporated for post-processing. The CRF helps smooth segmentation boundaries, making the results more accurate. A pre-trained encoder-decoder structure model with dilated convolutions on ImageNet is used as the encoder θ for feature extraction. e and the decoder θ that extracts dense prediction results d The detection method described in this invention collects multiple features generated in the middle of the network. in: f i C represents the feature map extracted from the i-th layer. i H represents the number of channels in the feature map of the i-th layer. i W represents the height of the feature map at layer i. i The width of the i-th layer feature map is represented by the multi-scale pyramid pooling module. The decoder takes these features of contextual information at different scales as input and uses the multi-scale feature map pooling module to fuse all features to obtain a more accurate dense feature representation. This multi-scale feature is finally passed to the classifier to complete the output of the defect detection result.

[0081] Step 5, a defect detection method fine-tuned based on plain weave fabric quality inspection experience prompts. This step achieves rapid self-adaptation of a small amount of labeled data on the new production line, such as... Figure 3 As shown, the data marked on the new plain weave fabric production line. Input to encoder θ with dilated convolution e In this process, multi-scale features are obtained. Then it passes through a multi-scale pyramid pooling decoder θ d The output of the pixel-level predicted logical values ​​for the sample is obtained: After post-processing with a fully connected conditional random field and the Softmax function, pixel-level prediction outputs for the sample phase can be obtained: Finally, the original prediction output After a three-layer perceptron network Add the original prediction output Obtain the corrected prediction output Correction output and the original prediction output The defect detection loss functions are calculated as follows: as well as Final complete loss function w represents the weight of the loss function on the actively labeled samples of the new production line, typically set to 1.0, and is applied using the stochastic gradient descent algorithm. Perform backpropagation to update model parameters and minimize the loss function to improve the generalization and performance of the new production line;

[0082] Step 6: Training and evaluation of the industrial fabric defect detection model. By repeating steps 3 and 4 until the maximum number of iterations is reached or the model performance meets the requirements, based on the usage of the industrial fabric defect model, the fabric image data collected in real time on the new production line is input into the trained industrial fabric defect detection model for defect detection. The predicted category with the highest classification confidence in the model output results is selected as the final detection result.

[0083] Step 7, the use of industrial fabric defect detection methods, includes: inputting the newly collected data from the new plain weave fabric production line into the trained industrial fabric defect detection model for defect detection, and selecting the prediction result with the highest classification confidence in the model output as the final detection result.

[0084] This invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope. All such changes and modifications fall within the scope of the invention as defined by the appended claims.

Claims

1. A defect detection method based on fine-tuning of plain weave fabric quality inspection experience prompts, the detection method comprising the following steps: Step 1, Industrial Fabric Data Collection: Collect raw data captured by cameras installed at the edge of the water jet knitting machine during the weaving process under different actual production environments. Step 2, Fabric data preprocessing and pixel-level annotation: Decode the original fabric data to obtain JPEG format image data, and then perform pixel-level annotation on the defective fabric images. Step 3: Construct an industrial fabric defect detection knowledge base. Classify the pre-processed fabric image data according to defect types, including warp breaks, weft breaks, holes, stained yarns, and frayed edges. Store the labeled defect data and feature information in a label library to form a standardized dataset. Step 4: Construct an industrial fabric defect detection model, including building a plain weave fabric quality inspection experience module, a prompt fine-tuning network module, a black-box fabric defect detection network module, and a prediction bias correction network module, comprising the following steps: Step 4.1: Construct a plain weave fabric quality inspection experience module. Obtain typical defect characteristics and quality standards for plain weave fabrics from the industrial fabric defect detection knowledge base. Develop quality inspection rules and standards using historical data and expert experience. These experiences and standards will serve as initial prompts for model fine-tuning. ; Step 4.2: Construct the prompt fine-tuning network module, which consists of multiple prompt generation sub-networks and a random projection matrix, specifically including... Each prompt generates a sub-network module. The specific process for generating a sub-module for each prompt is as follows: From the same multidimensional Gaussian distribution A noise is generated using a sampling algorithm. : ……（1）， In the above formula (1), This indicates the sampling sequence number. It is the size of the group. It represents the mean of the distribution. This represents the number of steps. This represents the covariance of the distribution; Use a random matrix Perform matrix multiplication to remove noise. The dimension is increased to the same dimension as the input, resulting in Expert Tips As shown in equation (2): ……（2）， In the above formula (2), These are the initial prompt embedding features. It is a multidimensional Gaussian distribution The features obtained in the covariance matrix adaptive evolution strategy are used. It is a random projection matrix; Step 4.3: Construct a black-box fabric defect detection network module, specifically including an encoder with dilated convolutions. -decoder Structure, multi-scale pyramid pooling network, and fully connected conditional random field network modules; Step 4.4: Construct a network prediction bias correction model, including a three-layer perceptron learning prediction residual network; Step 4.5: Initialize the components of the industrial fabric defect detection model. Initialization includes the encoder. and decoder Initialize using the parameters of a self-supervised pre-trained model on the ImageNet dataset; Step 5: Quality inspection of plain weave fabric defects based on industrial fabric defect detection model; Step 6: Training and evaluation of the industrial fabric defect detection model; Step 7, Application of industrial fabric defect detection methods.

2. According to the detection method of claim 1, step 5, the quality detection of plain weave fabric defects based on the industrial fabric defect detection model, includes the following steps: Step 5.1, label the images of plain weave fabrics from the current new production line. Randomly sample a plain weave fabric data ,in The representation is shown in equation (3): ……（3）， In the above formula (3), This indicates data from the new fabric production line. These are image data of a plain weave fabric production line. This is the total number of data from the fabric production line. Indicates the first Data points for each fabric pattern. Indicates the production line number of patterned fabrics. The height of the picture is And its length is Each plain-textured image has 3 channels. Plain weave image Pixel-by-pixel annotation; Step 5.2, input plain texture image data With the generated The prompts are merged to obtain the input prompt. As shown in equation (4): ……（4）， In the above formula (4), Indicates the first Each expert prompt embeds a feature; Step 5.3, Input prompts for fusion A black-box network model is fed in; specifically, it passes through an encoder-decoder structure with dilated convolutions. Deep feature representation is then obtained. As shown in equation (5): ……（5）， In the above formula (5), This indicates an encoder with dilated convolution. This indicates that the data for the new plain weave fabric production line was generated by the encoder. Feature map of the layer The number of layers in the encoder output feature map is indicated by the following formula (6): The logical output value of the prediction of the new plain weave fabric data is shown in the following formula (6): ……（6）， In the above formula (6), This indicates that the feature map has been decoded. The predicted logical value obtained afterwards This indicates that the data for the new plain weave fabric production line was generated by the encoder. Feature map of the layer This indicates the number of layers in the encoder's output feature map. After passing through the Softmax function, pixel-level predicted outputs for plain weave fabric samples are obtained. As shown in equation (7): ……（7）， In the above formula (7), This refers to the activation function commonly used in models to convert logistic values ​​into predictions. This indicates that the feature map has been decoded. The predicted logical value obtained afterwards This indicates an encoder with dilated convolution. As shown in equation (8): ……（8）， In the above formula (8), Indicates the total number of categories. Represents logical values The One element; Step 5.4, bias correction of network prediction, pixel-level prediction output of the original plain weave fabric sample. After learning the predicted residuals using a three-layer perceptron, pixel-level residual prediction outputs are obtained. As shown in equation (9): ……（9）， In the above formula (9), Let represent the residual learning network module of a three-layer perceptron. The residual learning module is regarded as multiple summation operations. The parameters of different layers can be optimized separately. For a three-layer perceptron network, the parameters of the first and third layers are optimized first, as expressed in the following formula (10): ……（10）， In the above formula (10), It indicates the first Layered network, It is the first Parameters of the layer network It is the first Input to the layer network.

3. According to the detection method of claim 1, step 6, the training and evaluation of the industrial fabric defect detection model, includes the following steps: Step 6.1: Randomly sample a new plain weave fabric production line with labels. Input is based on a hybrid prompt-based fine-tuning model for industrial fabric defect detection; sample Input prompts are obtained through the expert prompt module. , Then it is fed into an encoder-decoder structure with dilated convolution. Deep feature representation is then obtained. These deep features are input into the decoder In the process, the predicted logical value output of the sample is obtained. Then, the pixel-level prediction output of the sample is obtained through the Softmax function. Finally, a three-layer perceptron is used to obtain the pixel-level residual prediction output. ; Step 6.2, use pixel-level residual prediction output and the original pixel-level prediction output Calculate the defect detection loss function The following equations are used respectively: (11) and (12): ……（11）， ……（12）， In equations (11) and (12) above, Represents the cross-entropy loss function. Represents the relative entropy loss function. Images of plain weave fabric The original network pixel-level prediction output, Indicates the predicted output Pixel-level residual prediction output; Step 6.3, calculate the complete loss. As shown in equation (13): ……（13）， In the above formula (13), The weight of the loss function is 1.

0. The stochastic gradient descent algorithm is used to backpropagate the loss function to update the parameters of the three-layer perceptron and minimize the loss function, thereby improving the prediction performance of the new production line model. Step 6.4: Calculate the loss by repeatedly requesting the interface of the black-box model as a guide for the sampling algorithm, and update the parameters of multiple prompts to improve the generalization of the new production line model, as shown in equation (14) below. ……（14）， In the above formula (14), Represents the cross-entropy loss function. Indicates sample Given the unknown gradient of the black-box model, a gradient-free optimization method is adopted to solve the optimization problem, namely the covariance matrix adaptive evolution strategy. The covariance matrix adaptive evolution strategy is a parameterized search distribution model that uses a multivariate normal distribution. In each iteration, the covariance matrix adaptive evolution strategy generates a new group of query solutions by sampling from the multivariate normal distribution. Step 6.5: Repeat the above steps alternately to train the model until the maximum number of iterations is reached or the training converges. Step 6.6: Evaluate the model using the validation set. If the model performance requirements are not met, repeat steps 6.1 to 6.5 to gradually improve the model performance.

4. The use of the industrial fabric defect detection method in step 7 according to claim 1 includes: The newly collected plain weave fabric production line data is input into the trained industrial fabric defect detection model for defect detection. The prediction result with the highest classification confidence in the model output is selected as the final detection result.

Images