Fabric defect detection method based on cascade hybrid model
By using a cascaded hybrid model and an improved YOLOv10n network, the problems of low efficiency and insufficient accuracy in traditional fabric defect detection are solved, achieving efficient and accurate defect localization and attribute recognition, which is suitable for industrial inspection of various fabric textures.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-11
- Publication Date
- 2026-03-10
AI Technical Summary
Traditional fabric defect detection relies on manual visual inspection, which is easily affected by subjective factors. Furthermore, a single model is difficult to accurately identify the location and attributes of defects at high speeds and with complex textures, resulting in low detection efficiency and a high false detection rate.
A cascaded hybrid model is adopted to decompose the defect detection task into two stages: localization and attribute recognition. Features are extracted using convolutional neural networks, and local residual, global difference and text prompt features are fused. Adaptive fusion of multiple feature levels is achieved through an anomaly scoring network, and deformable convolution and SimAM attention mechanism are introduced into the improved YOLOv10n network.
It significantly improves the accuracy and robustness of fabric defect detection, reduces false detection and false negative rates, enhances computational efficiency, adapts to various fabric textures, and can be deployed in real time in industrial vision inspection equipment.
Smart Images
Figure CN121639664A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of defect detection technology, and in particular to a method for detecting fabric defects based on a cascaded hybrid model. Background Technology
[0002] In the textile production process, the detection of fabric defects is a key step in ensuring product quality, directly affecting the quality and market price of the final product.
[0003] Common structural and surface defects in textile production (such as missing warp and weft, oil stains, foreign fibers, etc.) are mainly detected by manual visual inspection. However, this method is easily affected by subjective factors, and long working hours can lead to human fatigue, affecting detection efficiency and failing to meet the needs of high-speed production. Existing defect detection methods typically use a single model to identify the location and attributes of fabric defects. However, in practical applications, wide fabric widths and high fabric speeds require the detection system to process large amounts of image data in real time, resulting in complex model feature extraction and low computational efficiency in existing methods. Furthermore, when faced with varied and complex fabric textures, a single model struggles to accurately distinguish the location and attributes of defects. Summary of the Invention
[0004] To address the shortcomings of existing methods, this invention decomposes the defect detection task into two stages: localization and attribute recognition. For each sub-task, feature extraction and algorithm design are optimized. The localization stage focuses on screening defect areas, while the attribute recognition stage analyzes the types and sizes of defects in depth, thereby improving the detection accuracy and robustness of the overall system.
[0005] The technical solution adopted in this invention is: a fabric defect detection method based on a cascaded hybrid model, comprising the following steps: Step 1: Acquire an image of the fabric to be inspected and at least one reference fabric image; In a preferred embodiment of the present invention, the defects in the fabric image include: thick warp, thick weft, seam ends, joints, holes, missing warp, missing weft, loose threads, dirt, and wrinkles.
[0006] Step 2: Extract normal feature maps and query feature sub-maps using the reference image and the sub-image to be detected; calculate the residual features of the normal feature maps and query feature sub-maps; fuse the residual features of the normal feature maps and query feature maps with the text prompt features, score the fused abnormal feature vectors, and use the scores to determine defects, thus obtaining the defect detection model. In a preferred embodiment of the present invention, the extraction of normal feature maps and the query feature sub-maps are performed using a convolutional neural network with several block modules.
[0007] In a preferred embodiment of the present invention, the Block module includes: inputting an initial feature map into a first LN layer, a multi-head attention layer and a first Dropout layer, and then adding the residuals of the input feature map to the initial feature map to obtain a first feature map; the first feature map is then passed through a second LN layer, a first linear layer, a GELU function, a second linear layer and a second Dropout layer in sequence, and then added to the residuals of the input feature map to obtain a second feature map.
[0008] In a preferred embodiment of the present invention, the residual features of the normal feature map and the query feature sub-map are calculated; the fusion of the residual features of the full map and the sub-map with the sample hint features includes: Using query feature subgraphs Compared with normal feature map Calculate residual characteristics , d Indicates the first n The number of channels in the output feature map of a layer block; Sub-image In the n The feature maps extracted by each block module are located at... x The i The characteristic values of each channel; This indicates the normal reference image set in Location, same floor The average eigenvector of each block; calculate mean ; Extracting global features and global reference features ,right , Pixel-wise difference operations are performed to obtain image-level global difference features. ; calculate and text feature vectors The dot product of these two elements yields the text discriminative feature vector. ; Calculate the abnormal feature vector of the sub-image ; Will Input an anomaly scoring network, output a score S ij .
[0009] In a preferred embodiment of the present invention, the anomaly scoring network includes: two stacked linear layers, ReLU and BN layers as inputs to the feature vector; and then a linear layer and a Sigmoid function as inputs.
[0010] In a preferred embodiment of the present invention, the sample prompt features employ a Transformer model text encoder.
[0011] As a preferred embodiment of the present invention, it further includes: annotating the image output by the defect detection model, training the improved YOLOv10n network using the annotated image, and outputting the defect type and location.
[0012] As a preferred embodiment of the present invention, the improved YOLOv10n network includes: improving the Backbone layer by replacing the Conv in the 2nd and 4th layers with Dconv.
[0013] As a preferred embodiment of the present invention, the improved YOLOv10n network further includes: an improved Neck layer, in which SimAM is inserted before the two Upsample layers.
[0014] The beneficial effects of this invention are: 1. This invention introduces a weighted Euclidean distance with the variance of normal sample features as the weight in defect localization and detection, which effectively weakens the interference of high variance feature dimensions, highlights the contribution of stable features in anomaly discrimination, and enables the model to maintain high robustness under complex working conditions such as illumination changes and texture periodic fluctuations, and significantly reduces the false detection rate and false negative rate. 2. This invention integrates the text features of "normal / abnormal" semantic prompts, guiding the model to establish a more semantically discriminative discrimination boundary in the feature space, significantly enhancing the ability to identify fabric morphological abnormalities and subtle texture defects; 3. This invention integrates three information sources: local residual features, global difference features, and text prompt features. Through an anomaly scoring network, it achieves adaptive fusion of multiple feature levels, effectively balancing the contribution ratio of local and global features under different fabric types, and ensuring that the output defect scores are more robust and consistent. 4. The network structure of this invention adopts a lightweight and modular design, with a small number of overall parameters and high computational efficiency, making it easy to deploy in real time in industrial vision inspection equipment; at the same time, each functional module can be flexibly replaced and optimized according to different fabric texture characteristics, and has good scalability and generalization ability. 5. This invention addresses the problem that small defects often exist in fabrics, and the small proportion of the target makes them difficult to detect. Furthermore, when the model is upsampling to restore spatial resolution, it introduces noise or blurry details. Before the two upsampling steps, the SimAM attention mechanism is introduced. By assigning attention weights on the feature map, information-rich regions are prioritized for enhancement, enabling the upsampling operation to be performed based on higher-quality features, reducing information loss and preserving the key features of small targets. 6. In view of the diverse forms of fabric defects, this invention introduces deformable convolution, which adaptively adjusts the sampling parameters according to the shape and size of different defect targets, thereby enhancing the backbone network's ability to extract multi-scale fabric defect features. Attached Figure Description
[0015] Figure 1 This is a flowchart of the fabric defect detection method based on a cascaded hybrid model of the present invention; Figure 2 This is a schematic diagram of the defect detection model of the present invention; Figure 3 This is a schematic diagram of the Block module structure of the present invention; Figure 4 This is a schematic diagram of the anomaly scoring network of the present invention; Figure 5 This is a defect location effect diagram of the present invention; Figure 6 This is a diagram of the improved YOLOv10n network structure of the present invention; Figure 7 This is a schematic diagram of DConv; Figure 8 This is a schematic diagram of SimAM; Figure 9 This is a comparison of the effects of the present invention and existing models. Figure 10 This is the second comparison of the effects of the present invention and existing models. Detailed Implementation
[0016] The present invention will be further described below with reference to the accompanying drawings and embodiments. The drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.
[0017] like Figure 1 As shown, a fabric defect detection method based on a cascaded hybrid model includes the following steps: Step 1: Acquire an image of the fabric to be inspected and at least one reference fabric image; The raw images of the fabric are acquired using industrial cameras; these raw images are unprocessed, full-width images of the fabric surface. Fabric defects include structural and surface defects, such as thick warp, thick weft, seam ends, joints, holes, missing warp, missing weft, loose threads, dirt, and wrinkles.
[0018] The original image is divided into regions using a sliding window cropping method. The image is divided into several grid sub-images according to a preset step size and window size, resulting in a set of sub-images. ; i , j It uses row and column indexes and synchronously records the coordinates of the top-left corner of each sub-image;A Global image; original image A Alternatively, it can be a subplot that directly matches the size, eliminating the need for window partitioning; thus forming a coordinate set. ; for The coordinates of the top left corner are used for subsequent defect location and area backtracking.
[0019] Will A Each of them The images are uniformly converted to a three-channel image format and scaled to a preset fixed resolution according to the input requirements of subsequent models; and each... Normalization and standardization are performed to obtain a preprocessed image set. A 1 , as input to the subsequent model.
[0020] Step 2: Construct a defect detection model; like Figure 2 sub-images or Perform anomaly area identification to determine if defects exist; by For example, a pre-collected image of a flawless fabric is used as a normal reference image, and the image to be queried is... The images are input into a convolutional neural network composed of several block modules to extract differences between image patches at different levels; a normal reference image and the sub-image to be queried are also input. Different levels of patch features were extracted by the Block modules at each layer, and are denoted as follows: (Query feature subgraph) and (Normal feature diagram); like Figure 3 The Block module includes: inputting the initial feature map into the first LN layer, the multi-head attention layer and the first Dropout layer, and then adding the residuals of the input to the initial feature map to obtain the first feature map; the first feature map is then passed through the second LN layer, the first linear layer, the GELU function, the second linear layer and the second Dropout layer in sequence, and then added to the residuals of the input to the first feature map to obtain the second feature map; The output second feature map size is (B, L, C), where B refers to the batch size, L refers to the image sequence length, and C refers to the feature dimension. based on and Calculate the total patch residual features To measure the degree of deviation of the query sub-image at the local feature level, this invention designs a residual formula based on multi-level features and weighted Euclidean distance as follows:
[0021] in, It is the variance of the reference image in the feature dimension; It is a small constant to prevent division by zero; d Indicates the first n The number of channels in the output feature map of a layer block; Represents the sub-image to be detected In the n The feature maps extracted by each block module are located at... x The i The characteristic values of each channel; This indicates that the normal reference image set is in the same position. The same floor The average feature vector of each block, i.e., the statistical mean of the reference features; where,
[0022] in, N This indicates the number of normal samples. Represented as the first k Zhang Zhengchang sample in the first n The feature vector at the corresponding position of the layer block.
[0023] Patch residual features for each layer of Block modules Calculate the mean and output the multi-level patch residual features. The formula is:
[0024] in, N Indicates the number of reference images. n Indicates the first n Each level.
[0025] global image A Global features are extracted from a convolutional neural network consisting of several block modules with the same structure as the global reference image input. , ;right , Perform pixel-by-pixel difference operations to obtain image-level global difference features. , used to characterize the degree of difference at the global level; A text encoder is used to encode normal and abnormal sample prompts, generating text feature vectors with unified semantics. The text encoder is based on the Transformer model; Normal samples correspond to normal images, while abnormal samples correspond to defective images. Calculate image-level global features and text feature vectors The dot product of these two elements yields the text-guided text discriminative feature vector. This is used to enhance semantic-assisted discrimination capabilities; Calculate the abnormal feature vector of the sub-image The formula is:
[0026] in, This indicates that each pixel is added together.
[0027] Will Input an anomaly scoring network, output a score S ij ; like Figure 4 The anomaly scoring network consists of: two stacked linear layers, ReLU and BN layers as input to the feature vectors; and then a linear layer and a Sigmoid function as input. Set anomaly scoring threshold ,Will and Compare and filter out those greater than sub-images Output the corresponding region coordinates. Complete the location of the defective area.
[0028] This invention introduces a weighted Euclidean distance with the variance of normal sample features as the weight in defect localization and detection, which effectively weakens the interference of high variance feature dimensions, highlights the contribution of stable features in anomaly discrimination, and enables the model to maintain high robustness under complex working conditions such as illumination changes and texture periodic fluctuations, significantly reducing the false detection rate and false negative rate. This invention integrates the text features of "normal / abnormal" semantic prompts, guiding the model to establish a more semantically discriminative discrimination boundary in the feature space, significantly enhancing the ability to identify fabric morphological abnormalities and subtle texture defects. This invention integrates three information sources: local residual features, global difference features, and text prompt features. Through an anomaly scoring network, it achieves adaptive fusion of multiple feature levels, effectively balancing the contribution ratio of local and global features under different fabric types, and ensuring that the output defect scores are more robust and consistent. The network structure of this invention adopts a lightweight and modular design, with a small number of overall parameters and high computational efficiency, making it easy to deploy in real time in industrial vision inspection equipment. At the same time, each functional module can be flexibly replaced and optimized according to different fabric texture characteristics, and has good scalability and generalization ability.
[0029] like Figure 5 The a and ai are reference images for different types of fabrics. Figure 5a, d, and g represent normal fabric images without defects, while the two columns on the right represent fabric images with defects detected based on the reference image. This shows that the model can accurately detect defect images and has good detection performance even against complex texture backgrounds.
[0030] Step 3: Construct a defect attribute detection model; To identify the types and sizes of defects, a fabric defect target detection model based on an improved YOLOv10n network was constructed to extract and identify the attribute information of target defects.
[0031] The model is divided into two stages: training and inference. The images of the defects are labeled to form a defect image dataset, which is then divided into a training set and a validation set in an 8:2 ratio. The pre-trained weights with the best performance are selected as the initial model parameters based on the detection results of the validation set. like Figure 6 As shown, the Backbone layer is improved by replacing Conv in layers 2 and 4 with Dconv; In convolutional neural networks, shallow layers are usually responsible for extracting low-level features, such as edges and textures. Considering the diverse forms of fabric defects, deformable convolution (DConv) is introduced into the shallow Conv layers (layers 2 and 4) of the original model. This convolutional structure can adaptively adjust the sampling parameters according to the shape and size of different defect targets, thereby enhancing the backbone network's ability to extract multi-scale fabric defect features.
[0032] Deformable convolutional layers such as Figure 7 As shown, the green branch represents the offset acquisition of deformable convolution. The feature map with N channels is passed through the convolutional layer conv to obtain a feature map with 2N channels, where 2N represents the offset in the x and y directions of each channel.
[0033] Improvements to the Neck layer include: inserting SimAM between layer 11 PSA and layer 13 Upsample, and inserting SimAM between layer 15 C2f and layer 17 Upsample; This invention addresses the problem of small defects in fabrics, which are difficult to detect due to their small size. Since the Neck layer selects feature maps from layers 5, 7, and 11 of the Backbone network for feature fusion, and the model introduces noise or blurred details when upsampling to restore spatial resolution, the SimAM attention mechanism is introduced before the two upsampling operations. By assigning attention weights to the feature maps, information-rich regions are prioritized for enhancement, enabling the upsampling operation to be based on higher-quality features, reducing information loss and preserving the key features of small targets.
[0034] SimAM attention mechanism, such as Figure 8As shown, without changing the size of the input feature map, attention weights are added to each point in the channel and spatial dimensions to achieve adaptive adjustment of the features.
[0035] The Head component adopts a dual-head structure. During the training phase, a one-to-many head is used to generate multiple prediction boxes for each defect region to enhance the supervision signal and improve feature robustness. During the inference phase, a one-to-one prediction head is used to generate a unique prediction result for each defect and avoid redundant output.
[0036] The prediction bounding box output by the prediction head includes: x-axis X y-axis Y ,width W ,high H and defect category prediction labels L Generate a unique prediction result for each defect. This serves as the attribute identification output for the defect.
[0037] Validity verification: The experimental environment included a CPU of Intel(R) Core(TM) i9-14900K 3.20 GHz and a GPU of NVIDIA GeForce RTX 4090. like Figures 9-10 The detection performance of the proposed model and the YOLOv10n model were tested under the same experimental environment. The evaluation metrics included Precision, Recal1 (recall), and mAP (Mean Average Precision). The experimental results are shown in the table below: Table 1. Comparison of detection results between the model of this invention and the YOLOv10n model.
[0038] The ablation experiments of the modules were conducted under the same experimental conditions, and the test results are shown in Table 2: Table 2 Ablation Experiment
[0039] Defect Result Integration and Output: Based on the Obtained Region Coordinates and defect attribute prediction results O This process determines the absolute location of each defect in the original image; the final defect detection results are output, including the defect category. L Defect size ( W , H and the location of the defects This enables the location and attribute identification of fabric defects.
[0040] Based on the above-described preferred embodiments of the present invention, and through the foregoing description, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.
Claims
1. A fabric defect detection method based on a cascaded mixture model, characterized in that, The method comprises the following steps: Step one, collecting the image of the fabric to be detected and at least one reference fabric image; Step two, extracting normal feature maps and query feature sub-maps using the reference image and the sub-image to be detected; calculating the residual features of the normal feature maps and the query feature sub-maps; Fusing the residual features of the normal feature maps and the query feature maps with the text prompt features, scoring the fused abnormal feature vectors, using the scores for defect judgment, and obtaining a defect detection model.
2. The fabric defect detection method based on a cascaded hybrid model according to claim 1, wherein, Calculating the residual features of the normal feature maps and the query feature sub-maps; Fusing the residual features of the normal feature maps and the query feature maps with the text prompt features includes: Utilizing query feature subgraphs with normal feature maps Computing residual features , d representing the n channel number of the Block output feature map of the representing the feature value of the n feature map extracted by the x th Block module at position i th channel of the representing the average feature vector of the same layer th Block at computing mean ; extracting global features and global reference features , to , pixel-wise difference operation to obtain image-level global difference features ; Computing the dot product of the text feature vector and the text discriminative feature vector , obtaining a text discriminative feature vector Computing an anomaly feature vector for a sub-image ; will be described below. inputting the abnormality score network, outputting the score S ij .
3. The fabric defect detection method based on a cascaded mixture model according to claim 2, wherein, The convolutional neural network of the Block module is used to extract the normal feature maps and the query feature sub-maps.
4. The fabric defect detection method based on a cascaded mixture model according to claim 3, wherein, The Block module comprises: inputting the initial feature map into the first LN layer, the multi-head attention layer and the first Dropout layer, and then performing residual addition with the initial feature map to obtain the first feature map; the first feature map is sequentially subjected to the second LN layer, the first linear layer, the GELU function, the second linear layer and the second Dropout layer, and then performs residual addition with the first feature map to obtain the second feature map.
5. The fabric defect detection method based on a cascaded mixture model according to claim 2, wherein, The abnormal score network comprises: inputting the feature vector into two stacked linear layers, Relu and BN layers; and then inputting into a linear layer and a Sigmoid function.
6. The fabric defect detection method based on a cascaded mixture model according to claim 1, wherein, It also includes: Labeling the image output by the defect detection model, training the improved YOLOv10n network using the labeled image, and outputting the defect type and position.
7. The fabric defect detection method based on a cascaded mixture model according to claim 6, wherein, The improved YOLOv10n network comprises: improving the Backbone layer, replacing the Conv of the 2nd layer and the 4th layer with Dconv.
8. The fabric defect detection method based on a cascaded mixture model according to claim 6, wherein, The improved YOLOv10n network further comprises: improving the Neck layer, inserting SimAM before two Upsample.
9. The fabric defect detection method based on a cascaded mixture model according to claim 1, wherein, The defects of the fabric image include: coarse warp, coarse weft, seam, joint, hole, missing warp, missing weft, thread end, dirt and wrinkle.
10. The fabric defect detection method based on a cascaded mixture model according to claim 1, wherein, The sample prompt feature uses the text encoder of the Transformer model.