A fabric defect detection method based on an improved YOLOv7 model

Abstract

The present application relates to a fabric defect detection method based on an improved YOLOv7 model, which collects fabric defect pictures and pre-processes the original image dataset, converting from the RGB color space to the YCrCb color space; combines the Bottleneck Transformer, Efficient Channel Attention and YOLOv7 model to improve the YOLOv7 model and build an ECTS-YOLOv7 network model; determines the model loss function and performance evaluation index according to GIoU; obtains the trained ECTS-YOLOv7 network model; inputs the dataset for processing to obtain the final prediction frame diagram. Using this method can effectively solve the influence of noise and small defects in the fabric defect image, improve the detection accuracy while ensuring the processing speed, and the model deployment difficulty is small, which has strong industrial production practicability.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and more specifically to a method for detecting fabric defects based on an improved YOLOv7 model. Background Technology

[0002] The textile industry has always held an important position in my country's national economy. With rapid economic development and improved living standards, the consumer market has placed new quality demands on textile products. Common methods for detecting textile defects mainly fall into two categories: the first is purely manual inspection, where inspectors stand in front of the inspection equipment and visually inspect and mark or record defects. Besides high employee training costs, this method suffers from high false negative rates, slow speed, insufficient database, and worker fatigue under continuous work. The second method is a stand-alone quality inspection mode, which primarily uses machine vision and computer program analysis to detect and classify textile defects. This method typically embeds the program into the terminal equipment, which is problematic due to the limited variety of textile types, insufficient self-learning capabilities, and the need for separate maintenance of each terminal device, leading to complex system integration and high operating costs. Therefore, designing an efficient method for detecting textile image defects is essential.

[0003] Deep learning neural networks offer advantages such as high accuracy and wide recognition capabilities. In the same recognition scenario, using deep learning neural networks for target recognition can effectively resist interference from color, texture, and lighting conditions. However, their computational demands are extremely high, requiring sophisticated hardware and limiting their deployment on hardware with limited performance. Meanwhile, existing fabric defect detection models suffer from noise interference and slow detection speed, while also needing improvement in detection accuracy and efficiency. Fabric defect detection models tend to focus on global information in the dataset, while fabric defects are often minute. Overemphasizing global information can easily overlook crucial and important local information about fabric defects, leading to inaccurate predictions. Furthermore, the input typically consists of RGB color images acquired from industrial cameras and converted to grayscale for detection. However, each RGB channel contains brightness, saturation, and hue information, making the image pixel values ​​highly susceptible to influence from the acquisition environment. To obtain more realistic and stable fabric grayscale images, a color space less sensitive to natural factors needs to be selected. Based on this, the present invention designs a method for generating grayscale images after color space conversion, and further optimizes the existing network model to reduce the number of parameters while ensuring accuracy, thereby realizing the detection and location of fabric defects. Summary of the Invention

[0004] This invention discloses a fabric defect detection method based on an improved YOLOv7 model. The method preprocesses the dataset, converting the RGB color images acquired by the industrial camera to the YCrCb color space before converting them to grayscale for detection. This effectively reduces interference from the external environment of the camera-captured images. In the model, by adding an attention mechanism ECANet module at specific locations in the backbone network and introducing a BoTNet module in the head network, important local information about fabric defects is preserved, further improving accuracy while reducing the number of parameters.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A fabric defect detection method based on an improved YOLOv7 model includes the following:

[0007] Step 1: Establish a fabric defect image dataset. After converting the image dataset from the RGB color space to the YCrCb color space, divide it into training set, validation set, and test set.

[0008] Step 2: Construct the ECTS-YOLOv7 network model;

[0009] The ECTS-YOLOv7 network model is obtained by replacing the last layer Bconv of each E-ELAN layer in the YOLOv7 backbone network with the ECANet attention mechanism module, and replacing the original ELAN-W module in the YOLOv7 head network with the BoTNet module.

[0010] Step 3: Select images from the training set and input them into the ECTS-YOLOv7 network model for training. The ECANet module obtains the vectors corresponding to the channels as evaluation scores, evaluates the importance of each feature map, and assigns weight values ​​to each feature channel according to the importance.

[0011] Step 4: Select images from the training set and input them into the ECTS-YOLOv7 network model for processing. Each image generates a specific layer of effective feature maps in the backbone network. A band matrix W is used in the ECANet attention module. k To learn channel attention, after using non-dimensionality-reduced GAP aggregation convolutional features, the kernel size k is first adaptively determined, then one-dimensional convolution is performed, and then the Sigmoid function is used to learn channel attention. Finally, the weights are applied to the original channel features through the path aggregation network to obtain the training prediction feature map of the corresponding layer.

[0012] Step 5: Use the Kmeans++ clustering algorithm to cluster all target GT boxes on the extended fabric image in the image dataset to obtain K prior boxes. Distribute the K prior boxes evenly and adjust them according to the training prediction feature map. Each prior box is adjusted to obtain a training prediction box. Select several training prediction boxes as training candidate boxes based on the target GT boxes.

[0013] Step 6: Calculate the overall classification and regression loss of the ECTS-YOLOv7 network model based on the difference between the trained predicted feature map, the adjusted candidate box and the target GT box, and backpropagate the overall loss value to the ECTS-YOLOv7 network model. Then, use gradient descent to update the parameters of the ECTS-YOLOv7 network model to obtain the parameter-updated ECTS-YOLOv7 network model.

[0014] Step 7: Repeat steps 3 to 6 to process each image in the training set. After the selected image is output in step 4, repeat steps 5 and 6 to update the parameters of the ECTS-YOLOv7 network model until all images in the training set have been trained. The ECTS-YOLOv7 network model with updated parameters can be used as a pre-trained model.

[0015] Step 8: Input the extended fabric images in the validation set into the pre-trained model for processing. After performing the same operation as the training set in Step 4, the validation prediction feature maps of the corresponding levels are obtained. Perform the same operation as the training prediction feature maps in Step 5 on each output validation prediction feature map to output the validation prediction box. Calculate the validation set loss based on the validation prediction box and the target GT box, and calculate the average precision AP of a single category and the average precision mAP of all categories of defects in the fabric defect image dataset after processing by the pre-trained model.

[0016] Step 9: Repeat steps 7 and 8 until the average precision (AP) and average precision (mAP) of each category of defects in the fabric defect image dataset tend to a stable value. At this point, the total loss curve of the network converges, and the trained ECTS-YOLOv7 network model is obtained.

[0017] Step 10: Input the extended fabric images from the test set into the trained ECTS-YOLOv7 network model for processing. Perform the same operation as the training prediction feature map in Step 5 on each output test prediction feature map to output test prediction boxes. Filter according to the NMS (Non-Maximum Suppression) principle to remove blank boxes in the test prediction boxes. Filter the remaining test prediction boxes according to confidence to obtain the final test prediction boxes, thus realizing the detection and localization of fabric defects.

[0018] Furthermore, the method for constructing the fabric defect image dataset in step 1 is as follows:

[0019] Several images of fabrics with defects were acquired using an industrial camera. Each defect in each image was labeled with its category and location using the minimum bounding rectangle in LabelImg software. The target ground truth (GT) box must completely enclose each defect. The target GT box was labeled as (class, xmin, ymin, xmax, ymax), where class represents the category of defect, xmin and ymin represent the x and y coordinates of the top left corner of the target GT box, respectively, and xmax and ymax represent the x and y coordinates of the bottom right corner of the target GT box.

[0020] Data augmentation was performed on the data to obtain labeled and enhanced fabric images. All collected images and enhanced fabric images were then expanded into a fabric defect image dataset with no fewer than 5,000 images. At the same time, the category label XML files were converted into YOLO label TXT format.

[0021] Furthermore, the method for obtaining K prior boxes in step 5 is as follows:

[0022] The Kmeans++ clustering algorithm is used to cluster all target ground truth boxes on each extended fabric image in the image dataset. A sample point is randomly selected from the target ground truth boxes X in the image dataset as the first initial cluster center. The shortest distance D(x) between each sample and the current cluster center is calculated, and the probability P(x) of each sample point being selected as the next cluster center is calculated.

[0023]

[0024] Finally, the sample point corresponding to the highest probability value is selected as the next cluster center. This selection is repeated until K cluster centers are obtained. The prior boxes are composed of the x-coordinate and y-coordinate of each cluster center as the width and height, and finally K prior boxes are obtained.

[0025] Furthermore, in step 1, the method for converting the RGB color space to the YCrCb color space is as follows:

[0026] Y'=0.257*R'+0.504*G'+0.098*B'+16

[0027] Cb'=-0.148*R'-0.291*G'+0.439*B'+128

[0028] Cr'=0.439*R'-0.368*G'-0.071*B'+128

[0029] R'=1.164*(Y'-16)+1.596*(Cr'-128)

[0030] G'=1.164*(Y'-16)-0.813*(Cr'-128)-0.392*(Cb'-128)

[0031] B'=1.164*(Y'-16)+2.017*(Cb'-128).

[0032] Furthermore, in step 4, the ECA attention mechanism first calculates the average value of each channel through a global average pooling operation, then performs a one-dimensional convolution on the average value of each channel to calculate the attention coefficient of that channel; finally, the attention coefficient is multiplied by the original input data, and the result is reassembled into the same shape as the original input to obtain weighted input data.

[0033] The attention weight formula is as follows:

[0034]

[0035] In the above formula, the parameters have the following meanings:

[0036] X i,j,c This represents the pixel value of the i-th row, j-th column, and c-th channel of the input feature map X;

[0037] σ is the sigmoid activation function; W C H represents the attention weight; H represents the height of the feature map; W represents the width of the feature map.

[0038] This represents the feature map after global pooling, where c represents the number of classes. j This represents the probability that the j-th channel belongs to the c-th category;

[0039] The channel attention weighting formula is as follows:

[0040] Y C =W C X C

[0041] In the above formula, Y C X represents the weighted channel feature map. C This represents the c-th channel of the input feature map X;

[0042] The formula for deep attention weights is as follows:

[0043]

[0044] In the above formula, the parameters have the following meanings:

[0045] Y*,*,c′ This represents the value of the C′-th channel of the weighted channel feature map Y at all pixel locations;

[0046] W 2,c′ It is a 1×1 convolution kernel;

[0047] W d For deep attention weights;

[0048] The weighted formula for deep attention is as follows:

[0049] Z = W d Y

[0050] Where Z represents the weighted output feature map and Y represents the weighted channel feature map;

[0051] The formula for calculating the output feature map is as follows:

[0052]

[0053] Where F(X) represents the feature extractor.

[0054] Further, in step 7, the training prediction feature map obtained from each training group is divided into H×W grid cells, where H and W represent the height and width of the training prediction feature map, respectively, and the center of the grid cell is called the anchor point; the K prior boxes from step 5 are sorted by scale and evenly divided into N groups of prior boxes; similarly, the training prediction feature map is sorted and divided by scale, and each group of prior boxes is assigned to a training prediction feature map. Then, prior boxes are superimposed on the anchor points, and the position and scale of each prior box are adjusted according to the position and scale information of each one-dimensional adjustment vector, thereby obtaining a training prediction box; for each target GT box, the intersection-union ratio (IoU) loss between the target GT box and each training prediction box is calculated, and the training prediction box with the largest IoU loss with the target GT box is selected as the training candidate box;

[0055] For any two validation boxes A and B, first find the smallest box C that can enclose them, then calculate the ratio of the area of ​​C\(A∪B) to the area of ​​C, and then subtract this ratio from the IoU values ​​of A and B to get the GIoU.

[0056] The formula for calculating IoU is as follows:

[0057]

[0058] The formula for calculating GIoU is as follows:

[0059]

[0060] When IoU = 0, the formula for calculating GIoU is as follows:

[0061]

[0062] Furthermore, K in the K prior boxes is set to 9, and N in the N prior boxes is set to 3, ensuring that K / N is an integer.

[0063] Furthermore, after performing step 10, the detection results in step 10 are analyzed using the following object detection metrics to determine the detection performance of the ECTS-YOLOv7 network model after training; the object detection metrics include: average precision (AP) for a single class, average precision (mAP) for all classes, precision, recall, and speed (FPS).

[0064] The fabric defect detection method disclosed in this invention has the following advantages:

[0065] (1) This invention utilizes the unique brightness and chromaticity advantages of the YCrCb color space to obtain grayscale images with different color space characteristics, effectively reducing external environmental interference.

[0066] (2) This invention improves the backbone network and adds the attention mechanism module ECANet to reweight the convolutional feature channels, enhance the interdependence between important features, and assign weights according to the importance of different channel features, thereby improving the recognition performance of the model while ensuring speed.

[0067] (3) This invention utilizes the BoTNet module, which effectively reduces the number of parameters and is more suitable for industrial production applications. At the same time, because it introduces two-dimensional position coding, it is more suitable for small target detection.

[0068] In summary, the detection method of this invention can effectively solve the problems caused by noise and small defects in fabric defect images, improve detection accuracy while ensuring processing speed, and has low model deployment difficulty, making it highly practical for industrial production. Attached Figure Description

[0069] Figure 1 This is a flowchart of the fabric defect detection method of the present invention;

[0070] Figure 2 This is a schematic diagram of the ECTS-YOLOv7 network model of the present invention;

[0071] Figure 3 This is a schematic diagram of the ECANet structure of the present invention;

[0072] Figure 4 This is a schematic diagram of the BoTNet structure of the present invention;

[0073] Figure 5The precision-recall curves for detecting ten types of defects using the detection method of this invention are shown.

[0074] Figure 6 This is a comparison chart of the predicted and actual results of the detection method of the present invention for detecting ten types of defects;

[0075] Figure 7 The images show the results of using the detection method of this invention to detect defects in two types of fabrics. Detailed Implementation

[0076] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0077] This embodiment discloses a fabric defect detection method based on an improved YOLOv7 model, such as... Figure 1 As shown, it includes the following:

[0078] S1) Use a high-precision industrial camera to collect several images of fabrics with defects, manually label the defect categories, and expand all the collected images and enhanced images into a fabric defect image dataset. The expanded dataset must contain a certain proportion of expanded fabric images, and the dataset must meet the sample quantity requirements of the network model.

[0079] Specifically, each acquired fabric image is sequentially labeled with defects, and data augmentation processing is performed simultaneously. After processing, the images are collectively compiled into a fabric defect image dataset. The specific steps are as follows: First, the minimum bounding rectangle (MOU) in LabelImg software is used to label the category and location of each defect in each image. The target ground truth (GT) bounding box must completely enclose each defect. The target GT bounding box is labeled as (class, xmin, ymin, xmax, ymax), where class represents the defect category, xmin and ymin represent the x and y coordinates of the top-left vertex of the target GT bounding box, and xmax and ymax represent the x and y coordinates of the bottom-right vertex. After data augmentation processing (such as image flipping, rotation, adding noise, changing contrast, changing brightness, etc.), labeled and augmented fabric images are obtained, and the label XML file containing the saved categories is converted to YOLO label TXT format.

[0080] S2) Use the Kmeans++ clustering algorithm to cluster all target ground truth boxes on the extended fabric images in the dataset to obtain K prior boxes.

[0081] Specifically, the Kmeans++ clustering algorithm is used to cluster all target ground truth (GT) boxes on each extended fabric image in the dataset. A sample point is randomly selected from the target GT boxes X in the dataset as the first initial cluster center. The shortest distance D(x) between each sample and the current cluster center is calculated, and the probability P(x) of each sample point being selected as the next cluster center is calculated.

[0082] The probability P(x) is shown in formula (1):

[0083]

[0084] Finally, the sample point corresponding to the highest probability value is selected as the next cluster center. This selection is repeated until K cluster centers are obtained. The prior boxes are composed of the x-coordinate and y-coordinate of each cluster center as the width and height, and finally K prior boxes are obtained.

[0085] S3) After converting the dataset from the RGB color space to the YCrCb color space, the processed dataset is divided into training set, validation set and test set in an 8:1:1 ratio.

[0086] Compared to the RGB color space, the YCrCb color space has the following advantages: In the YCrCb color space, the luminance component Y and the chrominance components Cb and Cr are represented separately, while in the RGB color space, the luminance components R, G, and B are represented separately. This separation method can better preserve the detail information in the image. The chrominance components Cb and Cr in the YCrCb color space are obtained by subtracting the luminance component Y, and no luminance value information is introduced during the calculation process, effectively resisting the influence of external ambient light sources.

[0087] To convert from the RGB color space to the YCrCb color space, follow these steps:

[0088] Y'=0.257*R'+0.504*G'+0.098*B'+16

[0089] Cb'=-0.148*R'-0.291*G'+0.439*B'+128

[0090] Cr'=0.439*R'-0.368*G'-0.071*B'+128

[0091] R'=1.164*(Y'-16)+1.596*(Cr'-128)

[0092] G'=1.164*(Y'-16)-0.813*(Cr'-128)-0.392*(Cb'-128)

[0093] B'=1.164*(Y'-16)+2.017*(Cb'-128).

[0094] After color space conversion, the processed dataset is divided into training, validation, and test sets in an 8:1:1 ratio. The processed label files are placed in the `train`, `test`, and `val` folders under the `labels` folder, and the processed fabric defect images are placed in the `train`, `test`, and `val` folders under the `images` folder, ensuring a one-to-one correspondence. A YAML file is also created to save the paths to the training, validation, and test sets, as well as the types of fabric defect labels.

[0095] S4) as Figures 2 to 4 As shown, an ECTS-YOLOv7 network model was constructed. The last Bconv layer of each E-ELAN layer in the backbone network of the YOLOv7 network structure was replaced with an ECANet attention mechanism module. By assigning weights to the extracted features, more attention was paid to the local information of fabric defects. The original ELAN-W module in the YOLOv7 head network was replaced with a BoTNet module to reduce the number of parameters and facilitate industrial applications.

[0096] The original YOLOv7 model tends to focus on global information in the dataset, while fabric defects are often minor. Overemphasizing global information can easily overlook some key and important local information about fabric defects, leading to inaccurate predictions. ECANet is added, updating channel weights during training through backpropagation, and then using gradient descent or other optimization algorithms to further update the channel weights. This increases the model's resistance to background interference, allowing it to distinguish the degree of attention given to useful and useless features. ECANet implements channel attention weighting in two steps: 1. Extracting channel features; 2. Calculating channel weights. Local cross-channel interactions are achieved through fast one-dimensional convolutions of size k. To avoid manually adjusting k through cross-validation, an adaptive method is developed to determine k, where the coverage of cross-channel interactions (i.e., kernel size k) is proportional to the channel dimension.

[0097] Employing bottleneck blocks significantly reduces computation and improves network efficiency. With greater network depth, it reduces overfitting and enhances generalization ability. It enables better learning of input data feature representations, improving model accuracy. In small object detection tasks, objects are relatively small with detailed features; BoTNet's nonlinear transformations and cross-layer connections help extract these features and effectively reduce information loss during network propagation. BoTNet's multi-head attention mechanism helps the model better focus on local and global features, enhancing the network's ability to represent small objects. This method significantly improves baseline performance in instance segmentation and object detection while reducing parameters, thus minimizing latency. The MHSA Block is a core innovation of the BoTNet module. Compared to the traditional MHSA in Transformers, the BoTNet module uses three nonlinear activations, normalized using BatchNorm consistent with CNNs, and introduces two-dimensional positional encoding, improving its ability to detect small objects.

[0098] The overall workflow of the ECTS-YOLOv7 network model is as follows: First, the input image is preprocessed and aligned to a size of 640x640x3. The aligned image is then input into the Backbone network. The output of the Backbone network is then processed by the ECANet attention module using a band matrix Wk to learn channel attention. After using non-dimensionality-reduction GAP aggregation of convolutional features, the kernel size k is adaptively determined, followed by one-dimensional convolution. Then, the weights are normalized by Sigmoid and applied to the original channel features through a path aggregation network. This outputs three layers of feature maps of different sizes, which are finally input into the final Head network to predict three types of image detection tasks (classification, foreground / background classification, and bounding boxes), and output the final result.

[0099] The ECANet attention mechanism adaptively learns the importance of each channel and uses it to calculate the weights for each channel. In this process, the ECANet attention mechanism first calculates the average value of each channel through a global average pooling operation to obtain the importance distribution of that channel across the entire dataset. Next, by applying a one-dimensional convolution operation to the average value of each channel, the attention coefficient for that channel can be calculated. Finally, the attention coefficients are multiplied by the original input data, and the results are reconstructed into the same shape as the original input to obtain the weighted input data.

[0100] The attention weight formula is as follows:

[0101]

[0102] In the above formula, the parameters have the following meanings:

[0103] X i,j,c This represents the pixel value of the i-th row, j-th column, and c-th channel of the input feature map X;

[0104] σ is the sigmoid activation function; W C H represents the attention weight; H represents the height of the feature map; W represents the width of the feature map.

[0105] This represents the feature map after global pooling, where c represents the number of classes. j This represents the probability that the j-th channel belongs to the c-th category;

[0106] The above formula linearly transforms the average value of each channel through a 1×1 convolutional layer to obtain the weight W of each channel. C Then, the sigmoid activation function is used to limit the weights to between 0 and 1, which is used to weight the original feature map in the channel dimension.

[0107] The channel attention weighting formula is as follows: In the ECA attention mechanism module, the channel attention weighting formula is used to extract the channel attention features of the input feature map.

[0108] Y C =W C X C

[0109] In the above formula, Y C X represents the weighted channel feature map. C This represents the c-th channel of the input feature map X;

[0110] The deep attention weight formula is as follows: This formula calculates the average value of the channel attention weights across the entire input feature map, used to measure the importance of the channel attention mechanism for different channels. In the ECA attention mechanism module, the role of the deep attention mechanism is to improve the expressive power of the channel attention mechanism, allowing the attention weights of different channels to be adjusted according to the importance of the entire input feature map.

[0111]

[0112] In the above formula, the parameters have the following meanings:

[0113] Y *,*,c′ This represents the value of the C′-th channel of the weighted channel feature map Y at all pixel locations;

[0114] W 2,c′ It is a 1×1 convolution kernel;

[0115] W dFor deep attention weights;

[0116] The above formula linearly transforms the average value at each depth through a 1×1 convolutional layer to obtain the weight W at each depth. d Then, the σ activation function is used to restrict the weights to between 0 and 1, which is used to weight the weighted channel feature maps in the depth dimension.

[0117] The deep attention weighting formula is as follows: This formula means that the channel attention features and deep attention features of the input feature map are weighted to obtain the final weighted feature map.

[0118] Z = W d Y

[0119] Where Z represents the weighted output feature map and Y represents the weighted channel feature map.

[0120] The final output feature map is calculated using the following formula:

[0121]

[0122] Here, F(X) represents the feature extractor, which extracts features from the input feature map X through operations such as convolution to obtain the output feature map Y. σ is the sigmoid function, used to restrict the channel attention weights to the range [0,1]. The final output feature map Y is the feature extractor's output multiplied by the channel attention weights.

[0123] The ECANet attention module performs average pooling on the three output feature maps, followed by two fully connected layers to obtain the correlation between channels, reducing parameters and computational cost. Finally, after sigmoid normalization, the weights are applied to the original channel features through a path aggregation network. Using the ECANet module reweights the convolutional feature channels, enhancing the interdependence between important features and learning the importance of features in different channels, thus producing better results and improving recognition performance. In some cases, the global features of fabric defects are not significantly different; for example, the distinction between sundries and oil stains is not obvious, and some felters and broken ends are extremely similar. The ECANet attention mechanism can enhance important features of fabric defects and weaken unimportant features, making the extracted defect features more representative and preserving important local information. For example, sundries and oil stains share similar global features such as color, size, and degree of contamination, but differ in surface texture and shape. Before ECANet operation, their similar feature information makes them difficult to distinguish. After ECANet operation, less important information is weakened, while important features such as surface texture and shape are highlighted, facilitating accurate identification of sundries and oil stains. The BoTNet module uses three non-linear activations and employs Batch Norm normalization consistent with CNNs. It also introduces two-dimensional positional encoding, effectively reducing the number of parameters and minimizing latency while improving the accuracy of small target detection. This is beneficial for practical application and deployment in production. The multi-head attention mechanism in BoTNet helps the model better focus on local and global features, enhancing the network's ability to represent small targets. The use of bottleneck blocks significantly reduces computational cost and improves network efficiency. With a large network depth, it can reduce overfitting and improve the network's generalization ability. It can better learn the feature representation of input data, improving model accuracy. In small object detection tasks, objects are relatively small in size and have detailed features. BoTNet's non-linear transformations and cross-layer connections help extract these features and effectively reduce information loss during network propagation. BoTNet's multi-head attention mechanism helps the model better focus on local and global features, enhancing the network's ability to represent small objects. This method significantly improves baseline performance in instance segmentation and object detection while reducing parameters, thus minimizing latency.The MHSA Block is the core innovation of the BoTNet module. Compared with the MHSA in the traditional Transformer, the BoTNet module uses three non-linear activations, normalizes with Batch Norm consistent with CNN, and introduces two-dimensional position encoding, which improves its ability to detect small objects.

[0124] S5) Select a certain number of extended fabric images from the training set and input them into the ECTS-YOLOv7 network model for training. Extract features through the Backbone network and use the ECANet module to obtain one-dimensional vectors corresponding to each channel as evaluation scores to obtain effective features for fabric image defect detection. Then, fuse the features through a path aggregation network. Evaluate the importance of each feature map and assign a weight value to each feature channel based on its importance. This ensures that effective feature maps have higher weights and ineffective or less effective feature maps have lower weights, thereby achieving better training results for the model.

[0125] S6) Select a certain number of extended fabric images from the training set and process them through the ECTS-YOLOv7 network model. Each image generates an effective feature map of a specific layer in the backbone network. A band matrix W is used in the ECANet attention module. k To learn channel attention, after using non-dimensionality-reduced GAP aggregation convolutional features, the kernel size k is adaptively determined first, then one-dimensional convolution is performed, followed by Sigmoid function to learn channel attention, and finally the weights are applied to the original channel features through the path aggregation network to obtain the training prediction feature map of the corresponding layer.

[0126] By introducing the BoTNet module, three non-linear activations are used, and Batch Norm is employed for normalization consistent with CNNs. At the same time, two-dimensional position encoding is introduced, which improves the accuracy of small target detection, effectively reduces the number of parameters, and minimizes latency, which is helpful for practical application and deployment in production.

[0127] S7) For each training prediction feature map, the K prior boxes processed by the K-means++ clustering algorithm are evenly distributed. The K prior boxes are adjusted according to the training prediction feature map, and each prior box is adjusted to obtain a training prediction box. Based on the target GT box, several training prediction boxes are selected as training candidate boxes.

[0128] Specifically, each training prediction feature map is divided into H×W grid cells, where H and W represent the height and width of the training prediction feature map, respectively, and the center of each grid cell is called an anchor point. The K prior boxes processed by the K-means++ clustering algorithm are sorted by scale and evenly divided into N groups of prior boxes. Typically, K is set to 9 and N to 3, ensuring that K / N is an integer. Similarly, the training prediction feature maps are sorted and divided by scale, and each group of prior boxes is assigned to a single training prediction feature map. Prior boxes are then superimposed on anchor points, and the position and scale of each prior box are adjusted based on the position and scale information of each one-dimensional adjustment vector, thus obtaining a training prediction box. For each target ground truth (GT) box, the Intersection over Union (IoU) loss between the target GT box and each training prediction box is calculated. The training prediction box with the largest IoU loss with the target GT box is selected as a training candidate box.

[0129] IoU has two drawbacks: 1. If two validation boxes do not overlap, the IoU will be 0, failing to reflect the distance between the two targets. In the case of non-overlapping validation boxes, using IoU as the loss function results in a gradient of 0, making optimization impossible; 2. IoU cannot distinguish between different alignment methods between two validation boxes. Two overlapping validation boxes with the same intersection level in different directions will have completely equal IoU. Therefore, this invention uses GIoU to solve the problem of non-overlapping validation boxes. That is: for any two validation boxes A and B, first find the smallest box C that can contain them, then calculate the ratio of the area of ​​C\(A∪B) to the area of ​​C, and then subtract this ratio from the IoU values ​​of A and B to obtain the GIoU. When using GIoU as a metric, it is always the lower bound of IoU, possessing non-negativity, and also taking into account non-overlapping areas not considered by IoU, thus reflecting the overlap method of validation boxes A and B.

[0130] The formula for calculating IoU is as follows:

[0131]

[0132] The formula for calculating GIoU is as follows:

[0133]

[0134] When IoU = 0, the formula for calculating GIoU is as follows:

[0135]

[0136] S8) Calculate the overall classification and regression loss of the ECTS-YOLOv7 network model based on the difference between the training predicted feature map, the adjusted candidate box and the target GT box, and backpropagate the overall loss value to the ECTS-YOLOv7 network model. Then, use gradient descent to update the parameters of the ECTS-YOLOv7 network model to obtain the parameter-updated ECTS-YOLOv7 network model.

[0137] S9) Repeat steps S4)-S8) to process each fabric image in the training set. After the selected image is output by S6), repeat steps S7)-S8) to update the parameters of the ECTS-YOLOv7 network model until all images in the training set have been trained. The ECTS-YOLOv7 network model with updated parameters can be used as a pre-trained model.

[0138] S10) The extended fabric images in the validation set are input into the pre-trained ECTS-YOLOv7 network model for processing. After performing the same operations as the training set in step S6), the validation prediction feature maps of the corresponding layers are obtained. Each output validation prediction feature map is processed in the same way as the training prediction feature map in step S7), and a validation prediction box is output. The validation set loss is calculated based on the validation prediction box and the target ground truth box to assess the generalization performance of the model. The average precision (AP) of a single class and the average precision (mAP) of all classes of defects in the fabric defect image dataset after processing by the pre-trained ECTS-YOLOv7 network model are also calculated.

[0139] S11) Repeat steps S9)-S10) to backpropagate the overall loss value to the ECTS-YOLOv7 network model. At the same time, use gradient descent to update and optimize the model parameters until the average precision AP and average precision mAP of each category of defects in the fabric defect image dataset tend to a stable value. At this time, the total loss curve of the network converges, and the trained ECTS-YOLOv7 network model is obtained. Then update the model weight file (.pth file).

[0140] S12) The extended fabric images from the test set are input into the trained ECTS-YOLOv7 network model for processing. The same operation as the training prediction feature map in step S7) is performed on each output test prediction feature map to output test prediction boxes. Blank boxes in the test prediction boxes are removed according to the Non-Maximum Suppression (NMS) principle. The final test prediction boxes are then selected based on confidence levels from the remaining test prediction boxes, thus achieving the detection and localization of fabric defects.

[0141] S13) Analyze the fabric defect detection results in step S12) using the following commonly used target detection metrics: average precision (AP) for a single category, average precision (mAP) for all categories, precision, recall, and FPS, and examine the detection performance of the ECTS-YOLOv7 network model.

[0142] To verify the performance of the detection method proposed in this invention, the German DAGM2007 was used as the fabric image dataset to be detected. This is a surface defect image dataset containing 10 types of defects and various textures. The existing YOLOv7 network model and the ECTS-YOLOv7 network model of this invention were used to predict and compare the above dataset. The comparison of detection data before and after the network structure improvement is shown in Table 2. Table 1 shows the detection data of the improved model of this invention. Figure 5 The curves representing precision versus recall calculated using the model of this invention are shown. Figure 6 This is a comparison chart of the predicted results calculated using the model of this invention and the actual results. Figure 7 The images show the actual detection results of two types of defects selected after using the model of this invention.

[0143] In this example, the prediction results are calculated as follows: mAP@.5 represents the AP of all images in each class when IoU is set to 0.5, and then the average of all classes is calculated; mAP@.5:.95 represents the precision of averaging all classes with different IoU thresholds from 0.5 to 0.95 in steps of 0.05; Precision represents the proportion of accurate predictions; Recall represents the proportion of correct predictions.

[0144] Table (1):

[0145]

[0146] Table (2):

[0147]

[0148] As shown in Table (1), the detection method of the present invention can simultaneously detect multiple types of fabric defects and meet the accuracy requirements of the detection task. As shown in Table (2), the detection method of the present invention has better detection performance than the traditional YOLOv7 model.

[0149] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A fabric defect detection method based on an improved YOLOv7 model, characterized in that, Includes the following: Step 1: Establish a fabric defect image dataset. After converting the image dataset from the RGB color space to the YCrCb color space, divide it into training set, validation set and test set. Step 2: Construct the ECTS-YOLOv7 network model; The ECTS-YOLOv7 network model is obtained by replacing the last layer Bconv of each E-ELAN layer in the YOLOv7 backbone network with the ECANet attention mechanism module, and replacing the original ELAN-W module in the YOLOv7 head network with the BoTNet module. Step 3: Select images from the training set and input them into the ECTS-YOLOv7 network model for training. The ECANet module obtains the vector corresponding to each channel as an evaluation score, evaluates the importance of each feature map, and assigns a weight value to each feature channel according to its importance. Step 4: Select images from the training set and input them into the ECTS-YOLOv7 network model for processing. Each image generates a specific layer of effective feature maps in the backbone network. A band matrix W is used in the ECANet attention module. k To learn channel attention, after using non-dimensionality-reduced GAP aggregation convolutional features, the kernel size k is first adaptively determined, then one-dimensional convolution is performed, and then the Sigmoid function is used to learn channel attention. Finally, the weights are applied to the original channel features through the path aggregation network to obtain the training prediction feature map of the corresponding layer. Step 5: Use the Kmeans++ clustering algorithm to cluster all target GT boxes on the extended fabric image in the image dataset to obtain K prior boxes. Distribute the K prior boxes evenly and adjust them according to the training prediction feature map. Each prior box is adjusted to obtain a training prediction box. Select several training prediction boxes as training candidate boxes based on the target GT boxes. Step 6: Calculate the overall classification and regression loss of the ECTS-YOLOv7 network model based on the difference between the trained predicted feature map, the adjusted candidate box and the target GT box, and backpropagate the overall loss value to the ECTS-YOLOv7 network model. Then, use gradient descent to update the parameters of the ECTS-YOLOv7 network model to obtain the parameter-updated ECTS-YOLOv7 network model. Step 7: Repeat steps 3 to 6 to process each image in the training set. After the selected image is output in step 4, repeat steps 5 and 6 to update the parameters of the ECTS-YOLOv7 network model until all images in the training set have been trained. The ECTS-YOLOv7 network model with updated parameters can be used as a pre-trained model. Step 8: Input the extended fabric images in the validation set into the pre-trained model for processing. After performing the same operation as the training set in Step 4, the validation prediction feature maps of the corresponding levels are obtained. Perform the same operation as the training prediction feature maps in Step 5 on each output validation prediction feature map to output the validation prediction box. Calculate the validation set loss based on the validation prediction box and the target GT box, and calculate the average precision AP of a single category and the average precision mAP of all categories of defects in the fabric defect image dataset after processing by the pre-trained model. Step 9: Repeat steps 7 and 8 until the average precision (AP) and average precision (mAP) of each category of defects in the fabric defect image dataset tend to a stable value. At this point, the total loss curve of the network converges, and the trained ECTS-YOLOv7 network model is obtained. Step 10: Input the extended fabric images in the test set into the trained ECTS-YOLOv7 network model for processing. Perform the same operation on each output test prediction feature map as the training prediction feature map in step 5, and output the test prediction box. Based on the nonmaximum suppression principle of NMS, blank boxes in the test prediction boxes are removed. The final test prediction boxes are then selected from the remaining test prediction boxes according to the confidence level, thereby realizing the detection and location of fabric defects.

2. The fabric defect detection method based on the improved YOLOv7 model according to claim 1, characterized in that: The method for constructing the fabric defect image dataset in step 1 is as follows: Several images of fabrics with defects were acquired using an industrial camera. Each defect in each image was labeled with its category and location using the minimum bounding rectangle in LabelImg software. The target ground truth (GT) box must completely enclose each defect. The target GT box was labeled as (class, xmin, ymin, xmax, ymax), where class represents the category of defect, xmin and ymin represent the x and y coordinates of the top left corner of the target GT box, respectively, and xmax and ymax represent the x and y coordinates of the bottom right corner of the target GT box. Data augmentation was performed on the data to obtain labeled and enhanced fabric images. All collected images and enhanced fabric images were then expanded into a fabric defect image dataset with no fewer than 5,000 images. At the same time, the category label XML files were converted into YOLO label TXT format.

3. The fabric defect detection method based on the improved YOLOv7 model according to claim 1, characterized in that: The method for obtaining the K prior boxes in step 5 is as follows: The Kmeans++ clustering algorithm is used to cluster all target ground truth boxes on each extended fabric image in the image dataset. A sample point is randomly selected from the target ground truth boxes X in the image dataset as the first initial cluster center. The shortest distance D(x) between each sample and the current cluster center is calculated, and the probability P(x) of each sample point being selected as the next cluster center is calculated. ， Finally, the sample point corresponding to the highest probability value is selected as the next cluster center. This selection is repeated until K cluster centers are obtained. The prior boxes are composed of the x-coordinate and y-coordinate of each cluster center as the width and height, and finally K prior boxes are obtained.

4. The fabric defect detection method based on the improved YOLOv7 model according to claim 1, characterized in that: In step 1, the method for converting the RGB color space to the YCrCb color space is as follows: Y'=0.257*R'+0.504*G'+0.098*B'+16 Cb'=-0.148*R'-0.291*G'+0.439*B'+128 Cr'=0.439*R'-0.368*G'-0.071*B'+128 R'=1.164*(Y'-16)+1.596*(Cr'-128) G'=1.164*(Y'-16)-0.813*(Cr'-128)-0.392*(Cb'-128) B'=1.164*(Y'-16)+2.017*(Cb'-128).

5. The fabric defect detection method based on the improved YOLOv7 model according to claim 1, characterized in that: In step 4, the ECA attention mechanism first calculates the average value of each channel through global average pooling, then performs a one-dimensional convolution on the average value of each channel to calculate the attention coefficient of that channel; finally, the attention coefficient is multiplied by the original input data, and the result is reassembled into the same shape as the original input to obtain weighted input data. The attention weight formula is as follows: ， In the above formula, the parameters have the following meanings: X i,j,c Represents the input feature map X The pixel value of the i-th row, j-th column, and c-th channel; σ is the sigmoid activation function; W c H represents the attention weight; H represents the height of the feature map; W represents the width of the feature map. W 1 , c,cj This represents the feature map after global pooling, where c represents the number of classes. j This represents the probability that the j-th channel belongs to the c-th category; The channel attention weighting formula is as follows: , In the above formula, Y c This represents the weighted channel feature map. X c Represents the input feature map X The c-th channel; The formula for deep attention weights is as follows: , In the above formula, the parameters have the following meanings: Represents the weighted channel feature map Y The first at all pixel positions The value of each channel; It is a 1×1 convolution kernel; W d For deep attention weights; The weighted formula for deep attention is as follows: , Where Z represents the weighted output feature map. Y This represents the weighted channel feature map; The formula for calculating the output feature map is as follows: , in, F ( X ) represents a feature extractor.

6. The fabric defect detection method based on the improved YOLOv7 model according to claim 1, characterized in that: In step 7, the training prediction feature map obtained from each training group is divided into H×W grid cells, where H and W represent the height and width of the training prediction feature map, respectively, and the center of the grid cell is called the anchor point. The K prior boxes from step 5 are sorted by scale and evenly divided into N groups of prior boxes. Similarly, the training prediction feature map is sorted and divided by scale, and each group of prior boxes is assigned to a training prediction feature map. Then, prior boxes are superimposed on the anchor points, and the position and scale of each prior box are adjusted according to the position and scale information of each one-dimensional adjustment vector, thereby obtaining a training prediction box. For each target ground truth (GT) box, calculate the intersection-union (IoU) loss between the target GT box and each training prediction box, and select the training prediction box with the largest IoU loss with the target GT box as the training candidate box. For any two validation boxes A and B, first find the smallest box C that can enclose them, then calculate the ratio of the area of ​​C\(A∪B) to the area of ​​C, and then subtract this ratio from the IoU values ​​of A and B to get the GIoU. The formula for calculating IoU is as follows: ， The formula for calculating GIoU is as follows: ， When IoU=0, the formula for calculating GIoU is as follows: .

7. The fabric defect detection method based on the improved YOLOv7 model according to claim 6, characterized in that: K is set to 9 in the K prior boxes and N is set to 3 in the N prior boxes, ensuring that K / N is an integer.

8. The fabric defect detection method based on the improved YOLOv7 model according to claim 1, characterized in that: After performing step 10, the detection results in step 10 are analyzed using the following object detection metrics to determine the detection performance of the ECTS-YOLOv7 network model after training. The object detection metrics include: average precision (AP) for a single class, average precision (mAP) for all classes, precision, recall, and speed (FPS).

Images