A fabric defect detection method, computer device and medium

Abstract

The application belongs to the technical field of image processing, and specifically discloses a fabric defect detection method, computer equipment and a medium. The application builds a fabric defect detection model based on an improved YOLOv8 model, uses a GhostNet V2 bottleneck to replace a bottleneck in a C2f module to obtain a C2f-GhostNet V2 module, uses an optimized SPPCSPC module to replace an original SPPF module, and two convolution layers before a pooling layer of the original SPPCSPC module are cropped off, and a SimAM attention mechanism is introduced; secondly, a neck network PANet structure is changed into an improved BiFPN structure; finally, a small target detection layer is introduced in a detection head part, the number of detection heads is increased from three to four, and a PConv convolution is introduced to improve the detection head to reduce the calculation burden caused by the increase of the small target detection head. Through the model, the detection accuracy and speed of fabric defects are improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and relates to a fabric defect detection method, a computer device and a medium. BACKGROUND

[0002] Fabric defect detection is an important process in quality control in textile production. Current manual visual inspection is still the main quality inspection method, and through three processes of rough inspection, intermediate inspection and final inspection, defect detection is realized. The efficiency of manual detection is easily affected by factors such as mood, body, fatigue and environment, and subjective misjudgment and missed detection are prone to occur, which affects production efficiency and product quality. With the excellent effect of deep learning in image feature extraction and target positioning, it is gradually applied in the field of fabric defect detection.

[0003] The detection algorithm based on deep learning is represented by the single-stage detection algorithm (One-tage) and the two-stage detection algorithm (Two-stage). The Two-stage algorithm first generates target candidate boxes, and then classifies and position regresses the candidate boxes. The Faster RCNN series and Mask RCNN series algorithms are representative, and the detection accuracy of the target is high, but the detection speed is slow, the calculation amount and time complexity are high, and it is difficult to adapt to the real-time requirements of industrial detection. The One-stage algorithm directly uses a convolutional neural network (CNN) to predict the class and position of the target, and does not need to generate candidate regions. The SSD, YOLO series and other algorithms are representative, and the calculation speed is fast and the real-time performance is good, so it is easy to meet the needs of industrial real-time detection.

[0004] Patent document 1 discloses a fabric defect detection method based on an improved YOLOv7 model. The method collects fabric defect pictures and performs data preprocessing on the original image dataset, converts from the RGB color space to the YCrCb color space; combines Bottleneck Transformer, Efficient Channel Attention and YOLOv7 model to improve the YOLOv7 model. This method can effectively solve the influence of noise and small defects in fabric defect images, improve detection accuracy while ensuring processing speed, and has small model deployment difficulty and strong industrial production practicability, but the model can still be improved in detection accuracy and detection time, and the detection result is not ideal for complex background fabric materials.

[0005] Patent Document 2 discloses a fabric defect detection method, device, electronic equipment and storage medium. The fabric defect detection method comprises: obtaining a plurality of defect images of a plant sample, and establishing a first defect data set. Based on the trained DCGAN network, the defect images in the first defect data set are enhanced to obtain enhanced defect images, the defect images of a specified defect category in the enhanced defect images are selected, a second defect data set is established, and the second defect data set is divided into a training sample set and a test sample set. Based on the training sample set and the test sample set, an improved YOLOv5 model is trained to obtain a fabric defect detection model. The improved YOLOv5 model has a Backbone layer and a Neck layer with an attention mechanism module. The fabric defect detection model in Patent Document 2 can improve the detection accuracy, speed up the detection, and reduce the missed detection rate of plant defects. However, the training time of the model is relatively long, and satisfactory results have not been achieved for complex pattern fabrics.

[0006] In summary, although the existing methods have achieved certain results in defect detection, in practical applications, the defect detection accuracy and speed for multi-scale and small target are still the bottleneck problems in the industry. With the emergence of YOLOv8, the accuracy and speed of the target detection algorithm have been further improved. YOLOv8 has higher accuracy and faster speed than the previous YOLO algorithm under the same size.

[0007] References

[0008] Patent Document 1: Chinese Invention Patent Application Publication No. CN117372332A, published on January 9, 2024;

[0009] Patent Document 2: Chinese Invention Patent Application Publication No. CN117830247A, published on April 5, 2024. SUMMARY

[0010] In view of the characteristics of fabric defects of different scales, small targets and variable shapes, the present application proposes a fabric defect detection method based on an improved YOLOv8 model for defect detection to improve the defect detection accuracy and speed.

[0011] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0012] A fabric defect detection method comprises the following steps:

[0013] Step 1. Obtain fabric images with defects, and perform preprocessing and labeling operations on the images to create a training data set;

[0014] Step 2. Build a fabric defect detection model based on the improved YOLOv8 model architecture;

[0015] The improved YOLOv8 model is obtained by improving the original YOLOv8 model as follows:

[0016] Firstly, in the backbone network of the YOLOv8 model, the GhostNet V2 Bottleneck is used to replace the Bottleneck in the C2f module to obtain the C2f-GhostNetV2 module; the optimized SPPCSPC module is used to replace the SPPF module of the original YOLOv8 model, and the reconstructed SPPCSPC module is cropped from two convolution layers before the pooling layer of the original SPPCSPC module, and the SimAM attention mechanism is introduced; secondly, the neck network PANet structure of the YOLOv8 model is changed to the improved BiFPN structure; finally, the small target detection layer is introduced in the detection head part, the number of detection heads in the YOLOv8 model is increased from three to four, and the PConv convolution is introduced to improve the detection head to reduce the calculation burden caused by the increase of the small target detection head;

[0017] Step 3. Training the improved YOLOv8 model in step 2 based on the training data set obtained in step 1 to obtain a trained fabric defect detection model, and using the trained fabric defect detection model to detect fabric defects.

[0018] In addition, on the basis of the fabric defect detection method, the application further provides a computer device, which comprises a memory and one or more processors. The memory stores executable code.

[0019] When the processor executes the executable code, the fabric defect detection method is implemented.

[0020] In addition, on the basis of the fabric defect detection method, the application further provides a computer readable storage medium, which stores a program, and the program is executed by a processor to implement the fabric defect detection method.

[0021] The application has the following advantages:

[0022] As described above, the present application relates to a fabric defect detection method based on a YOLOv8 model for fabric defect detection, wherein the YOLOv8 model is a single-stage target detection algorithm, and both speed and accuracy are taken into account. The present application makes the following improvements based on the original YOLOv8 model. In the backbone network, the GhostNet V2 Bottleneck is used to replace the Bottleneck in the C2f module, which enhances the ability of the model to capture complex feature information. The optimized SPPCSPC module is used to replace the SPPF module, and the two convolution layers before the pooling layer of the SPPCSPC module are cropped to reduce the filtering of small target edge information by the convolution layer. This also reduces the computational amount of the neural network, and the SimAM attention mechanism is introduced to solve the problem of insufficient small target extraction, effectively eliminating interference factors. The neck network PANet structure is changed to the improved BiFPN structure, realizing the effective fusion of bidirectional information flow between feature layers and multi-scale features such as multi-level features. In the detection head part, the small target detection layer is introduced, and the number of detection heads in the YOLOv8 model is increased from three to four. To reduce the computational burden caused by the increase of the small target detection head, the PConv convolution is introduced to improve the detection head, and the parameter sharing mechanism is used to simplify the detection branch, and four lightweight detection heads are designed. The loss function GIOU is used to replace the loss function of the original model. Through the fabric defect detection model based on the improved YOLOv8 model architecture, the detection accuracy and speed of fabric defects are improved. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 The flowchart of the fabric defect detection method in the embodiment of the present application is shown in (a) and (b) of FIG.

[0024] Figure 2 The overall network structure diagram of the improved YOLOv8 model in the embodiment of the present application is shown in (a) and (b) of FIG.

[0025] Figure 3 The network structure diagram of the C2f-GhostNetV2 module in the embodiment of the present application is shown in (a) and (b) of FIG.

[0026] Figure 4 The structure diagram of the GhostNetV2 Bottleneck module in the embodiment of the present application is shown in (a) and (b) of FIG. Figure 4 (a) and (b) respectively show the network structure when the step size is 1 and 2.

[0027] Figure 5 The structure diagram of the Ghost module and the network structure diagram of the DFC attention module in the embodiment of the present application are shown in (a) and (b) of FIG.

[0028] Figure 6 The network structure diagram of the improved SPPCSPC in the embodiment of the present application is shown in (a) and (b) of FIG.

[0029] Figure 7 The structural diagram of the SimAM attention module in the embodiment of the application is shown in the following figure:

[0030] Figure 8 The network structure diagram of the improved BiFPN in the embodiment of the application is shown in the following figure:

[0031] Figure 9 The network structure diagram of the weight fusion module in the embodiment of the application is shown in the following figure:

[0032] Figure 10 The network structure diagram of the lightweight probe head in the embodiment of the application is shown in the following figure. DETAILED DESCRIPTION

[0033] The application will be further described in detail below in combination with the drawings and specific embodiments:

[0034] Embodiment 1

[0035] The embodiment 1 describes a fabric defect detection method, which is based on the YOLOv8 model framework, and the backbone network, neck network and detection head of the YOLOv8 model framework are optimized and innovated. Specifically, first, the GhostNet V2 Bottleneck is used to replace the Bottleneck in the C2f module in the backbone network, which enhances the ability of the model to capture complex feature information; the optimized SPPCSPC module is used to replace the SPPF module, and the two convolutional layers before the pooling layer of the reconstructed SPPCSPC module are cropped to reduce the filtering of small target edge information by the convolutional layer, which also reduces the computational load of the neural network; the SimAM attention mechanism is introduced to solve the problem of insufficient small target extraction, which effectively eliminates the interference factors. Secondly, the PANet structure of the neck network is changed to the improved BiFPN structure, which realizes the effective fusion of bidirectional information flow between feature layers and multi-scale features such as multi-level features; finally, the small target detection layer is introduced in the detection head part, and the number of detection heads in the YOLOv8 model is increased from three to four. In order to reduce the computational burden caused by the increase of the small target detection head, the PConv convolution is also introduced to improve the detection head, and the parameter sharing mechanism is used to simplify the detection branch, and four lightweight detection heads are designed.

[0036] As shown in the following figure, the fabric defect detection method in the embodiment includes the following steps: Figure 1

[0037] Step 1. Obtain the fabric image with defects, and perform preprocessing and labeling operations on the image to make a training data set.

[0038] ​The fabric images are collected using an industrial camera for fabric defect detection dataset. The collected images are subjected to homomorphic filtering, size normalization, grayscale processing, angle adjustment, and other data augmentation operations. The size normalization specification is 640x640.

[0039] The collected fabric dataset is subjected to data annotation processing.

[0040] For example, for the TILDA dataset, the present application uses Labelimg software to label the defect images according to four types of defects: holes, foreign matter, oil stains, and broken lines, to obtain a labeled fabric image dataset.

[0041] The preprocessed dataset is divided into training set, validation set, and test set according to the ratio of 8:1:1 for model training.

[0042] The model will predict the bounding box of each defect in the fabric image, i.e., the specific location of the defect in the image. These bounding boxes are usually represented by four parameters: center coordinates (x, y) and the width (w) and height (h) of the bounding box.

[0043] In addition to the location information, the model will also assign one or more class labels to each predicted defect, indicating the specific type of the defect. These class labels usually correspond to the defect types labeled during training.

[0044] In addition, a confidence score can be output for each predicted defect location and class, indicating the degree of certainty of the model's prediction. The higher the confidence score, the more confident the model is that the prediction is correct.

[0045] Step 2. Build a fabric defect detection model based on the improved YOLOv8 model architecture.

[0046] Firstly, in the backbone network of the YOLOv8 model, the GhostNet V2 Bottleneck is used to replace the Bottleneck in the C2f module to obtain the C2f-GhostNetV2 module, which enhances the ability of the model to capture complex feature information; the optimized SPPCSPC module is used to replace the original SPPF module of the YOLOv8 model, and the two convolutional layers before the pooling layer of the original SPPCSPC module are cropped to reduce the filtering of small target edge information by the convolutional layers, which also reduces the computational load of the neural network. At the same time, the SimAM attention mechanism is introduced to solve the problem of insufficient small target extraction, effectively eliminating interference factors; secondly, the neck network PANet structure is changed to the improved BiFPN structure, realizing the effective fusion of bidirectional information flow and multi-scale features between feature layers; finally, the small target detection layer is introduced in the detection head part, and the number of detection heads in the YOLOv8 model is increased from three to four. To reduce the computational burden caused by the addition of the small target detection head, PConv convolution is introduced to improve the detection head, and the parameter sharing mechanism is used to simplify the detection branch, and four lightweight detection heads are designed. The improved YOLOv8 network model not only ensures the detection accuracy, but also realizes lightweight and real-time to meet the industrial demand.

[0047] The input image enters the backbone network part and is processed by the Conv, C2f-GhostNetV2 module and SPPCSPC module to extract image features; the output feature map enters the neck network part, and through the fusion of high-level features and low-level features, high-resolution information and high-semantic information are integrated; in the head network part, the feature information is decoupled by the four lightweight detection heads to accurately locate the position and category of the target object.

[0048] As shown in Figure 2 , the improved YOLOv8 model in the embodiment includes a backbone network, a neck network and a detection head.

[0049] The backbone network includes a convolutional block, a C2f-GhostNetV2 module and a SPPCSPC module.

[0050] The convolutional block has five and is defined as the first, second, third, fourth and fifth convolutional blocks respectively; the C2f-GhostNetV2 module has four and is defined as the first, second, third and fourth C2f-GhostNetV2 modules respectively.

[0051] The processing flow of the backbone network is as follows:

[0052] The input feature map first passes through the first and second convolutional blocks, then passes through the first C2f-GhostNetV2 module, then passes through the third convolutional block, then passes through the second C2f-GhostNetV2 module, then passes through the fourth convolutional block, then passes through the third C2f-GhostNetV2 module, then passes through the fifth convolutional block, then passes through the fourth C2f-GhostNetV2 module, and then outputs after passing through the SPPCSPC module.

[0053] The neck network comprises an up-sampling module, a weight fusion module, a SimAM attention mechanism module and a convolutional block.

[0054] There are three up-sampling modules, which are defined as the first, second and third up-sampling modules respectively; there are six weight fusion modules, which are defined as the first, second, third, fourth, fifth and sixth weight fusion modules respectively; there are four SimAM attention mechanism modules, which are defined as the first, second, third and fourth SimAM attention mechanism modules respectively; and there are seven convolutional blocks, which are defined as the sixth, seventh, eighth, ninth, tenth, eleventh and twelfth convolutional blocks respectively.

[0055] The processing flow of the neck network is as follows:

[0056] The output feature of the SPPCSPC module passes through the sixth convolutional block and the first up-sampling module in turn and enters the first weight fusion module, while the output feature of the third C2f-GhostNetV2 module passes through the seventh convolutional block to the first weight fusion module in turn, and the output features of the first up-sampling module and the seventh convolutional block are fused in the first weight fusion module;

[0057] The output feature of the first weight fusion module passes through the first SimAM attention mechanism module and the second up-sampling module in turn and enters the second weight fusion module, while the output feature of the second C2f-GhostNetV2 module passes through the eighth convolutional block to the second weight fusion module in turn, and the output features of the second up-sampling module and the eighth convolutional block are fused in the second weight fusion module;

[0058] The output feature of the second weight fusion module and the output feature of the first C2f-GhostNetV2 module enter the second SimAM attention mechanism module for adaptive weighting operation;

[0059] The output feature of the second SimAM attention mechanism module passes through the third up-sampling module and the ninth convolutional block in turn and enters the third weight fusion module, and is fused with the output feature of the first C2f-GhostNetV2 module in the third weight fusion module;

[0060] The output feature of the third weight fusion module sequentially passes through a third SimAM attention mechanism module and an eighth convolutional block, and enters a fourth weight fusion module, and the output feature of the eighth convolutional block is fused in the fourth weight fusion module;

[0061] The output feature of the fourth weight fusion module sequentially passes through a fourth SimAM attention mechanism module and an eleventh convolutional block, and enters a fifth weight fusion module, and the output feature of the seventh convolutional block is fused in the fifth weight fusion module;

[0062] The output feature of the fifth weight fusion module sequentially passes through a fifth SimAM attention mechanism module and a twelfth convolutional block, and enters a sixth weight fusion module, and the output feature of the sixth convolutional block is fused in the sixth weight fusion module;

[0063] The detection head has four, and the third, fourth, fifth and sixth weight fusion modules are connected with one detection head respectively.

[0064] In the original YOLOv8 model, the traditional convolutional network is prone to produce a large number of similar feature maps when extracting global information, which leads to the defects of large model parameter quantity and long calculation time.

[0065] The feature fusion part of the application introduces a GhostNet V2 module to realize lightweight operation of the model. The C2f-GhostNetV2 module includes two convolutional layers and n GhostNet V2 Bottleneck modules.

[0066] As shown in Figure 3 The processing flow of the C2f-GhostNetV2 module is as follows: the input feature map first passes through a convolutional layer to generate a feature map containing basic level information. Then, the feature map is divided into two parts in the channel dimension through a channel segmentation operation, which is used to process different dimensional feature information in parallel. The main branch focuses on feature extraction, and through n GhostNet V2 Bottleneck modules, the segmented feature map is deeply learned and represented. At the same time, the other part of the feature map passes along a parallel gradient flow branch to preserve the original feature information. Then, the feature map after parallel processing of feature extraction and transmission passes through a feature fusion module, which fuses the complementary feature information from different branches. This step can integrate the advantages of each branch to form a more rich and comprehensive feature representation. Finally, the fused feature map is processed through a convolutional layer for feature enhancement, which learns the nonlinear relationship between features through further learning, and outputs the final output feature map for use in subsequent network layers.

[0067] The GhostV2 Bottleneck module is a reverse residual bottleneck structure, mainly consisting of a DFC module and two Ghost modules, as shown in the diagram. Figure 4 As shown, where Figure 4 Images (a) and (b) show the network structures with strides of 1 and 2, respectively. The first Ghost module runs in parallel with the DFC module branch, extracting features from the input information to enhance and expand the features. Then, element-wise multiplication is used to enrich the semantic information. When the stride is 2, a depthwise separable convolutional module is added, and the features are then fed into the second Ghost module for channel compression to produce the output features. This structure enables GhostV2 to achieve long-range encoding capabilities, greatly enhancing the network's feature extraction ability.

[0068] The stride is the number of pixels the convolutional kernel traverses in each movement as it slides across the input feature map. A stride of 1 means the kernel moves one pixel at a time, while a stride of 2 means it moves two pixels at a time. In GhostNetV2, strides of 1 and 2 refer to the size of the step the convolutional layer takes as it moves across the input feature map.

[0069] Depthwise separable convolution algorithms are relatively easy to implement. They use a convolution kernel on each channel and concatenate the outputs of each layer to obtain the final output. Depthwise separable convolution is computationally significantly more efficient than ordinary convolution.

[0070] The structure diagram of the Ghost module is as follows: Figure 5 As shown in (a), the input feature map in this Ghost module generates a large number of feature maps through identity mapping and a small number of convolution operations. This helps reduce the computational cost and number of parameters in the network while maintaining high performance. Specifically, the Ghost module can first generate a portion of the feature maps (called main feature maps) through a small number of regular convolution operations, and then generate more feature maps (called phantom feature maps) from the main feature maps through inexpensive operations (such as linear transformations), thereby achieving a "multiplication" of feature maps. In this way, the Ghost module can significantly reduce computational complexity and model size while maintaining network performance. "Identity mapping" means that the input data bypasses some processing layers and is directly transmitted to subsequent layers. This helps to retain some information of the original data and avoids the loss of important features in deeper layers of the network.

[0071] The structure diagram of the DFC module is as follows: Figure 5 As shown in (b), channel features are aggregated into a single value through global average pooling. These values ​​are then processed by horizontal and vertical fully connected layers and superimposed to form an attention map.

[0072] This step effectively enhances the network's performance in capturing long-range spatial position dependence, significantly reduces computational complexity by decoupling the fully connected manner, and maintains high deployment efficiency for mobile devices.

[0073] The SPPF network has limited ability in detecting small defects on fabric. In order to improve its defect detection capability and accurately locate the defects, the original SPPF module is replaced by an SPPCSPC module. The novel SPPCSPC module integrates a spatial pyramid pool (SPP) and a cross-level partial channel (CSPC), and its performance exceeds that of a single SPPC.

[0074] By adopting serial computation, the SPPCSPC module can adapt to images of different resolutions and accelerate the convergence speed. In order to better adapt to the spatial pyramid pool of the fabric surface features, the SPPCSPC module is restructured.

[0075] The network structure diagram of the improved SPPCSPC is shown in Figure 6 .

[0076] After the SPPCSPC module replaces the SPPF module, the following optimizations are performed: the two convolutional layers located before the pooling layer of the SPPCSPC module are trimmed to reduce the filtering of small target edge information by the convolutional layers, which also reduces the computational load of the neural network. Secondly, the SimAM attention mechanism is introduced to solve the problem of insufficient small target extraction. SimAM allocates a 3D attention weight to each neuron, and divides the pixels between complex background areas and small target areas; SimAM is added before the pooling layer and the final channel merging to help divide the defect area and the non-defect area, enhance the network's attention to the small target area, reduce feature loss, improve the robustness of the model by reducing the influence of the fabric background on the defects.

[0077] The processing process of the SPPCSPC module is as follows: first, the SimAM attention mechanism is used to process the input feature map, emphasizing the key features of small targets and subdividing complex backgrounds and small target areas; secondly, different sizes (such as 5x5, 7x7, 9x9) of pooling kernels are used to extract features at multiple scales to more comprehensively capture the feature information of small targets; thirdly, feature fusion is performed on the branch flow to enhance feature representation; finally, the SimAM attention mechanism is used to further focus on dense small target areas to weaken the interference of complex backgrounds and improve the robustness of the model, and the final feature map is output.

[0078] As shown in Figure 7As shown, SimAM is a simple attention mechanism that can effectively improve the performance of the model. The design idea of SimAM comes from SENet, but unlike the complex structure of SENet, SimAM only uses a global pooling layer and several fully connected layers. Most existing attention modules generate 1-D or 2-D weights from features, and then extend the generated weights to channel and spatial attention. SimAM attention mechanism directly uses features to generate 3-D weights without generating additional parameters.

[0079] In each subgraph, the same color represents each channel, and a single scalar is used for each point on the spatial position or feature. Compared with equations (1)-(3), SimAM can derive the weight of each neuron in the external lesion potential feature map by defining a minimization energy function of linear separability:

[0080] (1)

[0081] (2)

[0082] (3)

[0083] where and represent the mean and variance of all neurons except is the number of neurons in each channel, , represents the height (Height), that is, the size of the data in a certain dimension, represents the width (Width), which refers to the size of the data in the dimension perpendicular to the height, represents the neuron index, and represent the target neuron and other neurons in a single channel of the input feature, represents a hyperparameter, and the energy is lower, the more distinct the neuron is from the adjacent neurons, and the more important the neuron is to the visual processing, and the importance of each neuron can be represented by .

[0084] The refinement formula of the entire module is represented by equation (4):

[0085] (4)

[0086] ​where X is the input image matrix, and refers to the neuron value at a certain channel (Channel), height (Height), and width (Width) position in the feature map. These values reflect certain feature information of the image at that position. represents the attention weight after weighted fusion, is the grouping of all energy related dimensions in the channel and spatial dimensions, and the relative importance of each neuron is ensured by combining function to limit some values from being too large.

[0087] SimAM considers the spatial dimensions (i.e., height and width) and channel dimensions when calculating attention weights, is an adaptive weighted average of the values of all non-target neurons (i.e., neurons other than the one currently being evaluated) within a certain channel. This mean value serves as a benchmark for evaluating the difference between the current neuron and its surrounding neurons.

[0088] The bidirectional connection in the weighted bidirectional feature pyramid network (BiFPN) helps better capture feature representations at different scales, improving the network's ability to handle objects of different sizes and complexities. This is particularly important in object detection tasks. Through the weighted bidirectional feature pyramid network, features can be quickly and effectively fused into each level. Through multi-level feature fusion, it can improve the fusion effect of features, and through the up-down sampling structure, it can reduce the parameter calculation amount, realizing the detection of images of different scales. However, when using the YOLOv8 model for fabric defect detection, the previous feature fusion method cannot accurately distinguish the importance of different scale features, and some scale features may be more important than others, having a greater impact on the results, leading to an imbalance in the contribution of these features to the final results.

[0089] To solve this problem, the original BiFPN structure is improved, that is, the original 5 feature inputs of the BiFPN structure are modified to 4 feature inputs adapted to the YOLOv8 model structure. Because the YOLOv8 target detection network contains three sizes of detection heads, small, medium and large targets are detected respectively. If the input image is 640 size, then the feature maps of 80, 40 and 20 sizes can be corresponded respectively, and the targets greater than or equal to 8, 16 and 32 pixel sizes are detected. For high-resolution images of fabric defect detection, the proportion of the target relative to the original image is generally less than 5%. The application adds a small target detection layer to alleviate the problem of serious loss of small targets in fabric defect images. After adding the small target detection layer, when the input image is 640 size, a feature map of 160 size is obtained. After adding the small target detection layer, the optimized BiFPN network supports 4 feature outputs.

[0090] The improved BiFPN model structure is shown in Figure 8 With a picture of 640x640 size as input, each weight fusion module in the neck network will fuse the skip connection from the backbone network, and cross-layer feature fusion between features of different scales will be performed to better capture target information of different scales. After the weight fusion module, the SimAM attention mechanism is added to emphasize the key features of small targets and subdivide the complex background and small target area. The model finally outputs prediction information corresponding to P2, P3, P4 and P5 level features, predicting very small, small, medium and large targets.

[0091] Figure 8 TD means top-down, which is used to describe the top-down feature fusion path in the feature pyramid.

[0092] P2 comes from a shallower convolutional layer, whose receptive field is small and captures more detailed information. P5 may come from a deeper convolutional layer, whose receptive field is large and captures more semantic information but less detailed information.

[0093] The specific source of P2 to P5 layers is Figure 2 the four outputs of the top-down backbone network. After processing the input feature maps of P2 to P5 layers, a series of output feature maps fused with multi-scale information are generated. These output feature maps not only contain the semantic and detailed information of the original feature maps, but also enhance the feature representation ability through weighted fusion. The output feature maps after weighted fusion are the inputs corresponding to the prediction of very small, small, medium and large targets in turn.

[0094] On the basis of network structure modification, the application also designs a module named weight fusion, which aims to improve the model performance through advanced feature fusion strategy. The structure diagram of the weight fusion module is as followsFigure 9 As shown.

[0095] The weight fusion module adjusts each layer feature map through multiple 1x1 convolutions, splices these feature maps together, and generates attention weights through a convolution layer with a Sigmoid activation function. Then, through the Sum module, the input multiple attention weight feature maps are merged into one through summation, so as to integrate the multi-level features and finally output the feature map after weighted fusion processing, thereby strengthening the recognition ability of the fabric surface defect features and improving the detection accuracy and stability.

[0096] The improved BiFPN allows features to be passed between different levels, which helps to capture the context information of the features and extract and strengthen the detailed features of small targets. The weighted BiFPN effectively enhances the model's recognition ability for small size defects through a carefully designed multi-scale feature fusion strategy, especially combined with the weight fusion module. Multi-scale fusion provides rich context information for small targets, helping the model accurately distinguish and locate the defect area in a complex background, significantly improving the detection accuracy. Through these advantages, the improved BiFPN can achieve good results in the task of fabric defect detection.

[0097] In addition, the present application also designs four lightweight detection heads, which improve the detection layer by increasing the small target detection head. The improved lightweight target detection head has the following characteristics:

[0098] 1. Efficiency: can complete the target detection task in a short time.

[0099] 2. Accuracy: can maintain high-speed detection while having high target detection accuracy.

[0100] 3. Universality: can adapt to new data sets and scenes, maintaining high detection performance.

[0101] In addition, in order to reduce the computational burden caused by the addition of the small target detection head, the PConv convolution is introduced to improve the lightweight detection head. The improved lightweight detection head is as shown in Figure 10 .

[0102] The feature map first passes through the PConv convolution module, then passes through the 1*1 convolution to reduce the channel number and reduce the calculation amount, and then is converted into the corresponding channel number. One branch uses Bbox Loss to reduce the error between the boundary box predicted by the model and the real boundary box, improve the positioning accuracy of the model to the target object position, and the other branch uses Cls Loss to reduce the difference between the model predicted category and the actual category, thereby improving the model's recognition ability to the target object category. Finally, the loss is calculated, which can effectively reduce the feature redundancy in the convolutional neural network, promote the learning of key features, share the parameters of the original double-branch structure, and form a smaller and lighter head structure. Using PConv convolution instead of conventional convolution can greatly reduce the calculation amount of the model, reduce information loss and enhance the expression ability of the model. The calculation amount is:

[0103] (5)

[0104] (6)

[0105] In the formula: is the size of the convolution kernel; is the channel participating in the convolution; is the participation rate of the convolution, which is generally taken as 0.25 in actual application, that is, the calculation amount of PConv convolution is 1 / 16 of the ordinary convolution.

[0106] The memory access amount of PConv is

[0107] (7)

[0108] The ordinary memory access amount is:

[0109] (8)

[0110] Since the convolution process occupies less memory, the memory access amount of PConv convolution is only times of the ordinary convolution.

[0111] The present application only calculates the input channel part, and the subsequent features are not lost. In addition, a 1*1 convolution layer is used in the reserved channel to extract features again, so as to ensure that all feature information is transmitted in the channel.

[0112] Step 3. Training the improved YOLOv8 model in step 2 based on the training data set obtained in step 1 to obtain a trained fabric defect detection model, and using the trained fabric defect detection model to detect fabric defects.

[0113] Specifically, the training set and the validation set are respectively sent into the improved YOLOv8 model constructed to complete the training of the model, the training number is set to 200, and the initial learning rate is 0.01. The test set image is input into the optimized model obtained after training for detection, and after the detection is completed, the fabric defect image containing the defect label frame and the detection confidence is obtained.

[0114] During the training process of the model, the loss function GIOU is used to replace the loss function of the original YOLOv8 model.

[0115] In defect detection, the intersection over union (IOU) is used as an index to evaluate the similarity between the detection frame and the real frame; its calculation is to divide the intersection of the predicted frame A and the real frame B by the union of the two:

[0116] (9)

[0117] wherein, represents the intersection over union, the higher the value of IOU, the higher the coincidence degree of A frame and B frame, which represents that the model detection is more accurate; otherwise, the model performance is poorer.

[0118] The original YOLOv8 model boundary regression loss function adopts CIOU Loss+DFL Loss, and its calculation formula is:

[0119] (10)

[0120] (11)

[0121] (12)

[0122] wherein, is the real distance between and , is the boundary center point of the predicted frame and the real frame, is the diagonal distance of the smallest closed region that can contain the predicted frame and the real frame, and are the width and height of the predicted frame, and are the width and height of the real frame, is a weight function, used to measure the consistency of the width-height ratio. However, CIOU does not consider the balance of difficult and easy samples, which will also hinder the effective optimization of the model similarity.

[0123] However, CIOU does not consider the balance of difficult and easy samples, which will also hinder the effective optimization of the model similarity.

[0124] ​Therefore in the method of the present application, GIOU Loss is selected, whose formula is:

[0125] (13)

[0126] wherein, represents the loss function GIOU; first calculate the minimum closed region area of the prediction box and the real box (the area of the minimum box containing the prediction box and the real box), then calculate the area in the closed region which does not belong to the two boxes . wherein represents the area of the union region of the prediction box and the real box. Finally, subtract the proportion of from the IOU to obtain the GIOU.

[0127] GIOU introduces the minimum circumscribed box on the basis of IOU, solves the problem that the loss is equal to 0 when the detection box and the real box do not overlap, and thus can better reflect the coincidence degree of the prediction box and the real box.

[0128] Finally, the present application adopts precision (P), recall (R) and mean average precision (mAP) as the evaluation standard of the fabric defect detection model.

[0129] After the model training is completed, the trained fabric defect detection model is used for fabric defect detection to obtain a detection result. Through the fabric defect detection model provided by the present application, the detection precision and speed of the fabric defect are improved.

[0130] Embodiment 2

[0131] This embodiment 2 describes a computer device, which comprises a memory and one or more processors. The executable code is stored in the memory. When the processor executes the executable code, the steps of the fabric defect detection method in the above-mentioned embodiment 1 are implemented.

[0132] The computer device in this embodiment is any device or apparatus with data processing capability, which will not be described here.

[0133] Embodiment 3

[0134] This embodiment 3 describes a computer readable storage medium, which stores a program. When the program is executed by a processor, the steps of the fabric defect detection method in the above-mentioned embodiment 1 are implemented.

[0135] The computer readable storage medium can be an internal storage unit of any data processing capable device or apparatus, such as a hard disk or a memory, or an external storage device of any data processing capable device, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc.

[0136] Of course, the above description is merely preferred embodiments of the present application, and the present application is not limited to the above-described embodiments. It should be understood that any equivalent substitutions or obvious modifications made by those skilled in the art based on the teachings of the present specification fall within the scope of the present specification, and should be protected by the present application.

Claims

1. A fabric defect detection method characterized by, Comprising the following steps: Step 1. Obtain the image of the fabric with defects, and perform preprocessing and labeling operations on the image to create a training data set; Step 2. Build a fabric defect detection model based on the improved YOLOv8 model architecture; The improved YOLOv8 model is obtained by improving the original YOLOv8 model as follows: First, in the backbone network of the YOLOv8 model, the GhostNet V2 Bottleneck is used to replace the Bottleneck in the C2f module to obtain the C2f-GhostNetV2 module; the optimized SPPCSPC module is used to replace the original SPPF module of the YOLOv8 model, and the reconstructed SPPCSPC module is cut off at the two convolutional layers before the pooling layer of the original SPPCSPC module, and the SimAM attention mechanism is introduced; secondly, the neck network PANet structure of the YOLOv8 model is changed to the improved BiFPN structure, in which the original 5 feature inputs are modified to adapt to the 4 feature inputs of the YOLOv8 model, and the optimized BiFPN network supports 4 feature outputs; each weight fusion module in the neck network will fuse the skip connection from the backbone network, and cross-layer feature fusion between different scale features is also performed; The SimAM attention mechanism is added after the weight fusion module, which emphasizes the key features of small targets and subdivides the complex background and small target regions; finally, a small target detection layer is introduced in the detection head, the number of detection heads in the YOLOv8 model is increased from three to four, PConv convolution is introduced to improve the detection head, and the parameter sharing mechanism is used to simplify the detection branch; in the detection head, the feature map first passes through the PConv convolution, then passes through the 1x1 convolution to reduce the channel number and reduce the calculation amount, and finally passes through the convolution to become the corresponding channel number; the loss function GIOU is used to replace the loss function of the original YOLOv8 model; the weight fusion module adjusts the feature maps of each layer through multiple 1x1 convolutions, concatenates these feature maps together, and generates attention weights through a convolution layer with a Sigmoid activation function, then merges multiple attention weight feature maps into one through summation through a Sum module, to integrate multiple levels of features and finally output the feature map after weighted fusion processing, to strengthen the recognition ability of the fabric surface defect features; Step 3. Train the improved YOLOv8 model in step 2 based on the training data set obtained in step 1 to obtain a trained fabric defect detection model, and use the trained fabric defect detection model to detect fabric defects.

2. The fabric defect detection method according to claim 1, characterized in that, The improved YOLOv8 model includes a backbone network, a neck network, and a detection head; The backbone network includes a convolution block, a C2f-GhostNetV2 module, and an SPPCSPC module; The five convolutional blocks are respectively defined as a first, a second, a third, a fourth and a fifth convolutional block; and the four C2f-GhostNetV2 modules are respectively defined as a first, a second, a third and a fourth C2f-GhostNetV2 module; The processing procedure of the backbone network is as follows: The input feature map firstly passes through the first and second convolutional blocks, then passes through the first C2f-GhostNetV2 module, then passes through the third convolutional block, then passes through the second C2f-GhostNetV2 module, then passes through the fourth convolutional block, then passes through the third C2f-GhostNetV2 module, then passes through the fifth convolutional block, then passes through the fourth C2f-GhostNetV2 module, and then outputs after passing through the SPPCSPC module; The neck network comprises three up-sampling modules, six weight fusion modules, four SimAM attention mechanism modules and seven convolutional blocks; The three up-sampling modules are respectively defined as a first, a second and a third up-sampling module; the six weight fusion modules are respectively defined as a first, a second, a third, a fourth, a fifth and a sixth weight fusion module; the four SimAM attention mechanism modules are respectively defined as a first, a second, a third and a fourth SimAM attention mechanism module; and the seven convolutional blocks are respectively defined as a sixth, a seventh, an eighth, a ninth, a tenth, an eleventh and a twelfth convolutional block; The processing procedure of the neck network is as follows: The output feature of the SPPCSPC module passes through the sixth convolutional block and the first up-sampling module in sequence, and enters the first weight fusion module, while the output feature of the third C2f-GhostNetV2 module passes through the seventh convolutional block to the first weight fusion module in sequence, and the output features of the first up-sampling module and the seventh convolutional block are fused in the first weight fusion module; The output feature of the first weight fusion module passes through the first SimAM attention mechanism module and the second up-sampling module in sequence, and enters the second weight fusion module, while the output feature of the second C2f-GhostNetV2 module passes through the eighth convolutional block to the second weight fusion module in sequence, and the output features of the second up-sampling module and the eighth convolutional block are fused in the second weight fusion module; The output features of the second weight fusion module and the first C2f-GhostNetV2 module enter the second SimAM attention mechanism module for adaptive weighting operation; The output feature of the second SimAM attention mechanism module passes through the third up-sampling module and the ninth convolutional block in sequence, and enters the third weight fusion module, and the output feature of the first C2f-GhostNetV2 module is fused in the third weight fusion module; The output feature of the third weight fusion module passes through the third SimAM attention mechanism module and the tenth convolutional block in sequence, and enters the fourth weight fusion module, and the output feature of the eighth convolutional block is fused in the fourth weight fusion module; The output feature of the fourth weight fusion module sequentially passes through a fourth SimAM attention mechanism module and an eleventh convolutional block, and enters a fifth weight fusion module, where the output feature is fused with the output feature of a seventh convolutional block; The output feature of the fifth weight fusion module sequentially passes through a fifth SimAM attention mechanism module and a twelfth convolutional block, and enters a sixth weight fusion module, where the output feature is fused with the output feature of a sixth convolutional block; The detection head has four, and the third, fourth, fifth and sixth weight fusion modules are respectively connected with one of the detection heads.

3. The fabric defect detection method of claim 2, wherein, The C2f-GhostNetV2 module comprises two convolutional layers and n GhostNet V2 Bottleneck modules; The processing procedure of the C2f-GhostNetV2 module is as follows: the input feature map first passes through a convolutional layer to generate a feature map containing basic level information; then, the feature map is divided into two parts in the channel dimension through a channel segmentation operation; the main branch focuses on feature extraction, and the segmented feature map is deeply learned and represented through n GhostNet V2 Bottleneck modules; at the same time, the other part of the feature map is transmitted along a parallel gradient flow branch to retain the original feature information; Then, the feature map after parallel processing of feature extraction and transmission passes through a feature fusion module, which fuses the complementary feature information from different branches; finally, the fused feature map is processed through a convolutional layer for feature enhancement, and the final output feature map is output.

4. The fabric defect detection method of claim 1, wherein, The SPPCSPC module replaces the SPPF module and is optimized as follows: the two convolutional layers before the pooling layer of the SPPCSPC module are cropped to reduce the filtering of small target edge information by the convolutional layers and reduce the computational load of the neural network; secondly, the SimAM attention mechanism is introduced to solve the problem of insufficient small target extraction, and SimAM allocates a 3D attention weight to each neuron, while dividing the pixels between complex background areas and small target areas; SimAM is added before the pooling layer and the final channel merging to help divide the defect area and the non-defect area, enhance the attention of the network to the small target area, reduce feature loss, improve the robustness of the model by reducing the influence of the fabric background on the defect, and improve the robustness of the model; The processing procedure of the SPPCSPC module is as follows: in the SPPCSPC module, the input feature map is first processed by the SimAM attention mechanism, which emphasizes the key features of small targets and subdivides complex backgrounds and small target areas; secondly, different sizes of pooling kernels are used to extract features at multiple scales to more comprehensively capture the feature information of small targets; and thirdly, the branch flow is fused to enhance feature representation. Finally, the SimAM attention mechanism is used to further focus on the dense small target area to reduce the interference of complex background and improve the robustness of the model, and the final feature map is output.

5. The fabric defect detection method of claim 1, wherein, In the detection head, the feature map first passes through the PConv convolution module, then passes through the 1x1 convolution to reduce the channel number and reduce the calculation amount, and then is converted into the corresponding channel number through the convolution, one branch uses the boundary box loss Bbox Loss to reduce the error between the predicted boundary box of the model and the real boundary box, thereby improving the positioning accuracy of the model to the position of the target object, and the other branch uses the classification loss Cls Loss to reduce the difference between the predicted category of the model and the actual category, thereby improving the identification ability of the model to the category of the target object; finally, the loss is calculated to effectively reduce the feature redundancy in the convolutional neural network, promote the learning of key features, share the parameters of the original double-branch structure, and form a smaller and lighter head structure.

6. The fabric defect detection method of claim 1, wherein, In step 3, during the training process of the improved YOLOv8 model, the loss function GIOU is used to replace the loss function of the original YOLOv8 model; and the intersection over union IOU is used as an index to evaluate the similarity between the detection frame and the real frame. The calculation of the intersection over union IOU is the intersection of the predicted frame A and the real frame B divided by the union of the two: ； First, calculate the minimum closure area of ​​the predicted bounding box and the ground truth bounding box. Then calculate the regions within the closure area that do not belong to either of the two boxes. Finally, subtract from IOU. The specific gravity is obtained from GIOU using the following formula: ； wherein, represents a loss function GIOU, represents the union area of the predicted box A and the real box B.

7. A computer device comprising a memory and one or more processors; the memory having stored therein executable code; the computer device being characterized by, When the processor executes the executable code, the fabric defect detection method of any one of the above claims 1 to 6 is implemented.

8. A computer readable storage medium having stored thereon a program; characterized in that, When the processor executes the executable code, the fabric defect detection method of any one of the above claims 1 to 6 is implemented.

Images