A fabric defect detection method based on an improved YOLOv7-Tiny network model

Abstract

The application provides a fabric defect detection method based on an improved YOLOv7-Tiny network model, which comprises the following steps: classifying fabric defects into hole defects, long strip defects and short strip defects, collecting a fabric defect dataset according to the fabric defect classification, and dividing the fabric defect dataset into a training set, a verification set and a test set; improving a YOLOv7-Tiny network model having an input unit, a backbone network unit and a head network unit to construct a fabric defect detection model; training the fabric defect detection model through the training set and the verification set, and finally testing the trained fabric defect detection model through the test set; and deploying the fabric defect detection model that passes the final test to detect defects of a target fabric. The application introduces an SPD-Conv module and a mixed attention mechanism to construct a new fabric defect detection model, enhances the feature extraction capability of the model, improves the fabric defect detection precision, and greatly reduces the calculation amount.

Description

TECHNICAL FIELD

[0001] The application relates to a fabric defect detection method, in particular to a fabric defect detection method based on an improved YOLOv7-Tiny network model, and belongs to the technical field of image recognition. BACKGROUND

[0002] The target detection network can obtain accurate target position and classification information at the same time, and is the best network for completing the fabric defect detection task. The target detection network can be divided into a single-stage network and a two-stage network. The two-stage target detection network needs to extract a selection frame according to an image first, and then performs secondary correction based on the candidate region to obtain a detection result. The detection accuracy is high but the speed is slow, and the two-stage target detection network mainly includes R-CNN, Fast R-CNN, Faster R-CNN, Mask R-CNN and Libra R-CNN. The single-stage target detection network directly calculates the classification and positioning of the target based on the image, the detection accuracy is slightly low but the detection speed is fast, and the single-stage target detection network mainly includes a YOLO series, SSD and RetinaNet. Compared with the two-stage target detection network, the single-stage target detection network can have both detection efficiency and detection accuracy, and is more popular in industrial target detection.

[0003] YOLOv7 is one of the best single-stage target detection networks for realizing defect detection at present. However, due to the complex fabric texture background, the various defect types and shapes, and the different sizes of defects, some subtle defects are very similar to the fabric texture and are also difficult for the human eye to distinguish. It is difficult to directly use the YOLOv7 network for fabric defect detection. Especially, a large circular machine is often used in fabric manufacturing, and the fabric is manufactured by the rotation of the needle line of the large circular machine. Imaging and detection of the fabric defects manufactured by the large circular machine face the following difficulties: 1. The fabric has complex texture, so some defects are similar to the fabric texture, and it is easy to miss detection or misjudgment; 2. Due to the structural problem of the large circular machine, the clarity and gray scale of the collected images are inconsistent, which affects the defect detection accuracy; 3. Due to the problems of raw materials and environment, some flying objects are inevitably generated in the fabric production process, and the flying objects are similar to the imaging of broken warp, broken weft, holes and scratches, and are easy to misjudge; 4. The size and shape of the fabric defects are obviously different, and the detection model needs to have good detection effect on multi-scale targets. SUMMARY

[0004] Based on the above background, the purpose of the application is to provide a fabric defect detection method based on an improved YOLOv7-Tiny network model. By introducing a SPD-Conv module and a hybrid attention mechanism, a new fabric defect detection model is constructed, the feature extraction capability of the model is enhanced, the fabric defect detection accuracy is improved, and the calculation amount is greatly reduced.

[0005] To achieve the above-mentioned object of the application, the application provides the following technical solutions:

[0006] A fabric defect detection method based on an improved YOLOv7-Tiny network model, the method comprising the following steps:

[0007] Classifying fabric defects into hole defects, long strip defects and short strip defects, collecting a fabric defect dataset according to the fabric defect classification, and dividing the fabric defect dataset into a training set, a validation set and a test set;

[0008] Improving a YOLOv7-Tiny network model having an input unit, a backbone network unit and a head network unit to construct a fabric defect detection model;

[0009] Model training the fabric defect detection model through the training set and the validation set, and finally testing the trained fabric defect detection model through the test set;

[0010] Deploying the fabric defect detection model that passes the final test to detect defects of a target fabric;

[0011] The improvement of the YOLOv7-Tiny network model having the input unit, the backbone network unit and the head network unit to construct the fabric defect detection model comprises:

[0012] Introducing an SPD-Conv module between the second standard convolution module and the first multi-branch module in the backbone network unit of the YOLOv7-Tiny network model, the SPD-Conv module comprising a space-to-depth layer and a non-stride convolution layer;

[0013] Modifying the first two standard convolution modules in the backbone network unit of the YOLOv7-Tiny network model into convolution modules with a hybrid attention mechanism, the convolution modules with the hybrid attention mechanism comprising a hybrid attention module, a normalization module and a Leakyrelu activation function;

[0014] Modifying the last standard convolution module in each multi-branch module in the backbone network unit of the YOLOv7-Tiny network model into a convolution module with a hybrid attention mechanism.

[0015] The SPD-Conv module extracts deep features by downsampling the image once, thereby enhancing the feature extraction capability for small targets or low-resolution images; the hybrid attention module introduces a hybrid attention mechanism into the model, and a new feature map is obtained by superimposing the output features of the spatial attention and the output features of the channel module, wherein the channel attention has rich channel information, thereby improving the feature extraction of the model, and the spatial attention can make the position information of the defects more accurate, thereby effectively improving the feature extraction capability of the model; the combination of the SPD-Conv module and the hybrid attention mechanism enhances the feature extraction capability of the model, improves the fabric defect detection precision, and greatly reduces the computational amount.

[0016] As preferred, the classifying the fabric defects into hole defects, long strip defects and short strip defects comprises:

[0017] The fabric defect types are summarized as broken weft, loose weft, loose warp, broken warp, missing needle, oil stain, hole and warp hole;

[0018] The hole and the warp hole are classified as hole defects, the broken weft, loose weft, loose warp and broken warp with a length greater than 3 cm are classified as long strip defects, and the missing needle and oil stain with a length not greater than 3 cm are classified as short strip defects.

[0019] As preferred, the improving the YOLOv7-Tiny network model having an input unit, a backbone network unit and a head network unit to build a fabric defect detection model further comprises:

[0020] The last standard convolution module in each multi-branch module in the head network unit of the YOLOv7-Tiny network model is modified into a convolution module with a hybrid attention mechanism.

[0021] As preferred, the hybrid attention module comprises an SE channel attention module and a SAM spatial attention module, the output features of the SE channel attention module are taken as the input features of the SAM spatial attention module, and the output features of the SAM spatial attention module are superimposed with the output features of the SE channel attention module, and the superimposition result is taken as the output features of the hybrid attention module.

[0022] As preferred, the output features of the SE channel attention module are obtained by the following method: the input features sequentially undergo convolution operation, spatial compression, channel feature learning and scale operation to obtain the output features; wherein the spatial compression is used to realize global average pooling in the spatial dimension, convert the features of each channel into global features with a global receptive field, and obtain a vector aggregating global information in the channel dimension; the scale operation is used to multiply each channel weight value obtained by channel feature learning calculation with the two-dimensional matrix corresponding to the input feature channel to obtain the output features.

[0023] Preferably, the output features of the SAM spatial attention module are obtained by the following method: the input features are subjected to average pooling and maximum pooling in the channel dimension, two intermediate features are obtained, the two intermediate features are subjected to channel concatenation, the feature obtained by the concatenation is subjected to channel compression by a convolution layer with a convolution kernel size of 7x7, and finally the output features are obtained by calculation through an activation function.

[0024] Preferably, the backbone network unit of the fabric defect detection model comprises a first convolution module, a second convolution module, an SPD-Conv module, a first multi-branch module, a first maximum pooling layer, a second multi-branch module, a second maximum pooling layer, a third multi-branch module, a third maximum pooling layer, a fourth multi-branch module and an SPPCSP module arranged in sequence, wherein the first convolution module and the second convolution module are both convolution modules with a hybrid attention mechanism, and the last standard convolution module in the first multi-branch module, the second multi-branch module, the third multi-branch module and the fourth multi-branch module are all convolution modules with a hybrid attention mechanism.

[0025] Preferably, the model training parameters of the fabric defect detection model are as follows: the number of single batch training samples is 16, the dynamic parameter is 0.937, the learning rate is 0.01, the cosine annealing learning rate is 0.01, the input image size is 448x448 pixels, and the number of training rounds is 300.

[0026] Preferably, in the model training of the fabric defect detection model, data augmentation is performed, and the method of the data augmentation includes a Mixup method, a Cutmix method and a Mosaic method.

[0027] Preferably, the mathematical expression of the loss function used in the model training of the fabric defect detection model is as follows:

[0028] loss=w cls L cls +w box L box +w obj L obj

[0029] In the formula, L cls is a classification loss function, L box is a positioning loss function, and L obj is a confidence loss function, w cls , w box and w obj are weight coefficients corresponding to the above functions respectively, the classification loss function and the confidence loss function both adopt a binary cross-entropy loss function, and the positioning loss function adopts a CIoU loss function.

[0030] Compared with the prior art, the present application has the following advantages:

[0031] The fabric defect detection method based on the improved YOLOv7-Tiny network model can reduce the loss of fine-grained information, improve the learning ability of low-resolution image and small target feature representation, and thus improve the feature extraction capability and learning efficiency of the constructed fabric defect detection model.

[0032] The SE channel attention module and the SAM spatial attention module are combined to construct a hybrid attention module, which is introduced into the convolution module to obtain a convolution module with a hybrid attention mechanism.

[0033] The present application is based on the YOLOv7-Tiny network model, combined with the SPD-Conv module and the hybrid attention module, and constructs a fabric defect detection model suitable for fabric defect detection. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiment or prior art description will be briefly introduced below.

[0035] Figure 1 is a flowchart of the fabric defect detection method based on the improved YOLOv7-Tiny network model of the present application;

[0036] Figure 2 is a structural diagram of the fabric defect detection model AYOLOv7-Tiny network model in the present application;

[0037] Figure 3 is a structural diagram of the head network unit of the fabric defect detection model AYOLOv7-Tiny network model in the present application;

[0038] Figure 4 is a structural diagram of the SPD-Conv module in the present application;

[0039] Figure 5 is a structural diagram of the SE channel attention module in the present application;

[0040] Figure 6 Figure 1 is a structural schematic diagram of the SAM spatial attention module in the present application;

[0041] Figure 7 Figure 2 is a structural schematic diagram of the mixed attention module in the present application;

[0042] Figure 8 Figure 3 is a fabric defect type diagram of the production of the circular knitting machine in the present application;

[0043] Figure 9 Figure 4 is a fabric defect classification diagram of the production of the circular knitting machine in the present application;

[0044] Figure 10 Figure 5 is an accuracy curve diagram of the AYOLOv7-tiny network model in the present application;

[0045] Figure 11 Figure 6 is a loss curve diagram of the AYOLOv7-tiny network model in the present application;

[0046] Figure 12 Figure 7 is a defect test result diagram of each network model;

[0047] Figure 13 Figure 8 is a detection result diagram of each network model image clear or defect obvious;

[0048] Figure 14 Figure 9 is a detection result diagram of each network model image blurred or defect not obvious;

[0049] Figure 15 Figure 10 is a detection result diagram of each network model on flying object image;

[0050] Figure 16 Figure 11 is a detection result diagram of each network model on glass fiber fabric. DETAILED DESCRIPTION

[0051] The technical solutions of the present application will be further described in detail below by specific embodiments, and in conjunction with the drawings. It should be understood that the implementation of the present application is not limited to the following examples, and any form of variation and / or change of the present application will fall within the scope of protection of the present application.

[0052] In the present application, unless specified, all parts, percentages are weight units, and the equipment and raw materials used can be purchased from the market or commonly used in the art. The methods in the following examples are conventional methods in the art, unless otherwise specified. The components or equipment in the following examples are general standard components or components known to those skilled in the art, and their structure and principles can be known to those skilled in the art through technical manuals or through conventional experimental methods.

[0053] Embodiments of the present application are described in detail below with reference to the attached drawing figures, wherein the implementations can be presented in the order of explanation and not necessarily of priority. Embodiments of the present application can be implemented in any conventional hardware, software, firmware, or virtual reality implementation or combinations thereof.

[0054] As shown in Figure 1 Embodiments of the present application disclose a fabric defect detection method based on an improved YOLOv7-Tiny network model, which comprises the following steps:

[0055] S1. classifying fabric defects into hole defects, long strip defects and short strip defects, collecting a fabric defect dataset according to the fabric defect classification, and dividing the fabric defect dataset into a training set, a validation set and a test set;

[0056] S2. improving a YOLOv7-Tiny network model having an input unit, a backbone network unit and a head network unit to construct a fabric defect detection model;

[0057] S3. performing model training on the fabric defect detection model through the training set and the validation set, and performing final testing on the trained fabric defect detection model through the test set;

[0058] S4. deploying the fabric defect detection model that passes the final testing to detect defects of a target fabric.

[0059] Among them, improving the YOLOv7-Tiny network model having the input unit, the backbone network unit and the head network unit to construct the fabric defect detection model comprises:

[0060] Introducing a SPD-Conv module between the second standard convolution module and the first multi-branch module in the backbone network unit of the YOLOv7-Tiny network model, the SPD-Conv module comprising a space-to-depth layer and a non-stride convolution layer;

[0061] Modifying the first two standard convolution modules in the backbone network unit of the YOLOv7-Tiny network model into convolution modules with a hybrid attention mechanism, the convolution modules with the hybrid attention mechanism comprising a hybrid attention module, a normalization module and a Leakyrelu activation function;

[0062] Modifying the last standard convolution module in each multi-branch module in the backbone network unit of the YOLOv7-Tiny network model into a convolution module with a hybrid attention mechanism;

[0063] The last standard convolution module in each multi-branch module in the head network unit of the YOLOv7-Tiny network model is modified into a convolution module with a hybrid attention mechanism.

[0064] YOLOv7-Tiny is a lightweight model of YOLOv7. Compared with the multi-branch stacking module (ELAN), the pooling module (MP), and the SPPCSPC module of YOLOv7, the multi-branch module (C5), the pooling module (MP), and the SPPCSP module adopted by YOLOv7-Tiny are relatively lightweight. Although the accuracy decreases slightly, the computational amount is greatly reduced and the detection speed is improved, which is more convenient for industrial deployment. YOLOv7-Tiny uses a convolution kernel with a size of 3x3 and a step of 2 in the network, and multiple pooling operations are performed on the picture to downsample the picture, reduce the dimension of the feature, and obtain the location information of the feature, thereby reducing the parameter amount and simplifying the complexity of network calculation. However, this may cause the loss of texture features and target class information, and therefore, for small targets and low-resolution images, the detection effect of YOLOv7-tiny is not ideal, and the target features cannot be effectively extracted and fused. In addition, since the types of fabric defects are diverse, the detection model needs to combine deep semantic information and shallow positioning information.

[0065] Based on the above analysis, in order to realize high-precision and high-efficiency detection of fabric defects, the YOLOv7-Tiny network model is improved in the embodiment. In order to facilitate distinction, the fabric defect detection model constructed by the method improved in the embodiment is referred to as AYOLOv7-Tiny network model. Similar to the YOLOv7-Tiny network model, the AYOLOv7-Tiny network is composed of an input unit, a backbone network unit, and a head network unit.

[0066] The function of the input unit is to transmit the pictures in the data set into the target detection network. Since the backbone network unit generates 6 layers of feature maps with different sizes after 6 times of downsampling, the resolution size of the input picture must be 6 times of 2. If the size of the input image does not meet the requirement, the network model will use the corresponding expansion method for processing. That is, the original image is embedded in a larger background, the aspect ratio is kept the same as the input image, and the black edge is filled. This expansion method does not lose the information of the image, and can also avoid cutting or stretching some objects in the input image, and increases the size of the feature extractor and the receptive field of the network to learn more spatial information and improve the performance of the model.

[0067] As Figure 2As shown, the backbone network unit includes, in sequence, a first convolution module, a second convolution module, an SPD-Conv module, a first multi-branch module, a first max-pooling layer, a second multi-branch module, a second max-pooling layer, a third multi-branch module, a third max-pooling layer, a fourth multi-branch module, and an SPPCSP module, wherein the first convolution module and the second convolution module are both convolution modules CBL-1 with a hybrid attention mechanism, the last standard convolution module in each of the first multi-branch module, the second multi-branch module, the third multi-branch module, and the fourth multi-branch module is a convolution module CBL-1 with a hybrid attention mechanism, each multi-branch module is denoted by C5-1, and each max-pooling layer is denoted by MP.

[0068] The convolution module with a hybrid attention mechanism includes a hybrid attention module SC, a normalization module BN, and a Leakyrelu activation function, the SC module uses a convolution kernel with a size of 3x3 and a step of 2, and performs down-sampling on the image together with the MP module to generate six layers of feature maps with different sizes of 224x224, 112x112, 56x56, 28x28, 14x14, and 7x7.

[0069] The SPD-Conv module extracts deep features by performing down-sampling on the image once, thereby enhancing the feature extraction capability for small targets or low-resolution images.

[0070] The C5-1 module is a high-efficiency network structure that controls the shortest and longest gradient paths, so that the network can learn more features and has stronger robustness. The C5-1 module has two branches, the first branch is a channel change through a 1x1 convolution, and the second branch first performs channel change through a 1x1 convolution module, then performs feature extraction through two 3x3 convolution modules, and then stacks the four features together to perform channel change again to obtain the final feature extraction result.

[0071] The SPPCSP module functions to obtain different receptive fields and increase the receptive field through max-pooling, so that the algorithm can adapt to different resolution images. In the first branch of the SPPCSP, there are four branches that have experienced the max-pooling layer MP, representing that it can process different objects and better distinguish small targets and large targets. The second branch performs a regular convolution operation, and finally combines the two parts together, reducing the computational amount by half, so that the speed is faster and the accuracy is higher.

[0072] The structure of the backbone network unit and the parameters of each module are shown in Table 1.

[0073] Table 1 Structure of the backbone network unit

[0074]

[0075]

[0076] The head network unit is composed of a neck network and a detection head. The PAFPN structure is used in the head network, which is composed of an FPN feature pyramid and a PAN pixel aggregation network, as shown in the structure. Figure 3 The FPN is a reinforced feature extraction network that conveys strong semantic features from top to bottom. High-level features are fused with low-level features through up-sampling to obtain a feature map for prediction. The three effective feature layers (28x28, 14x14, 7x7) obtained in the backbone network will be fused in this part. The purpose of feature fusion is to combine feature information of different scales so that the effective feature layers obtained can be used for further feature extraction. FPN transmits deep semantic features to shallow layers, enhancing semantic information at multiple scales but not positioning information. The PAN structure is a bottom-up feature pyramid that re-samples features to transmit positioning information from shallow layers to deep layers, enhancing positioning capabilities at multiple scales. Through the PAFPN double-pyramid structure, feature maps of different scales contain semantic information and position information of the image, ensuring accurate prediction of pictures of different sizes.

[0077] The SPD-Conv module and the hybrid attention module SC will be described in detail below.

[0078] The SPD-Conv module includes a spatial-to-depth layer SPD and a non-stride convolution layer Conv. Image transformation technology is used to down-sample feature maps inside and throughout the network model, as shown in the following formula: Figure 4 For example, after SPD feature transformation, the feature Figure X (S,S,C1) is transformed into the intermediate feature Figure X '(S / scale,S / scale,scale 2 C1). Then, the non-stride convolution layer Conv with C2 filters (C2 2 C1) further transforms the intermediate feature Figure X '(S / scale,S / scale,scale 2 C1) into X”(S / scale,S / scale,C2). Figure X The hybrid attention module SC includes an SE channel attention module and a SAM spatial attention module.

[0079]

[0080] ​The output feature of the SE channel attention module is obtained by the following method: the input feature sequentially passes through a convolution operation, spatial compression, channel feature learning, and a scale operation to obtain the output feature; wherein the spatial compression is used to realize global average pooling in the spatial dimension, convert the feature of each channel into a global feature with a global receptive field, and obtain a vector aggregating global information in the channel dimension; and the scale operation is used to multiply each channel weight value obtained by the channel feature learning calculation with a two-dimensional matrix corresponding to the channel of the input feature to obtain the output feature.

[0081] Specifically, as shown in Figure 5 , the SE channel attention module mainly includes four parts of a convolution F tr , compression F sq , excitation F ex , and a scale operation. The input X=W'xH'xC' passes through the convolution operation F tr , and the output UeWxHxC. F tr is V=[v1, v2,..., v c ] represents a set of learned convolution kernels, and the output U=[u1, u2,..., u c ]. F sq The operation is a spatial compression of the input feature map with a dimension of HxWxC, realizing global average pooling in the spatial dimension, converting the feature of each channel into a global feature with a global receptive field, obtaining a 1x1xC vector, and aggregating global information in the channel dimension. F ex The operation is channel feature learning of the compressed vector, obtaining a feature map with channel attention through two fully connected layers (FC), which also has a dimension of 1x1xC, and then normalizing through a Sigmoid function to obtain channel weights. The scale operation multiplies each channel weight value calculated by the SE module with a two-dimensional matrix corresponding to the channel of the original feature map to obtain the input feature map required by the SAM spatial attention module.

[0082] The output feature of the SAM spatial attention module is obtained by the following method: the input feature passes through an average pooling and a maximum pooling in the channel dimension to obtain two intermediate features, the two intermediate features are channel spliced, a convolution layer with a convolution kernel size of 7x7 is used to compress the channel of the spliced feature, and finally the output feature is obtained by calculating through an activation function.

[0083] Specifically, as shown in Figure 6As shown, the feature map with input dimension of HxWxC is subjected to global average pooling and global max pooling in the channel dimension, to obtain two feature maps with dimension of HxWx1, and the two feature maps are concatenated in the channel dimension to obtain a feature map with dimension of HxWx2; then, a convolution layer with a convolution kernel of 7x7 is used to compress the channel dimension to obtain a feature map with dimension of HxWx1, and an activation function (Sigmoid) is used to output the final feature map.

[0084] The structure of the hybrid attention module SC is as shown in Figure 7 The output feature of the SE channel attention module is taken as the input feature of the SAM spatial attention module, and the output feature of the SAM spatial attention module is superimposed with the output feature of the SE channel attention module, and the superimposed result is taken as the output feature of the hybrid attention module. The hybrid attention module SC not only increases the weight of important features and enhances the features, effectively improving the feature extraction capability of the network, but also improves the image segmentation result, making the position information of the defects more accurate.

[0085] The above is a detailed description of step S2. The following is a detailed description of steps S1, S3 and S4.

[0086] Taking a circular knitting machine as an example, the fabric produced by the circular knitting machine has various defects, including Broken pick, Slack pick, Slack end, Broken end, Drop stitches, Oilstain, Hole, and Smash. Figure 8 The main defect types are classified. According to the defect conditions and considering the accuracy and efficiency of the fabric defect detection model, the defects are classified into three categories: Hole, L_line and S_line, as shown in Figure 9 The Hole and Smash are classified as Hole defects, as shown in Figure 9 (a). The defects with size exceeding 3 cm in Broken pick, Slack pick, Slack end and Broken end are classified as L_line defects, as shown in Figure 9 (b). The defects with size not exceeding 3 cm in Drop stitches and Oilstain are classified as S_line defects, as shown in Figure 9 (c).

[0087] The fabric defect dataset of the embodiment is from a real industrial field of a circular loom fabric. The fabric original image with a resolution of 2744x500 is collected by a 2k area array camera. Due to the high yield of the product, the defect images confirmed by the quality inspection personnel are manually selected and cropped to obtain 2764 images with a resolution of 400x400, and then classified and labeled by skilled technicians. Due to the different difficulties of detecting different defects, the number of collected images is also different. The shape of the hole defect is relatively regular and the characteristics are obvious, and 243 images are collected. The long strip defect and the short strip defect are relatively similar, and according to the obvious degree of the characteristics of the two defects, 1644 and 877 images are collected respectively. The data set of each defect is roughly divided into training set, validation set and test set according to 70%, 10% and 20%.

[0088] The hardware environment for model training is Linux operating system, Intel Xeon Platinum 8255C CPU, 40GRAM, RTX 3080 GPU, and the software version is PyTorch 1.8.1+Python 3.8(ubuntu18.04)+Cuda 11.1.

[0089] The parameters of model training are shown in Table 2.

[0090] Table 2 Model training parameters

[0091] Training parameters Value Batch size 16 Dynamic parameters 0.937 Learning rate 0.01 Cosine annealing learning rate 0.01 Data augmentation 1.0 Input image size 448×448 Epochs 300

[0092] In order to increase the amount of data and improve the robustness of the network during model training, data enhancement technology is adopted. Specifically, three methods are applied: Mixup, Cutmix and Mosaic. In the Mixup method, two images are randomly mixed according to a certain proportion, and the classification result after mixing is distributed to the original image according to the proportion. In the Cutmix method, an image is divided into two parts, and random interpolation is performed using non-original position pixel values to generate a new image. In the Mosaic method, four images are randomly selected and randomly rotated and flipped to form a large image for training. At the same time, the proportion of small targets in the training set is increased by random scaling. These methods can enrich the training data, thereby improving the accuracy and generalization ability of the model.

[0093] The mathematical expression of the loss function used in model training is:

[0094] loss=w cls L cls +w box L box +w obj L obj

[0095] wherein L cls is a classification loss function, L box is a positioning loss function, L obj is a confidence loss function, w cls , w box , w obj are weight coefficients corresponding to each function respectively, the classification loss function and the confidence loss function both adopt a binary cross-entropy loss function, and the positioning loss function adopts a CIoU loss function.

[0096] To test the performance of the AYOLOv7-tiny network model, the embodiment selects performance evaluation indexes such as mean accuracy (mAP), precision (AP), floating-point operation (GFlops) and FPS. AP is the area between the P-R curve and the coordinate axis, used to evaluate the prediction accuracy of each defect, mAP is the average value of AP in each category, and FPS is used to evaluate the detection speed of the model. The greater the FPS, the more images the network processes per second.

[0097] To verify the effectiveness of the AYOLOv7-tiny network model and analyze the influence of the SPD-Conv module and the SC module on the performance of the network model, the embodiment performs an ablation experiment. The experiment sets up 7 different combination modes, and the detection results of each combination on the fabric defect dataset are shown in Table 3.

[0098] Table 3 Ablation experiment results

[0099]

[0100] As can be seen from Table 3, the detection accuracy mAP of the YOLOv7-tiny network is 97.3%, and the detection speed FPS is 333. The detection accuracy mAP of the network with the addition of the SE module and the SAM module is 97.8% and 97.6%, respectively; when the CBAM attention mechanism is added, the mAP increases to 97.9%, but the detection speed FPS reaches the lowest 250; compared with the above, the mAP increases by 1.1% after adding the SC module, reaching 98.4%, and the detection speed FPS can reach 286, indicating that the SC module is superior to the CBAM module in feature fusion and feature representation. In the case of using only the SPD-Conv module, the mAP increases by 0.9%, and the detection speed also increases slightly. In terms of model complexity, compared with YOLOv7-tiny, the calculation amount GFlops increases by 0.5 and 0.3 after adding the CBAM and SC modules, respectively, and the calculation amount decreases by 8.7 after adding the SPD-Conv module. In the case of adding the SPD-Conv module and the SC module at the same time, the mAP value increases by 1.4%, reaching 98.7%; and the calculation amount decreases significantly from 13.0 to 4.4. At the same time, the detection speed does not change. This shows that the SPD-Conv module and the SC module are effective, and the combination of the two improves the feature extraction of the backbone network, retains more semantic information in the feature fusion stage, and reduces the damage of information in the transmission process; moreover, without reducing the detection speed, the accuracy mAP is effectively improved and the calculation amount GFlops is reduced, greatly improving the fabric defect detection capability.

[0101] In order to evaluate the accuracy and real-time performance of the AYOLOv7-tiny network model, the training loss, validation loss and mAP during training are shown in Figure 10 and Figure 11 . Figure 10 (a) represents the mAP curve, Figure 10 (b) represents the precision curve, Figure 10 (c) represents the recall curve. Figure 11 (a) represents the Position loss, Figure 11 (b) represents the Confidence loss. Figure 11 (c) represents the Classification loss. It can be seen that the training and validation loss function curves converge rapidly within the first 50 epochs, and completely converge when the epoch reaches 300, while the mAP curve, the precision curve and the recall curve also show an increasing trend with the increase of the number of training cycles.

[0102] The fabric defect data set of the present embodiment is substituted into different network models, and the results are shown in Table 4.

[0103] Table 4 Fabric defect dataset detection results on different network models

[0104]

[0105] It can be seen that the performance of the AYOLOv7-tiny network model of the embodiment is better than that of other target detection networks. In terms of detection accuracy, the mAP value of the AYOLOv7-tiny defect detection network is the highest (98.7%), which is improved by 0.5%, 1.9%, 0.9%, and 1.4% compared with YOLOv5, YOLOv6, YOLOv7, and YOLOv7-tiny, respectively; in terms of detection speed FPS, the detection speed of YOLOv5 is the fastest, reaching 476; in terms of calculation amount GFlops, the calculation amount of AYOLOv7-tiny is the least, only 4.4. Compared with YOLOv5 and AYOLOv7-tiny networks, AYOLOv7-tiny has more advantages in detection accuracy mAP, calculation amount, and parameter amount, and the detection speed also meets the real-time requirements.

[0106] In the embodiment, four windows are randomly selected to test YOLOv5, YOLOv6, YOLOv7, YOLOv7-tiny, and AYOLOv7-tiny networks, and the test results are as shown in Figure 12 Figure 12 (a) represents a hole defect, Figure 12 (b) represents a long strip defect, Figure 12 (c) represents a short strip defect. It can be seen that AYOLOv7-tiny has better detection performance.

[0107] In the embodiment, SPD-Conv modules are added to the model to improve the detection difficulties of low-resolution images, gray inconsistency, and small defects. In order to effectively verify the effectiveness of the SPD-Conv module, YOLOv7-tiny and YOLOv7-tiny (SPD-YOLOv7-tiny) with only SPD-Conv modules are selected to perform experiments on images with clear images or obvious defects, and images with blurred images or non-obvious defects. The detection effect of the image with clear image or obvious defect is as shown in Figure 13 , and the image with blurred image or non-obvious defect is as shown in Figure 14 . It can be seen that when the image is clear or the defect is obvious, the detection results of YOLOv7-tiny and SPD-YOLOv7-tiny are almost the same. When the image is blurred or the defect is not obvious, the detection effect of SPD-YOLOv7-tiny is obviously improved.

[0108] ​The imaging of flying objects in the fabric production process is similar to fabric defects. In order to test the misjudgment of the network to the flying objects, YOLOv5, YOLOv7 and AYOLOv7-tiny with higher mAP values in this embodiment are used for comparative experiments, and their test results are as shown in Figure 15

[0109] Flying objects are caused by the environment of the factory, wherein Figure 15 (a), Figure 15 (b), Figure 15 (c) are the images of different flying objects falling on the fabric. For Figure 15 (a), YOLOv5 misjudges it as a long strip defect, YOLOv7 misjudges it as a short strip defect, and AYOLOv7-tiny correctly identifies it; for Figure 15 (b), YOLOv5 misjudges it as a long strip defect and a hole defect, YOLOv7 misjudges it as a long strip defect, and AYOLOv7-tiny correctly identifies it; for Figure 15 (c), YOLOv5, YOLOv7 and AYOLOv7-tiny all misjudge it as a long strip defect. Although AYOLOv7-tiny still has a little misjudgment, compared with other models, it has better performance in detecting flying objects.

[0110] In order to further verify the effectiveness of the AYOLOv7-tiny network model in identifying other types of fabric defects, the data set of glass fabric defects is collected on a plain knitting machine. The types of defects in the data set include: the data set is divided into 9 categories: wave selvedge, coarse pick, slack end, sandmark, torn selvedge, slub, double pick, broken end and harness skip. At the same time, comparative experiments are carried out, and the experimental method is the same as that of the fabric defect data set of the circular knitting machine. The experimental results are shown in Table 5.

[0111] Table 5 Comparative experiment of glass fabric defect data set

[0112]

[0113]

[0114] It can be seen that the mAP value of AYOLOv7-tiny reaches the highest 97.5%, the detection speed FPS can also reach 357, and the AYOLOv7-tiny network model has low complexity and small calculation amount, and is more easily deployed on the embedded platform in the industrial field.​

[0115] The embodiment randomly takes 6 pictures from different types of defects to test YOLOv5, YOLOv7, YOLOv7-tiny and AYOLOv7-tiny network models respectively, and the test results of different network models are as shown in Figure 16 It can be seen that AYOLOv7-tiny has better detection performance compared with other models. It can be seen that AYOLOv7-tiny also has good effect for other types of textile fabric defect detection.

[0116] The above experimental results show that the AYOLOv7-tiny network model of the embodiment has the best performance, and the detection accuracy on the fabric defect data set can reach 98.7%, and the calculation amount is reduced by 66%.

[0117] The principles and implementation modes of the present application are described by applying specific examples in this paper, and the above embodiment is only used to help understand the method and core idea of the present application. It should be pointed out that for ordinary skilled persons in the technical field, some improvements and modifications can be made to the present application without departing from the principles of the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A fabric defect detection method based on an improved YOLOv7-Tiny network model, characterized in that: The method comprises the following steps: classifying fabric defects into hole defects, long strip defects and short strip defects, collecting a fabric defect dataset according to the fabric defect classification, and dividing the fabric defect dataset into a training set, a validation set and a test set; improving a YOLOv7-Tiny network model having an input unit, a backbone network unit and a head network unit to build a fabric defect detection model; training the fabric defect detection model through the training set and the validation set, and finally testing the trained fabric defect detection model through the test set; deploying the fabric defect detection model that passes the final test to detect defects in a target fabric; wherein the improvement of the YOLOv7-Tiny network model having the input unit, the backbone network unit and the head network unit to build the fabric defect detection model comprises: introducing an SPD-Conv module between the second standard convolution module and the first multi-branch module in the backbone network unit of the YOLOv7-Tiny network model, the SPD-Conv module comprising a space-to-depth layer and a non-stride convolution layer; modifying the first two standard convolution modules in the backbone network unit of the YOLOv7-Tiny network model into convolution modules with a hybrid attention mechanism, the convolution module with the hybrid attention mechanism comprising a hybrid attention module, a normalization module and a Leakyrelu activation function; modifying the last standard convolution module in each multi-branch module in the backbone network unit of the YOLOv7-Tiny network model into a convolution module with a hybrid attention mechanism.

2. The fabric defect detection method based on the improved YOLOv7-Tiny network model according to claim 1, characterized in that: The classification of fabric defects into hole defects, long strip defects and short strip defects comprises: summarizing fabric defect categories as weft breakage, loose weft, loose warp, warp breakage, missed needle, oil stain, hole and warp hole; classifying holes and warp holes as hole defects, classifying weft breakage, loose weft, loose warp and warp breakage with a length greater than 3 cm as long strip defects, and classifying missed needle and oil stain with a length not greater than 3 cm as short strip defects.

3. The fabric defect detection method based on the improved YOLOv7-Tiny network model according to claim 1, characterized in that: The improvement of the YOLOv7-Tiny network model having the input unit, the backbone network unit and the head network unit to build the fabric defect detection model further comprises: modifying the last standard convolution module in each multi-branch module in the head network unit of the YOLOv7-Tiny network model into a convolution module with a hybrid attention mechanism.

4. The fabric defect detection method based on the improved YOLOv7-Tiny network model according to claim 1, characterized in that: The hybrid attention module comprises an SE channel attention module and a SAM spatial attention module, the output features of the SE channel attention module are taken as the input features of the SAM spatial attention module, and the output features of the SAM spatial attention module are superimposed with the output features of the SE channel attention module, and the superimposed result is taken as the output features of the hybrid attention module.

5. The fabric defect detection method based on the improved YOLOv7-Tiny network model according to claim 4, characterized in that: The output feature of the SE channel attention module is obtained by the following method: the input feature sequentially passes through a convolution operation, spatial compression, channel feature learning, and a scale operation to obtain the output feature; wherein the spatial compression is used to realize global average pooling in the spatial dimension, convert the feature of each channel into a global feature with a global receptive field, and obtain a vector that aggregates global information in the channel dimension; the scale operation is used to multiply each channel weight value obtained by the channel feature learning calculation with a two-dimensional matrix corresponding to the input feature channel to obtain the output feature.

6. The fabric defect detection method based on the improved YOLOv7-Tiny network model according to claim 4, characterized in that: The output feature of the SAM spatial attention module is obtained by the following method: the input feature passes through an average pooling and a maximum pooling in the channel dimension to obtain two intermediate features, the two intermediate features are channel spliced, a channel compression is performed on the spliced feature by a convolution layer with a convolution kernel size of 7x7, and finally the output feature is obtained by an activation function calculation.

7. The fabric defect detection method based on the improved YOLOv7-Tiny network model according to claim 1, characterized in that: The backbone network unit of the fabric defect detection model comprises a first convolution module, a second convolution module, an SPD-Conv module, a first multi-branch module, a first maximum pooling layer, a second multi-branch module, a second maximum pooling layer, a third multi-branch module, a third maximum pooling layer, a fourth multi-branch module, and an SPPCSP module arranged in sequence, wherein the first convolution module and the second convolution module are both convolution modules with a hybrid attention mechanism, and the last standard convolution module in the first multi-branch module, the second multi-branch module, the third multi-branch module, and the fourth multi-branch module are all convolution modules with a hybrid attention mechanism.

8. The fabric defect detection method based on the improved YOLOv7-Tiny network model according to claim 1, characterized in that: The model training parameters of the fabric defect detection model are that the number of single batch training samples is 16, the dynamic parameter is 0.937, the learning rate is 0.01, the cosine annealing learning rate is 0.01, the input image size is 448x448 pixels, and the number of training rounds is 300.

9. The fabric defect detection method based on the improved YOLOv7-Tiny network model according to claim 1, characterized in that: In the model training of the fabric defect detection model, data augmentation is performed, and the method of the data augmentation includes a Mixup method, a Cutmix method, and a Mosaic method.

10. The fabric defect detection method based on the improved YOLOv7-Tiny network model according to claim 1, characterized in that: The mathematical expression of the loss function used in the model training of the fabric defect detection model is loss = w cls L cls +w box L box +w obj L obj In the formula, L cls is a classification loss function, L box is a positioning loss function, L obj is a confidence loss function, w cls , w box , w obj are weight coefficients corresponding to each function respectively, the classification loss function and the confidence loss function both adopt a binary cross-entropy loss function, and the positioning loss function adopts a CIoU loss function.

Images