Fabric defect detection method, device, terminal equipment and medium

Abstract

The present application is applicable to the field of target detection technology and provides a fabric defect detection method, apparatus, terminal device, and medium. The method comprises: obtaining an image of the fabric; annotating each fabric image with the location information and category information of the defect; inputting the fabric image into a pre-built LW-SSD model for training to obtain a trained LW-SSD model; the model comprises a basic model framework, a dual-channel attention mechanism module, a multi-branch parallel dilated convolution module, and a feature fusion module, which are respectively used to extract image features, expand the receptive field of the feature layer, capture multi-scale contextual information, and update the model weights and unify the resolution of the feature layer; inputting the image of the fabric to be detected into the model to obtain the location information and category information of the defect in the image of the fabric to be detected. The present application can reduce the number of model parameters, increase the detection speed, and improve the accuracy of the model in fabric defect detection.

Description

Technical Field

[0001] The present application belongs to the field of target detection technology, and in particular relates to a fabric defect detection method, apparatus, terminal equipment and medium. Background Art

[0002] In general fabric quality inspection, such as towel testing, manual visual inspection is the primary method used. While this method can identify defects in towel fabrics, it is easily affected by the user's emotions and the external environment. Furthermore, manual visual inspection is slow for large deformations and small defects, and has a high rate of false positives and missed detections. Currently, the accuracy of manual visual inspection is less than 70%.

[0003] Currently, using highly efficient and accurate deep learning methods to replace manual visual inspection for automated fabric defect detection has become a new research direction. With the development of convolutional neural network algorithms, deep learning-based object detection methods have been gradually applied to various real-life scenarios. These methods are mainly divided into two categories: single-stage object detection and two-stage object detection. Single-stage object detection algorithms include the Yolo (You only look once) series, SSD (Single Shot MultiBox Detector), and RetinaNet object detection. These methods directly implement object detection through regression, which is fast but has low accuracy for small objects. Two-stage object detection algorithms include R-CNN (Region-based Convolution Neural Networks), Fast-RCNN (Fast Region-based Convolutional Neural Network), and Faster-RCNN (Fast Region-based Convolutional Network). These methods first generate a priori bounding boxes and then perform classification and regression on candidate regions. This results in slower detection speed but high accuracy.

[0004] Considering the requirements for small target defects in industrial production in terms of detection accuracy, detection speed and the size of overall model parameters, the existing deep learning methods cannot meet these three needs well. Summary of the Invention

[0005] To overcome the problems existing in the related art, the embodiments of the present application provide a fabric defect detection method, apparatus, terminal device and medium, which can reduce the number of model parameters, increase the detection speed and improve the accuracy of the model in fabric defect detection.

[0006] This application is achieved through the following technical solutions:

[0007] In a first aspect, embodiments of the present application provide a method for detecting fabric defects, comprising:

[0008] Acquire images of fabrics; each fabric image is annotated with location information and category information of defects;

[0009] The fabric image is input into a pre-built LW-SSD model for training to obtain a trained LW-SSD model. The LW-SSD model includes a basic model framework, a raised efficient channel attention mechanism module (r-ECA) for deep convolutional neural networks, a multi-branch parallel dilated convolution module (MPD), and a feature fusion module (FFM). The basic model framework is used to extract features of the fabric image; the multi-branch parallel dilated convolution module is used to expand the receptive field of the feature layer in the basic model framework; the dual channel attention mechanism module is used to capture multi-scale contextual information and update the weights of the feature layer in the basic model framework; and the feature fusion module is used to unify the resolution of the feature layer in the basic model framework.

[0010] The image of the fabric to be detected is input into the trained LW-SSD model to obtain the location information and category information of the defects in the image of the fabric to be detected.

[0011] In one possible implementation, the basic model framework includes 7 feature layers, including Conv3_3 feature layer, Conv4_3 feature layer, Fc7 feature layer, Conv6_2 feature layer, Conv7_2 feature layer, Conv8_2 feature layer and Conv9_2 feature layer;

[0012] Among them, the Conv3_3 feature layer, Conv4_3 feature layer, Fc7 feature layer and Conv6_2 feature layer are the feature layers of the lightweight network; the Conv7_2 feature layer, Conv8_2 feature layer and Conv9_2 feature layer are the feature layers of the SSD algorithm.

[0013] In one possible implementation, the output ends of the Conv4_3 feature layer, the Fc7 feature layer, the Conv6_2 feature layer, the Conv7_2 feature layer, the Conv8_2 feature layer and the Conv9_2 feature layer are all provided with a dual-channel attention mechanism module, and the dual-channel attention mechanism module is also used to update the weights of the Conv4_3 feature layer, the Fc7 feature layer, the Conv6_2 feature layer, the Conv7_2 feature layer, the Conv8_2 feature layer and the Conv9_2 feature layer.

[0014] In a possible implementation, the multi-branch parallel void convolution module includes a first multi-branch parallel void convolution module, a second multi-branch parallel void convolution module, and a third multi-branch parallel void convolution module; the dual-channel attention mechanism module includes a first dual-channel attention mechanism module, a second dual-channel attention mechanism module, a third dual-channel attention mechanism module, a fourth dual-channel attention mechanism module, a fifth dual-channel attention mechanism module, and a sixth dual-channel attention mechanism module;

[0015] The fabric image is input to the Conv3_3 feature layer, and the Conv3_3 feature layer outputs the first feature map;

[0016] The first feature map is input to the Conv4_3 feature layer and the feature fusion module, and the Conv4_3 feature layer outputs the second feature map;

[0017] The second feature map is input to the Fc7 feature layer and the feature fusion module, and the Fc7 feature layer outputs the third feature map; the third feature map is processed in sequence by the second multi-branch parallel void convolution module and the second dual-channel attention mechanism module to output the second target feature map;

[0018] The second target feature map is input into the feature fusion module, which is used to fuse the first feature map, the second feature map and the second target feature map to output a fused feature map; the fused feature map is sequentially processed by the first multi-branch parallel void convolution module and the first dual-channel attention mechanism module to output the first target feature map;

[0019] The second target feature map is also input to the Conv6_2 feature layer, and the Conv6_2 feature layer outputs the fourth feature map; the fourth feature map is sequentially processed by the third multi-branch parallel void convolution module and the third dual-channel attention mechanism module to output the third target feature map;

[0020] The third target feature map is input to the Conv7_2 feature layer, and the Conv7_2 feature layer outputs the fifth feature map; the fifth feature map is processed by the fourth dual-channel attention mechanism module and outputs the fourth target feature map;

[0021] The fourth target feature map is input to the Conv8_2 feature layer, and the Conv8_2 feature layer outputs the sixth feature map; the sixth feature map is processed by the fifth dual-channel attention mechanism module and outputs the fifth target feature map;

[0022] The fifth target feature map is input to the Conv9_2 feature layer, and the Conv9_2 feature layer outputs the seventh feature map; the seventh feature map is processed by the sixth dual-channel attention mechanism module to output the sixth target feature map; the target feature maps include the first target feature map, the second target feature map, the third target feature map, the fourth target feature map, the fifth target feature map and the sixth target feature map; the target feature map is marked with a priori boxes.

[0023] In one possible implementation, the LW-SSD model also includes a screening module; the screening module is used to obtain a target feature map, and based on the non-maximum suppression principle, screen the prior boxes greater than the confidence threshold as prediction boxes, and output a prediction feature map containing the prediction boxes; the prediction boxes are marked with the location information and category information of the defects in the image of the fabric to be detected.

[0024] In a possible implementation, the fabric images include a training set and a validation set of set proportions;

[0025] Input the fabric image into the pre-built LW-SSD model for training to obtain the trained LW-SSD model, including:

[0026] Input the training set into the LW-SSD model for training and obtain the training set loss;

[0027] Input the validation set into the LW-SSD model for training and obtain the validation set loss;

[0028] According to the set number of iterations, the convergence of the training set loss and the validation set loss, the weight value of the LW-SSD model is updated to obtain the trained LW-SSD model.

[0029] In one possible implementation, each fabric image is annotated with a ground truth box to indicate the location and category of the defect;

[0030] The training set is input into the LW-SSD model for training, and the training set loss is obtained, including:

[0031] The training set is input into the LW-SSD model for training, and the training target feature map is output; the training target feature map is marked with a priori boxes;

[0032] During the training process, the training set loss is calculated based on the difference between the prior box and the true box on the training target feature map. The training set loss includes the classification loss and regression loss during the training process.

[0033] In a second aspect, an embodiment of the present application provides a fabric defect detection device, characterized in that it includes:

[0034] An image acquisition module is used to acquire images of fabrics; each image of the fabric is marked with the location information and category information of the defects;

[0035] The model training module is used to input the fabric image into the pre-built LW-SSD model for training to obtain the trained LW-SSD model. The LW-SSD model includes a basic model framework, a dual-channel attention mechanism module, a multi-branch parallel void convolution module, and a feature fusion module. The basic model framework is used to extract the features of the fabric image; the multi-branch parallel void convolution module is used to expand the receptive field of the feature layer in the basic model framework; the dual-channel attention mechanism module is used to capture multi-scale contextual information and update the weights of the feature layer in the basic model framework; and the feature fusion module is used to unify the resolution of the feature layer in the basic model framework.

[0036] The result output module is used to input the image of the fabric to be detected into the trained LW-SSD model to obtain the position information and category information of the defects in the image of the fabric to be detected.

[0037] In a third aspect, an embodiment of the present application provides a terminal device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and is characterized in that when the processor executes the computer program, the fabric defect detection method of the first aspect is implemented.

[0038] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and is characterized in that when the computer program is executed by a processor, the fabric defect detection method of the first aspect as described above is implemented.

[0039] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0040] In the embodiment of the present application, the original backbone network of the SSD algorithm is replaced with a lightweight network, which effectively reduces the number of model parameters; a multi-branch parallel void convolution module is added to the network to expand the receptive field of the shallow feature layer and improve the detection ability of small targets; a feature fusion module is used to enrich the semantics and detail information of the shallow feature map; a dual-channel attention mechanism module is added to the network to capture multi-scale contextual information and improve the model's detection ability for targets with large deformation scales.

[0041] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0043] Figure 1 1 is a flow chart of a fabric defect detection method provided in one embodiment of the present application;

[0044] Figure 2 This is a schematic diagram of a processing flow of a fabric defect detection method provided by an embodiment of the present application;

[0045] Figure 3 This is a schematic diagram of the overall network framework of the LW-SSD model provided in one embodiment of the present application;

[0046] Figure 4 1 is a schematic diagram of the structure of a multi-branch parallel dilated convolution module MPD provided in one embodiment of the present application;

[0047] Figure 5 Schematic diagram of the structure of the feature fusion module FFM provided in one embodiment of the present application;

[0048] Figure 6 2 is a schematic diagram of the structure of the dual-channel attention mechanism module r-ECA provided in one embodiment of the present application;

[0049] Figure 7 This is a schematic diagram of defect detection results of a towel provided in one embodiment of the present application;

[0050] Figure 8 This is a graph evaluating the detection performance of the LW-SSD model for a fabric image dataset provided by an embodiment of the present application;

[0051] Figure 9 This is a structural diagram of a fabric defect detection device provided in one embodiment of the present application;

[0052] Figure 10 It is a structural diagram of a terminal device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0053] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0054] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0055] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0056] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0057] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0058] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0059] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0060] Figure 1This is a schematic flow chart of a fabric defect detection method provided by an embodiment of the present application, with reference to Figure 1 and Figure 2 , the fabric defect detection method is described in detail as follows:

[0061] Step 101: Acquire images of fabrics; each fabric image is marked with location information and category information of defects.

[0062] For example, pictures of industrially similar fabrics may be collected as initial fabric images, defects in the images may be annotated, and label information of the locations and categories of various defects may be obtained.

[0063] Labelimg software can be used to mark ground truth boxes on the initial fabric image, classify and calibrate the fabric defects, and generate a corresponding information file. The information file is an XML file that contains the location and category information of the defects marked in each image. Labelimg software is a visual image calibration tool.

[0064] For example, a variety of different convolutional neural networks can be used to classify and identify the initial fabric image, with different defects being represented by images of different categories.

[0065] You can first use algorithms such as Fast R-CNN, Yolov3, SSD, RFBNet (Receptive Field Block), Yolov7, etc. to train the data of the initial fabric image. According to the different parameter settings of the network model, select the appropriate Batch size (the number of data samples captured in one training) size; different network models use the same magnitude of the initial learning rate value, and use the StepLR (a learning rate scheduler provided in PyTorch) strategy to adjust the learning rate; use the Adam (Adaptive Moment Estimation) optimizer to calculate the first-order moment estimation and second-order moment estimation of the gradient, and design independent adaptive learning rates for different parameter settings.

[0066] Then, the trained initial fabric images and their corresponding category information can be subjected to image preprocessing and data enhancement to increase the number of samples required for model training and obtain fabric images, which are used to train subsequent LW-SSD models.

[0067] Image preprocessing can include adding grayscale bars to images fed into the model to prevent distortion during image resizing. Data augmentation can include flipping, rotating, cropping, adding random brightness, and adding noise to images. These methods can expand the data, increasing the number of fabric images from 1,221 to 3,660 for model training, thereby improving image recognition accuracy.

[0068] Step 102, see Figure 3 , the fabric image is input into the pre-built LW-SSD model for training to obtain the trained LW-SSD model.

[0069] The LW-SSD model includes a basic model framework, a dual-channel attention mechanism module, a multi-branch parallel void convolution module and a feature fusion module; the basic model framework is used to extract the features of fabric images; the multi-branch parallel void convolution module is used to expand the receptive field of the feature layer in the basic model framework; the dual-channel attention mechanism module is used to capture multi-scale contextual information and update the weights of the feature layers in the basic model framework; the feature fusion module is used to unify the resolution of the feature layers in the basic model framework.

[0070] For example, the SSD algorithm is improved by replacing the original backbone network VGG16 with the lightweight (LW) network MobileNetv3 to form a basic model framework. The basic model framework includes seven feature layers, including Conv3_3 feature layer, Conv4_3 feature layer, Fc7 feature layer, Conv6_2 feature layer, Conv7_2 feature layer, Conv8_2 feature layer, and Conv9_2 feature layer.

[0071] Among them, the Conv3_3 feature layer, Conv4_3 feature layer, Fc7 feature layer and Conv6_2 feature layer are the feature layers of the lightweight network; the Conv7_2 feature layer, Conv8_2 feature layer and Conv9_2 feature layer are the feature layers of the SSD algorithm.

[0072] The above Conv3_3 feature layer, Conv4_3 feature layer and Fc7 feature layer are composed of the inverse residual structure block in the Mobilenetv3 network and the lightweight attention mechanism at its end.

[0073] Overall, Conv3_3, Conv4_3, Fc7, Conv6_2, Conv7_2, Conv8_2, and Conv9_2 are connected in sequence, and the output of the previous feature layer serves as the input of the next feature layer.

[0074] Exemplarily, the output ends of the Conv4_3 feature layer, the Fc7 feature layer, the Conv6_2 feature layer, the Conv7_2 feature layer, the Conv8_2 feature layer and the Conv9_2 feature layer are all provided with a dual-channel attention mechanism module, and the dual-channel attention mechanism module is also used to update the weights of the Conv4_3 feature layer, the Fc7 feature layer, the Conv6_2 feature layer, the Conv7_2 feature layer, the Conv8_2 feature layer and the Conv9_2 feature layer.

[0075] Among them, the multi-branch parallel void convolution module MPD is embedded after the output of the backbone network Fc7 feature layer. The multi-branch parallel void convolution module MPD then enters the dual-channel attention module r-ECA. After the output from the dual-channel attention module r-ECA, it serves as the input of the subsequent feature fusion feature layer Conc4_3, and is input into the feature fusion module together with the processed Conv3_3 feature layer and Fc7 feature layer.

[0076] Specifically, the multi-branch parallel void convolution module includes a first multi-branch parallel void convolution module MPD1, a second multi-branch parallel void convolution module MPD2 and a third multi-branch parallel void convolution module MPD3; the dual-channel attention mechanism module includes a first dual-channel attention mechanism module r-ECA1, a second dual-channel attention mechanism module r-ECA2, a third dual-channel attention mechanism module r-ECA3, a fourth dual-channel attention mechanism module r-ECA4, a fifth dual-channel attention mechanism module r-ECA5 and a sixth dual-channel attention mechanism module r-ECA6.

[0077] The image of the fabric is input to the Conv3_3 feature layer, and the Conv3_3 feature layer outputs the first feature map.

[0078] The first feature map is input to the Conv4_3 feature layer and the feature fusion module, and the Conv4_3 feature layer outputs the second feature map.

[0079] The second feature map is input into the Fc7 feature layer and the feature fusion module, and the Fc7 feature layer outputs the third feature map; the third feature map is processed in turn by the second multi-branch parallel void convolution module MPD2 and the second dual-channel attention mechanism module r-ECA2 to output the second target feature map.

[0080] The second target feature map is input into the feature fusion module, which is used to fuse the first feature map, the second feature map and the second target feature map, and output a fused feature map; the fused feature map is processed in turn by the first multi-branch parallel void convolution module MPD1 and the first dual-channel attention mechanism module r-ECA1 to output the first target feature map.

[0081] The second target feature map is also input into the Conv6_2 feature layer, and the Conv6_2 feature layer outputs the fourth feature map; the fourth feature map is processed in turn by the third multi-branch parallel void convolution module MPD3 and the third dual-channel attention mechanism module r-ECA3 to output the third target feature map.

[0082] The third target feature map is input to the Conv7_2 feature layer, and the Conv7_2 feature layer outputs the fifth feature map; the fifth feature map is processed by the fourth dual-channel attention mechanism module r-ECA4 and outputs the fourth target feature map.

[0083] The fourth target feature map is input to the Conv8_2 feature layer, and the Conv8_2 feature layer outputs the sixth feature map; the sixth feature map is processed by the fifth dual-channel attention mechanism module r-ECA5 and outputs the fifth target feature map.

[0084] The fifth target feature map is input to the Conv9_2 feature layer, which outputs the seventh feature map. The seventh feature map is processed by the sixth dual-channel attention mechanism module r-ECA6 to output the sixth target feature map. The target feature maps include the first target feature map, the second target feature map, the third target feature map, the fourth target feature map, the fifth target feature map, and the sixth target feature map. The target feature map is marked with a priori boxes.

[0085] The LW-SSD model also includes a screening module; the screening module is used to obtain the target feature map and, based on the non-maximum suppression principle, filter the prior boxes that are greater than the confidence threshold as prediction boxes, and output a predicted feature map containing the prediction boxes; the prediction boxes are marked with the location and category information of the defects in the image of the fabric to be detected.

[0086] In one embodiment, in order to more clearly illustrate the structure of the LW-SSD model, an input fabric image with a size of 300×300 is taken as an example to illustrate the processing process of the fabric image by the LW-SSD model.

[0087] For example, Figure 3 As shown in the figure, taking the input fabric image size of 300×300 as an example, the fabric image with a resolution of 300×300 is input to the Conv3_3 feature layer for convolution, and the first feature map with a size of 38×38 is output; the first feature map is input to the Conv4_3 feature layer for convolution, and the second feature map with a size of 19×19 is output; the second feature map is input to the Fc7 feature layer for convolution, and the third feature map with a size of 10×10 is output.

[0088] The third feature map is input into the second multi-branch parallel void convolution module MPD2 to output the third preprocessed feature map; the third preprocessed feature map is input into the second dual channel attention mechanism module r-ECA2 to output the second target feature map.

[0089] The third feature map is input to the second multi-branch parallel dilated convolution module MPD2, and the third pre-processed feature map is output. Figure 4 As shown, the first, second, and third multi-branch parallel atrous convolution modules MPD1, MPD2, and MPD3 share the same structure: the MPD module includes four 1×1 convolutions (con2d), eight 1×3 or 3×1 convolutions (con2d), and four 3×3 depthwise separable convolutions (Spconv) with dilation rates of 1, 3, 5, and 7, respectively. Parallel branches with different dilation rates perform convolution and cascade operations. Each sub-feature map obtained through the depthwise separable convolution is stacked. The stacked result, Concat+1×1con2d, is fused with the residual edge short cut in the module, and then output as a complete preprocessed feature map. The second multi-branch parallel atrous convolution module MPD2 outputs the third preprocessed feature map. Placing the second multi-branch parallel atrous convolution module MPD2 after the Fc7 feature layer expands the receptive field of the Fc7 feature layer and improves the detection capability of small objects.

[0090] The third preprocessed feature map is input into the second dual-channel attention mechanism module r-ECA2 for processing to obtain the second target feature map. Figure 5As shown, the first, second, third, fourth, fifth, and sixth dual-channel attention modules r-ECA1, r-ECA2, r-ECA3, r-ECA4, r-ECA5, and r-ECA6 all have the same structure: the input feature map x of size w×h×C is compressed using global average pooling (Avgpool) and maximum pooling (Maxpool), respectively, to produce two sets of compressed 1×1×C feature maps. A 1D convolution is used to learn per-channel weights for the two compressed feature maps, and the two weights are concatenated to obtain a weight value σ. The convolution kernel representing the 1D convolution is K = Ψ(C). The concatenated weights are processed using a sigmoid activation function to obtain a new weight value σ. This new weight value σ is multiplied by the input feature map x to produce an output feature map x′ of size w×h×C. Here, the third preprocessed feature map is multiplied by the new weight value to produce the second target feature map. The second dual-channel attention mechanism module r-ECA2 captures multi-scale contextual information through global average pooling Avgpool and maximum pooling Maxpool, and then improves the detection accuracy of targets with large deformation scales through weight splicing and updating.

[0091] In the network, the dual-channel attention mechanism module r-ECA is added to the ends of the Conv4_3 feature layer, Fc7 feature layer, Conv6_2 feature layer, Conv7_2 feature layer, Conv8_2 feature layer and Conv9_2 feature layer, which can capture multi-scale contextual information and improve the model's detection ability for targets with large deformation scales.

[0092] Exemplarily, the first feature map output by the Conv3_3 feature layer, the second feature map output by the Conv4_3 feature layer, and the second target feature map are input into the feature fusion module FFM for fusion, unifying the resolution to a 19×19 fused feature map. The Conv3_3 feature layer, the Fc7 feature layer, and the Conv4_3 feature layer are fused by scaling the images to the same resolution through up / down sampling. The fused feature map is input into the first multi-branch parallel dilated convolution module MPD1, and the output feature map is passed into the first dual-channel attention mechanism module r-ECA1 for weight redistribution, resulting in a new Conv4_3 feature layer and a new Fc7 feature layer.

[0093] like Figure 6As shown in the figure, the feature map of the Conv3_3 feature layer (size 38×38×80) is subjected to a Maxpooling operation, followed by a 1×1 Convolution operation, and then processed with a BN-ReLU normalization function to output the feature map. Simultaneously, the feature map of the Conv3_3 feature layer is first subjected to a 1×1 Convolution operation, then processed with a BN-ReLU normalization function, and then a 3×3 Convolution operation, followed by a BN-ReLU normalization function to output the feature map. The two output feature maps are stacked and concatenated to produce the first stacked feature map.

[0094] The feature map of the Conv4_3 feature layer with a size of 19×19×112 is first subjected to a 1×1 convolution Conv operation, and then processed by the normalization-activation function BN-ReLU to output the feature map.

[0095] The feature map of the Fc7 feature layer with a size of 10×10×960 is first subjected to a 1×1 convolution Conv operation, then processed by the normalization-activation function BN-ReLU, and then the inverse pooling operation Upsampling is performed to output the feature map.

[0096] The first stacked feature map output by the Conv3_3, Conv4_3, and Fc7 feature layers is stacked again with the other two feature maps, resulting in a second stacked feature map. This second stacked feature map is then subjected to a 3×3 sparse convolution (Spconv) operation with a uniform resolution of 19×19 to produce a fused feature map. The updated Conv4_3 feature layer has a size of 19×19×112. The Conv3_3, Fc7, and Conv4_3 feature layers are fused using the Feature Fusion Module (FFM). This enriches the semantics and detail of the feature maps in the shallow Conv4_3 feature layer, enhancing the representation of small objects.

[0097] Exemplarily, the fused feature map is input into the first multi-branch parallel hole convolution module MPD1 for convolution, and then passed into the first dual-channel attention mechanism module r-ECA1 to output the first target feature map.

[0098] Exemplarily, the second target feature map is input into the Conv6_2 feature layer for processing, outputting a fourth feature map of size 5×5. The fourth feature map is input into the third multi-branch parallel dilated convolution module MPD3, outputting a third preprocessed feature map. The third preprocessed feature map is then input into the third dual-channel attention mechanism module r-ECA3, outputting a third target feature map. The third dual-channel attention mechanism module r-ECA3 captures multi-scale contextual information through global average pooling (Avgpool) and maximum pooling (Maxpool). The Conv6_2 feature layer is then updated through weight concatenation and updating, improving the detection accuracy of objects with large deformation scales.

[0099] Exemplarily, based on the new Conv6_2 feature layer, the fourth dual-channel attention mechanism module r-ECA4, the fifth dual-channel attention mechanism module r-ECA5, and the sixth dual-channel attention mechanism module r-ECA6, the feature maps of the Conv7_2 feature layer, the Conv8_2 feature layer, and the Conv9_2 feature layer are processed to output the fourth target feature, the fifth target feature, and the fifth target feature, respectively. At the same time, the Conv7_2 feature layer, the Conv8_2 feature layer, and the Conv9_2 feature layer are updated through weight splicing and updating. The feature map of the new Conv4_3 feature layer, the feature map of the new Fc7 feature layer, the feature map of the new Conv6_2 feature layer, the feature map of the new Conv7_2 feature layer, the feature map of the new Conv8_2 feature layer, and the feature map of the new Conv9_2 feature layer are feature maps of different scales.

[0100] After processing each of the above feature layers, the 300×300 image is continuously reduced in size. The feature maps of the Conv7_2, Conv8_2, and Conv9_2 layers are processed by the dual-channel attention mechanism module r-ECA and are then directly output. The resolution of the feature map output by the Conv7_2 feature layer is 3×3, the resolution of the feature map output by the Conv8_2 feature layer is 2×2, and the resolution of the feature map output by the Conv9_2 feature layer is 1×1. These feature maps of three different scales are obtained, allowing detection of objects of different sizes.

[0101] The screening module obtains all the above-mentioned target feature maps, and in the prior frames on the target feature maps, screens the prior frames that are greater than the confidence threshold as prediction frames according to the non-maximum suppression principle, and outputs the prediction feature maps containing the prediction frames; the location information and category information of the defects in the image of the fabric to be detected are marked on the prediction frames.

[0102] In one embodiment, an LW-SSD model is trained based on the aforementioned LW-SSD model structure. During the training process, fabric images are input into the LW-SSD model for feature extraction, resulting in feature maps of seven feature layers. The fabric images comprise a training set and a validation set of set sizes.

[0103] The fabric image at this time is the fabric image processed in step 101. In the fabric image, the fabric image also includes a test set, such as Figure 2 As shown in the figure, the sample ratio of the training set to the validation set can be set to 9:1, and the sample ratio of the training set plus the validation set to the test set can be set to 9:1. Based on the set sample ratio, the LW-SSD model automatically generates the train.txt, val.txt, and test.txt files corresponding to the training, validation, and test sets, respectively. It then generates the 2007_train.txt and 2007_val.txt files based on the train.txt and val.txt files. These two files can contain information such as the image name, the location of the ground-truth bounding box, and defect classification information.

[0104] For example, the training set is input into the LW-SSD model for detection training, and the training target feature map is output; the training target feature map is marked with a priori boxes, such as Figure 3 As shown in the figure, the number of prior boxes in the overall target feature map is 2278. During model training, the training set loss is calculated based on the difference between the prior boxes and the true boxes. The training set loss includes the classification loss and regression loss during the detection training process, and the LW-SSD model is updated based on the training set loss.

[0105] For example, the validation set is fed into the LW-SSD model for validation training, and a validation target feature map is output. The validation target feature map is labeled with a priori bounding boxes. During model training, the validation set loss is calculated based on the difference between the priori bounding boxes and the true bounding boxes. The validation set loss includes the classification loss and regression loss during validation training, and the LW-SSD model is updated based on the validation set loss.

[0106] Specifically, softmax loss is used for classification loss, and smooth-L1 loss is used for regression loss. The total loss is the weighted sum of the classification error and the regression error of the sum of the training set loss and the validation set loss. After calculating the total loss, the model weight file is updated and saved with the suffix .h5.

[0107] The number of prior boxes for each feature point in each feature layer is pre-set, and the size of the prior boxes varies across each layer. After processing the feature layer, each layer's prior box is compared with the ground-truth box, and the one with the highest Intersection-over-Union (IOU) value is selected. The total number of prior boxes for each feature layer is: feature layer size × number of pre-set prior boxes for each feature point. For example, if the feature layer size is 19×19 and the number of prior boxes for each feature point is 4, the total number of prior boxes for the entire feature layer is 19×19×4.

[0108] Finally, the model convergence is determined based on the training set loss and the validation set loss, and the number of iterations is set to obtain the trained LW-SSD model.

[0109] Exemplarily, according to the training set loss and the validation set loss, the model convergence is determined according to the set number of iterations to obtain a trained LW-SSD model, including: according to the training set loss and the validation set loss, obtaining a loss curve of the training set and a loss curve of the validation set, determining the parameters at the time of model convergence according to reaching the set number of iterations and the loss curve, and obtaining a trained LW-SSD model.

[0110] For example, Figure 2 As shown in Figure 1, the performance of the trained LW-SSD model can be tested on the test set to finally determine the LW-SSD model.

[0111] Step 103, see Figure 7 , the image of the fabric to be detected is input into the trained LW-SSD model to obtain the location information and category information of the defects in the image of the fabric to be detected.

[0112] In one embodiment, taking towel fabric as an example, Figure 7 This example shows how to output the location and category information for towel defects. The category information includes thread residue, hair, unfilled corners, edge defects, smudges, wire drawing breaks, and misprints. It is also possible to detect at least one of these defect categories simultaneously on the same piece of fabric.

[0113] Among them, while outputting the location information and category information of the towel's defects, it also includes the confidence level of defect recognition to prevent recognition errors, such as Figure 7As shown in the figure, the confidence levels for identifying thread residue, hair, unfilled corner, edge defects, misprint, smudge, and wire drawing are 0.64, 0.70, 0.93, 0.80, 0.92, 0.68, and 0.79, respectively.

[0114] The fabric defect detection optimization method based on the LW-SSD model of the present invention further comprises: using common evaluation indicators of target detection to analyze the defect detection results in step 104, such as Figure 8 As shown in the figure, the performance of the network model is tested; the evaluation indicators are AP value, mAP value, Params value and FPS, where AP value is the average precision of a single target, mAP value is the average of the average accuracy AP of all categories, Params value is the size of the model parameters, and FPS is the detection speed of the model. The calculation formulas for these evaluation indicators are:

[0115]

[0116]

[0117]

[0118] Where N is the number of all target types, P is the precision of the target, R is the recall rate, TP is the number of correct target detections, and M is the total number of targets.

[0119] This example uses a fabric image dataset for testing, which contains 7 types of defects. The experimental results are as follows: Figure 8 As shown. Figure 8 The average precision of individual targets for detecting small defects such as thread residue, hair, unfilled corners, edge defects, smudges, wire drawing breaks, and misprints is above 80%, and the average accuracy rate reaches 90.16%, demonstrating high detection accuracy.

[0120] If the detection performance of the network model needs to be considered, analysis can be performed based on the defect detection result indicators in step 103. It can be seen that the present invention can meet the accuracy requirements of fabric detection tasks with multiple defect types.

[0121] It can be seen that the fabric defect detection method of the embodiment of the present application is implemented by constructing the LW-SSD model, which replaces the original backbone network VGG16 of the SSD algorithm with the lightweight network MobileNetv3, which can effectively reduce the number of parameters of the model so that it can be easily used under limited hardware conditions. Adding a multi-branch parallel void convolution module MPD to the network can expand the receptive field of the shallow feature layer and improve the detection ability of small targets; the shallow and deep feature maps are fused using the feature fusion module FFM to enrich the semantics and detail information of the shallow feature map and enhance the feature representation ability of small targets. Adding a dual-channel attention mechanism module to the network can capture multi-scale contextual information and improve the model's detection ability for targets with large deformation scales. The overall robustness of the LW-SSD model is stronger, and it is easier to converge during model training. After training with the collected fabric images, it can better overcome the interference of external conditions such as light intensity and machine failures.

[0122] It should be understood that the size of the serial numbers of the above steps does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0123] See also Figure 9 Applying the above-mentioned fabric defect detection method, the embodiment of the present application also provides a fabric defect detection device, including an image acquisition module 201, a model training module 202 and a result output module 203.

[0124] The image acquisition module 201 is used to acquire images of fabrics; each fabric image is marked with the location information and category information of the defects.

[0125] The model training module 202 is used to input the image of the fabric into a pre-built LW-SSD model for training to obtain a trained LW-SSD model; the LW-SSD model includes a basic model framework, a dual-channel attention mechanism module, a multi-branch parallel void convolution module and a feature fusion module; the basic model framework is used to extract the features of the fabric image; the multi-branch parallel void convolution module is used to expand the receptive field of the feature layer in the basic model framework; the dual-channel attention mechanism module is used to capture multi-scale contextual information and update the weights of the feature layer in the basic model framework; the feature fusion module is used to unify the resolution of the feature layer in the basic model framework.

[0126] The result output module 203 is used to input the image of the fabric to be detected into the trained LW-SSD model to obtain the position information and category information of the defects in the image of the fabric to be detected.

[0127] The present application also provides a terminal device. Figure 10The terminal device 300 may include: at least one processor 310 and a memory 320, wherein the memory 320 stores a computer program that can be run on the at least one processor 310, and when the processor 310 executes the computer program, the steps in any of the above-mentioned method embodiments are implemented, for example Figure 1 Steps 101 to 103 in the illustrated embodiment.

[0128] For example, the computer program may be divided into one or more modules / units, one or more modules / units being stored in the memory 320 and executed by the processor 310 to complete the present application. The one or more modules / units may be a series of computer program segments capable of completing specific functions, and the program segments are used to describe the execution process of the computer program in the terminal device 300.

[0129] Those skilled in the art will understand that Figure 10 These are merely examples of terminal devices and do not constitute a limitation on the terminal devices. The terminal devices may include more or fewer components than shown in the figure, or a combination of certain components, or different components, such as input and output devices, network access devices, buses, etc.

[0130] The processor 310 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0131] The memory 320 can be an internal storage unit of the terminal device or an external storage device of the terminal device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. The memory 320 is used to store the computer program and other programs and data required by the terminal device. The memory 320 can also be used to temporarily store data that has been output or is about to be output.

[0132] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be classified into address buses, data buses, and control buses. For ease of illustration, the buses in the drawings of this application are not limited to just one bus or just one type of bus.

[0133] The fabric defect detection method provided in the embodiment of the present application can be applied to terminal devices such as computers, tablet computers, laptop computers, netbooks, personal digital assistants (PDAs), etc. The embodiment of the present application does not impose any restrictions on the specific type of terminal devices.

[0134] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in each embodiment of the above-mentioned fabric defect detection method can be implemented.

[0135] An embodiment of the present application provides a computer program product. When the computer program product is run on a mobile terminal, the mobile terminal can implement the steps in each embodiment of the above-mentioned fabric defect detection method when executing the computer program product.

[0136] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process of the above-mentioned method embodiment by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can at least include: any entity or device capable of carrying computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, mobile hard drive, magnetic disk, or optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals or telecommunication signals.

[0137] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0138] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0139] In the embodiments provided in this application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0140] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0141] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A fabric defect detection method, characterized in that: include: Acquire images of fabrics; each image of the fabrics is marked with location information and category information of defects; The image of the fabric is input into a pre-built LW-SSD model for training to obtain a trained LW-SSD model; the LW-SSD model includes a basic model framework, a dual-channel attention mechanism module, a multi-branch parallel void convolution module and a feature fusion module; the basic model framework is used to extract features of the fabric image; the multi-branch parallel void convolution module is used to expand the receptive field of the feature layer in the basic model framework; the dual-channel attention mechanism module is used to capture multi-scale contextual information and update the weights of the feature layer in the basic model framework; The feature fusion module is used to unify the resolution of the feature layer in the basic model framework; Inputting the image of the fabric to be detected into the trained LW-SSD model to obtain location information and category information of defects in the image of the fabric to be detected; The basic model framework includes 7 feature layers, including Conv3_3 feature layer, Conv4_3 feature layer, Fc7 feature layer, Conv6_2 feature layer, Conv7_2 feature layer, Conv8_2 feature layer and Conv9_2 feature layer; Among them, the Conv3_3 feature layer, the Conv4_3 feature layer, the Fc7 feature layer and the Conv6_2 feature layer are feature layers of the lightweight network; the Conv7_2 feature layer, the Conv8_2 feature layer and the Conv9_2 feature layer are feature layers of the SSD algorithm; The output ends of the Conv4_3 feature layer, the Fc7 feature layer, the Conv6_2 feature layer, the Conv7_2 feature layer, the Conv8_2 feature layer and the Conv9_2 feature layer are all provided with the dual-channel attention mechanism module, and the dual-channel attention mechanism module is also used to update the weights of the Conv4_3 feature layer, the Fc7 feature layer, the Conv6_2 feature layer, the Conv7_2 feature layer, the Conv8_2 feature layer and the Conv9_2 feature layer.

2. The fabric defect detection method according to claim 1, wherein: The multi-branch parallel void convolution module includes a first multi-branch parallel void convolution module, a second multi-branch parallel void convolution module and a third multi-branch parallel void convolution module; the dual-channel attention mechanism module includes a first dual-channel attention mechanism module, a second dual-channel attention mechanism module, a third dual-channel attention mechanism module, a fourth dual-channel attention mechanism module, a fifth dual-channel attention mechanism module and a sixth dual-channel attention mechanism module; The fabric image is input to the Conv3_3 feature layer, and the Conv3_3 feature layer outputs a first feature map; The first feature map is input into the Conv4_3 feature layer and the feature fusion module, and the Conv4_3 feature layer outputs a second feature map; The second feature map is input into the Fc7 feature layer and the feature fusion module, and the Fc7 feature layer outputs a third feature map; the third feature map is sequentially processed by the second multi-branch parallel void convolution module and the second dual-channel attention mechanism module to output a second target feature map; The second target feature map is input into the feature fusion module, and the feature fusion module is used to fuse the first feature map, the second feature map and the second target feature map, and output a fused feature map; The fused feature map is processed in sequence by the first multi-branch parallel hole convolution module and the first dual-channel attention mechanism module to output a first target feature map; The second target feature map is also input into the Conv6_2 feature layer, and the Conv6_2 feature layer outputs a fourth feature map; the fourth feature map is sequentially processed by the third multi-branch parallel void convolution module and the third dual-channel attention mechanism module to output a third target feature map; The third target feature map is input to the Conv7_2 feature layer, and the Conv7_2 feature layer outputs a fifth feature map; the fifth feature map is processed by the fourth dual-channel attention mechanism module to output a fourth target feature map; The fourth target feature map is input to the Conv8_2 feature layer, and the Conv8_2 feature layer outputs a sixth feature map; The sixth feature map is processed by the fifth dual-channel attention mechanism module to output a fifth target feature map; The fifth target feature map is input to the Conv9_2 feature layer, and the Conv9_2 feature layer outputs the seventh feature map; the seventh feature map is processed by the sixth dual-channel attention mechanism module to output the sixth target feature map; the target feature map includes the first target feature map, the second target feature map, the third target feature map, the fourth target feature map, the fifth target feature map and the sixth target feature map; the target feature map is marked with a priori box.

3. The fabric defect detection method according to claim 2, wherein: The LW-SSD model also includes a screening module; the screening module is used to obtain the target feature map, and filter the prior boxes greater than the confidence threshold as prediction boxes based on the non-maximum suppression principle, and output a prediction feature map containing the prediction boxes; the location information and category information of the defects in the image of the fabric to be detected are marked on the prediction boxes.

4. The fabric defect detection method according to claim 1, wherein: The fabric images include a training set and a validation set of set proportions; The inputting the fabric image into a pre-built LW-SSD model for training to obtain a trained LW-SSD model includes: Inputting the training set into the LW-SSD model for training to obtain the training set loss; Input the validation set into the LW-SSD model for training to obtain the validation set loss; According to the set number of iterations and the convergence of the training set loss and the validation set loss, the weight value of the LW-SSD model is updated to obtain the trained LW-SSD model.

5. The fabric defect detection method according to claim 4, wherein: Each of the fabric images is marked with a real frame to indicate the location and category of the defect; Inputting the training set into the LW-SSD model for training to obtain the training set loss includes: Input the training set into the LW-SSD model for training, and output a training target feature map; the training target feature map is marked with a priori box; During the training process, the training set loss is calculated based on the difference between the prior frame and the true frame on the training target feature map, and the training set loss includes the classification loss and regression loss in the training process.

6. A fabric defect detection device, characterized in that: include: An image acquisition module, used for acquiring an image of the fabric; Each image of the fabric is marked with location information and category information of the defect; A model training module is configured to input the fabric image into a pre-built LW-SSD model for training to obtain a trained LW-SSD model; the LW-SSD model includes a basic model framework, a dual-channel attention mechanism module, a multi-branch parallel dilated convolution module, and a feature fusion module; the basic model framework is configured to extract features of the fabric image; the multi-branch parallel dilated convolution module is configured to expand the receptive field of the feature layer in the basic model framework; the dual-channel attention mechanism module is configured to capture multi-scale contextual information and update the weights of the feature layer in the basic model framework; The feature fusion module is used to unify the resolution of the feature layer in the basic model framework; A result output module, configured to input the image of the fabric to be detected into the trained LW-SSD model to obtain position information and category information of defects in the image of the fabric to be detected; The basic model framework includes 7 feature layers, including Conv3_3 feature layer, Conv4_3 feature layer, Fc7 feature layer, Conv6_2 feature layer, Conv7_2 feature layer, Conv8_2 feature layer and Conv9_2 feature layer; Among them, the Conv3_3 feature layer, the Conv4_3 feature layer, the Fc7 feature layer and the Conv6_2 feature layer are feature layers of the lightweight network; the Conv7_2 feature layer, the Conv8_2 feature layer and the Conv9_2 feature layer are feature layers of the SSD algorithm; The output ends of the Conv4_3 feature layer, the Fc7 feature layer, the Conv6_2 feature layer, the Conv7_2 feature layer, the Conv8_2 feature layer and the Conv9_2 feature layer are all provided with the dual-channel attention mechanism module, and the dual-channel attention mechanism module is also used to update the weights of the Conv4_3 feature layer, the Fc7 feature layer, the Conv6_2 feature layer, the Conv7_2 feature layer, the Conv8_2 feature layer and the Conv9_2 feature layer.

7. A terminal device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, the fabric defect detection method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the fabric defect detection method according to any one of claims 1 to 5 is implemented.

Images