A cloth defect detection method, device and readable medium

Abstract

The application discloses a cloth defect detection method and device and readable medium, and the cloth defect detection method comprises the steps: obtaining a cloth image to be detected and a template image; constructing an improved Yolov5s network and training to obtain a cloth defect detection model, wherein the improved Yolov5s network comprises a backbone feature extraction network, a neck feature fusion network and a head prediction network, the backbone feature extraction network adopts a CSPDarkNet, the neck feature fusion network comprises a feature filtering pyramid network and a path aggregation network of a fusion convolution block attention module, and a multi-class focus confidence loss function is used to replace a cross-entropy confidence loss function in a training process; and the cloth image to be detected and the template image are input into the cloth defect detection model to obtain a cloth detection result. The method focuses on the problems of uneven distribution of defect types, different detection difficulties of different defects and the like in actual cloth defect detection, filters redundant background features and purifies key foreground features, reduces network complexity, and improves detection precision and efficiency.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a method, apparatus, and readable medium for detecting fabric defects. Background Technology

[0002] With the development of intelligent manufacturing, the garment manufacturing industry uses fabric defect detection equipment to detect fabric defects and improve fabric production quality. However, the high speed of production lines makes traditional manual inspection methods for fabric defects too slow, inefficient, and prone to false positives and incorrect detections. Furthermore, these methods can cause irreversible damage to the eyes of quality inspectors. Therefore, machine learning-based and deep learning-based fabric defect detection methods have been proposed.

[0003] The most important factor in machine learning-based fabric defect detection methods is the design and selection of hand-crafted features. Because machine learning methods require manual design when extracting features, hand-crafted features often have low semantic levels, resulting in low detection accuracy on fabric images with small differences and complex textures.

[0004] Deep learning-based fabric defect detection methods no longer rely on manually designed shallow features, but instead utilize convolutional neural networks to automatically extract deep features. These methods strike a balance between detection accuracy and efficiency. Among them, YOLOv5s-based deep learning network models achieve satisfactory results on defective fabric datasets with large scale, simple textures, minimal background interference, and uniform category distribution. However, existing YOLOv5s-based fabric defect detection methods perform poorly on defective fabric datasets with significant background interference and uneven defect type distribution, limiting the model's detection accuracy and generalization ability. Summary of the Invention

[0005] In view of the aforementioned technical problems, the purpose of the embodiments of this application is to provide a method, apparatus, and readable medium for detecting fabric defects, thereby solving the technical problems mentioned in the background section.

[0006] In a first aspect, the present invention provides a method for detecting fabric defects, comprising the following steps:

[0007] Obtain the fabric image to be detected and the template image;

[0008] An improved Yolov5s network was constructed and trained to obtain a fabric defect detection model. The improved Yolov5s network includes a backbone feature extraction network, a neck feature fusion network, and a head prediction network. The backbone feature extraction network adopts CSPDarkNet, and the neck feature fusion network includes a feature filtering pyramid network and a path aggregation network that fuses convolutional block attention modules. During training, a multi-class focusing confidence loss function is used instead of the cross-entropy confidence loss function.

[0009] Input the fabric image to be detected and the template image into the fabric defect detection model to obtain the fabric detection results.

[0010] Preferably, CSPDarkNet includes a first unit, a second unit, a third unit, a fourth unit, and a fifth unit connected in sequence. The first unit includes a Focus module and a first CBS module connected in sequence. The second unit includes a second CBS module and a first CSP1_x module connected in sequence. The third unit includes a third CBS module and a second CSP1_x module connected in sequence. The fourth unit includes a fourth CBS module and a third CSP1_x module connected in sequence. The fifth unit includes a fifth CBS module, a fourth CSP1_x module, and an SPPF module connected in sequence. When a fabric image is input into CSPDarkNet, the first effective feature map, the second effective feature map, and the third effective feature map are output from the third unit, the fourth unit, and the fifth unit, respectively.

[0011] Preferably, the feature filtering pyramid network includes an FPN feature fusion network, a redundant feature filtering network, and a template feature extraction network. The first effective feature map, the second effective feature map, the third effective feature map, and the template image are respectively input into the FPN feature fusion network. The third effective feature map is input into the sixth CBS module to obtain the third intermediate feature map. The feature map obtained from the third intermediate feature map through the first Upsample module is concatted with the second effective feature map. Then, it is fused with the seventh CBS module through the first CSP2_3 module to obtain the second intermediate feature map. The feature map obtained from the second intermediate feature map through the second Upsample module is concatted with the first effective feature map. Then, it is fused with the eighth CBS module through the second CSP2_3 module to obtain the first intermediate feature map.

[0012] Preferably, the template image is input into the template feature extraction network. The template image is input into the ninth CBS module to obtain the first template feature map. The first template feature map is input into the Downsample1 module to obtain the second template feature map. The second template feature map is input into the Downsample2 module to obtain the third template feature map.

[0013] Preferably, the first intermediate feature map, the second intermediate feature map, the third intermediate feature map, the first template feature map, the second template feature map, and the third template feature map are input into a redundant feature filtering network. The first intermediate feature map and the first template feature map are concatted and then passed through a 1×1 convolutional kernel to obtain the first filtered feature map. The second intermediate feature map and the second template feature map are concatted and then passed through a 1×1 convolutional kernel to obtain the second filtered feature map. The third intermediate feature map and the third template feature map are concatted and then passed through a 1×1 convolutional kernel to obtain the third filtered feature map.

[0014] Preferably, the first, second, and third filtered feature maps are input into the path aggregation network of the fusion convolutional block attention module. The first filtered feature map is input into the first DWConv module to obtain the first fused feature map. The feature map obtained from the first fused feature map through the first CBAM module is concatted with the second filtered feature map, and then fused through five second DWConv modules to obtain the second fused feature map. The feature map obtained from the second fused feature map through the third DWConv module and the second CBAM module is concatted with the third filtered feature map, and then fused through five fourth DWConv modules to obtain the third fused feature map. The first, second, and third fused feature maps are then input into the head prediction network to obtain the fabric detection result.

[0015] As a preferred option, the multi-class focused confidence loss function is:

[0016]

[0017] Where y represents the label, p represents the confidence level, and (1-p) γ Let β represent the difficulty balance factor, where i represents the i-th type of fabric defect, N represents the total number of fabric defect types, and β c Represented as the class balance factor:

[0018] β c =ω / N c ;

[0019] Where ω represents the balance coefficient, N c This represents the number of samples belonging to category c.

[0020] Secondly, the present invention provides a fabric defect detection device, comprising:

[0021] The image acquisition module is configured to acquire the fabric image to be detected and the template image;

[0022] The model building module is configured to build and train an improved Yolov5s network to obtain a fabric defect detection model. The improved Yolov5s network includes a backbone feature extraction network, a neck feature fusion network, and a head prediction network. The backbone feature extraction network adopts CSPDarkNet, and the neck feature fusion network includes a feature filtering pyramid network and a path aggregation network that fuses convolutional block attention modules. During training, a multi-class focusing confidence loss function is used instead of the cross-entropy confidence loss function.

[0023] The prediction module is configured to input the fabric image to be detected and the template image into the fabric defect detection model to obtain the fabric detection results.

[0024] Thirdly, the present invention provides an electronic device including one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any implementation of the first aspect.

[0025] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any of the implementations of the first aspect.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] (1) The fabric defect detection method proposed in this invention filters out redundant feature information generated by FPN and PAN networks during feature fusion by using a neck feature fusion network composed of a feature filtering pyramid network and a path aggregation network of a fusion convolutional block attention module. Furthermore, by combining channel attention and spatial attention mechanisms, the information content of the fused feature map is improved, key features are highlighted, and the detection accuracy of the model is enhanced.

[0028] (2) The fabric defect detection method proposed in this invention adopts a first DWConv module and a second DWConv module in the path aggregation network of the fused convolutional block attention module. The first DWConv module and the second DWConv module reduce the network computation and improve the computation efficiency, thereby achieving network lightweighting.

[0029] (3) In the fabric defect detection method proposed in this invention, the multi-class focusing confidence loss function is used instead of the cross-entropy confidence loss function during the training process of the model. The multi-class focusing confidence loss function takes into account the problems of class imbalance and difficulty imbalance. By adding two weight adjustment factors, the extremely uneven distribution of fabrics with different defect types and the different detection difficulties of different types of defects are improved, thereby improving the generalization ability of the model. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is an exemplary device architecture diagram in which an embodiment of this application can be applied;

[0032] Figure 2 This is a schematic flowchart of a fabric defect detection method according to an embodiment of this application;

[0033] Figure 3 This is a schematic diagram of the backbone feature extraction network of the improved Yolov5s network for the fabric defect detection method according to an embodiment of this application.

[0034] Figure 4 A schematic diagram of the FPN feature fusion network of the improved Yolov5s network for the fabric defect detection method of this application.

[0035] Figure 5 This is a schematic diagram of the template feature extraction network of the improved Yolov5s network for the fabric defect detection method according to an embodiment of this application.

[0036] Figure 6 A schematic diagram of the redundant feature filtering network of the improved Yolov5s network for the fabric defect detection method of this application.

[0037] Figure 7 This is a schematic diagram of the path aggregation network of the fused convolutional block attention module of the improved Yolov5s network for the fabric defect detection method of this application.

[0038] Figure 8 A schematic diagram of the head prediction network of the improved Yolov5s network for the fabric defect detection method of this application.

[0039] Figure 9 This is a schematic diagram of a fabric defect detection device according to an embodiment of this application;

[0040] Figure 10 This is a schematic diagram of the structure of a computer device suitable for implementing the electronic device of the present application. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0042] Figure 1 An exemplary device architecture 100 is shown that can be applied to the fabric defect detection method or fabric defect detection apparatus of the embodiments of this application.

[0043] like Figure 1 As shown, the device architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, and 103 and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0044] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various applications, such as data processing applications and file processing applications, can be installed on terminal devices 101, 102, and 103.

[0045] Terminal devices 101, 102, and 103 can be either hardware or software. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices, including but not limited to smartphones, tablets, laptops, and desktop computers. When terminal devices 101, 102, and 103 are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software programs or software modules (e.g., software programs or software modules used to provide distributed services) or as a single software program or software module. No specific limitations are imposed here.

[0046] Server 105 can be a server that provides various services, such as a background data processing server that processes files or data uploaded by terminal devices 101, 102, and 103. The background data processing server can process the acquired files or data and generate processing results.

[0047] It should be noted that the fabric defect detection method provided in this application embodiment can be executed by server 105 or by terminal devices 101, 102, and 103. Correspondingly, the fabric defect detection device can be set in server 105 or in terminal devices 101, 102, and 103.

[0048] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Any number of terminal devices, networks, and servers can be included depending on implementation needs. If the data being processed does not need to be retrieved remotely, the above architecture may not include a network, requiring only servers or terminal devices.

[0049] Figure 2 An embodiment of this application provides a method for detecting fabric defects, comprising the following steps:

[0050] S1, Obtain the fabric image to be detected and the template image.

[0051] Specifically, in practical applications, images of the fabric to be inspected are captured by a camera, and fabric images with different levels of texture complexity are selected as template images to avoid interference with fabric defect detection caused by different levels of texture complexity.

[0052] S2. Construct and train an improved Yolov5s network to obtain a fabric defect detection model. The improved Yolov5s network includes a backbone feature extraction network, a neck feature fusion network, and a head prediction network. The backbone feature extraction network adopts CSPDarkNet, and the neck feature fusion network includes a feature filtering pyramid network and a path aggregation network that fuses convolutional block attention modules. During training, a multi-class focusing confidence loss function is used instead of the cross-entropy confidence loss function.

[0053] In a specific embodiment, CSPDarkNet includes a first unit, a second unit, a third unit, a fourth unit, and a fifth unit connected in sequence. The first unit includes a Focus module and a first CBS module connected in sequence. The second unit includes a second CBS module and a first CSP1_x module connected in sequence. The third unit includes a third CBS module and a second CSP1_x module connected in sequence. The fourth unit includes a fourth CBS module and a third CSP1_x module connected in sequence. The fifth unit includes a fifth CBS module, a fourth CSP1_x module, and an SPPF module connected in sequence. When a fabric image is input into CSPDarkNet, the third, fourth, and fifth units respectively output a first effective feature map, a second effective feature map, and a third effective feature map.

[0054] For details, please refer to Figure 3(b) The Focus module performs a one-pixel-interval sampling operation on the input image to achieve a 2x downsampling. This downsampling can preserve the information of the original image to the greatest extent and achieve a 4x increase in the number of feature channels, thereby reducing network parameters and improving computation speed. The first to ninth CBS modules all consist of three parts: convolution (Conv), batch normalization (BN), and SiLU (Sigmoid Linear Unit) activation. The CBS module is the most basic convolutional feature extraction module, and its function is to extract multi-layer feature maps from the image.

[0055] The first through fourth CSP1_x modules are all improvements on the CSP structure of the Cross Stage Partial Network (CSPNet). They divide the input into two branches: one branch consists of a cascaded CBS unit and a Bottleneck_1 unit, and the other branch consists of CBS units. The Bottleneck_1 unit borrows from the classic residual block structure, dividing the input into a direct mapping part and a residual mapping part.

[0056] The following gives the feature x of an arbitrary depth cell L after processing by the Bottleneck_1 cell. L The expression:

[0057]

[0058] Where, x L Let x be a characteristic of a cell of arbitrary depth L. l Represented as the characteristics of shallow unit l, F(x) i W i The function denoted as is the residual function between shallow unit l and arbitrarily deep unit L. A single CSP1_x module operation maintains the same scale and number of channels in the output feature map, but the feature map contains higher-level semantic information. The CSP1_x module avoids the gradient vanishing phenomenon caused by deepening the network.

[0059] The SPPF module is an improvement on the Spatial Pyramid Pooling (SPP) module. It obtains four feature maps at different scales by concatenating CBS and MaxPool2d units, then concatenates these feature maps, and finally adjusts the dimensions using a CBS unit. In a single SPPF operation, the scale and number of channels of the output feature map remain unchanged, but multi-scale feature fusion is achieved. Therefore, the SPPF module performs well in detecting targets with large scale differences.

[0060] The structure of CSPDarkNet is as follows Figure 3 As shown in (a), the stacking order is as follows:

[0061] First, the fabric image with a scale of 640×640 and 3 channels after data augmentation is processed by the first unit obtained by sequentially concatenating the Focus module and the first CBS module. The purpose is to adjust the feature map scale and increase the number of feature map channels to obtain a feature map with a scale of 320×320 and 64 channels.

[0062] Secondly, the feature map obtained from the first unit is processed by the second unit, which is composed of the second CBS module and the first CSP1_x module in sequence, and the third unit, which is composed of the third CBS module and the second CSP1_x module in sequence, to extract features. The purpose is to obtain the shallow features required by the Neck Feature Fusion Network, and to obtain the first effective feature map with a scale of 80×80 and 256 channels.

[0063] Secondly, the first effective feature map obtained from the third unit is used for feature extraction through the fourth unit, which is composed of the fourth CBS module and the third CSP1_x module in sequence. The purpose is to obtain the intermediate features required by the Neck Feature Fusion Network, and to obtain the second effective feature map with a scale of 40×40 and 512 channels.

[0064] Finally, the second effective feature map obtained from the fourth unit is used for feature extraction through the fifth unit, which consists of the fifth CBS module, the fourth CSP1_x module, and the SPPF module, in sequence. The purpose is to obtain the deep features required by the Neck Feature Fusion Network (Neck), resulting in a third effective feature map with a scale of 20×20 and 1024 channels.

[0065] In a specific embodiment, the feature filtering pyramid network includes an FPN feature fusion network, a redundant feature filtering network, and a template feature extraction network. The first effective feature map, the second effective feature map, the third effective feature map, and the template image are respectively input into the FPN feature fusion network. The third effective feature map is input into the sixth CBS module to obtain the third intermediate feature map. The feature map obtained from the third intermediate feature map through the first Upsample module is concatted with the second effective feature map, and then fused with the seventh CBS module through the first CSP2_3 module to obtain the second intermediate feature map. The feature map obtained from the second intermediate feature map through the second Upsample module is concatted with the first effective feature map, and then fused with the eighth CBS module through the second CSP2_3 module to obtain the first intermediate feature map.

[0066] For details, please refer to Figure 4(b) Both the first and second CSP2_3 modules belong to the CSP2_x module family. The CSP2_x module, like the aforementioned CSP1_x module, uses a CSP structure, differing only in application scenario and the Bottleneck unit: the CSP1_x module is used in the backbone feature extraction network (CSPDarkNet), while the CSP2_x module is used in the neck feature fusion network (Neck); the CSP1_x module uses the Bottleneck_1 unit, while the CSP2_x module uses the Bottleneck_2 unit. The Bottleneck_2 unit is composed of two sets of CBS modules cascaded together. A single CSP2_x module operation maintains the same feature map scale and number of channels, but the feature map contains higher-level semantic information. The CSP2_x module avoids the gradient vanishing phenomenon caused by network deepening.

[0067] The first and second Upsample modules interpolate the feature map to achieve a 2x upsampling of the feature map.

[0068] refer to Figure 4 (a) The FPN feature fusion network includes the following steps:

[0069] First, the deep features (i.e., the third effective feature map) obtained from the backbone feature extraction network (CSPDarkNet) are channel-adjusted by the sixth CBS module to obtain a third intermediate feature map that matches the third template feature map. Second, the feature map obtained from the third intermediate feature map by the first Upsample module is concatenated with the intermediate features (i.e., the second effective feature map) obtained from the CSPDarkNet network, and then fused by the first CSP2_3 module and the seventh CBS module. The objectives are: 1. To obtain a second intermediate feature map that matches the second template feature map; 2. To retain the fusion information of the second and third effective feature maps. Next, the feature map obtained from the second intermediate feature map by the second Upsample module is concatenated with the shallow features (i.e., the first effective feature map) obtained from the CSPDarkNet network, and then fused by the second CSP2_3 module and the eighth CBS module. The objectives are: 1. To obtain a first intermediate feature map that matches the first template feature map; 2. To retain the fusion information of the first, second, and third effective feature maps. Finally, feature fusion from the first to the third effective feature map is achieved. Finally, the first to third intermediate feature maps are obtained, including: a third intermediate feature map with a scale of 20×20 and 512 channels, a second intermediate feature map with a scale of 40×40 and 256 channels, and a first intermediate feature map with a scale of 80×80 and 256 channels.

[0070] The Feature Fusion Network (F-FPN) retains the top-down feature fusion idea of ​​FPN, fusing deep features at the end of the network with shallow features at the front end. After fusion, the amount of information increases and the detection effect of small targets is improved. F-FPN uses flawless template images containing different background information to adaptively remove redundant background features brought about by multi-scale feature fusion, thereby improving the purity of key foreground features.

[0071] In a specific embodiment, the template image is input into the template feature extraction network. The template image is input into the ninth CBS module to obtain the first template feature map. The first template feature map is input into the Downsample1 module to obtain the second template feature map. The second template feature map is input into the Downsample2 module to obtain the third template feature map.

[0072] For details, please refer to Figure 5 (b) Both the Downsample1 and Downsample2 modules are composed of CBS and CSP2_x modules connected in series. The CSP2_x modules all use 3 Bottleneck_2 units. The purpose of the Downsample1 and Downsample2 modules is to achieve a 2x downsampling of the template image.

[0073] refer to Figure 5 (a) The template feature extraction network includes the following steps:

[0074] Ten template images, each with a scale of 640×640 and 3 channels, are input into the template feature extraction network. The ninth CBS module performs channel adjustment to obtain a first template feature map that matches the first intermediate feature map. The first template feature map is then downsampled by a factor of 2 using the Downsample1 module to obtain a second template feature map that matches the second intermediate feature map. The second template feature map is then downsampled by a factor of 2 using the Downsample2 module to obtain a third template feature map that matches the third intermediate feature map. Finally, the first to third template feature maps are obtained, including: a third template feature map with a scale of 20×20 and 512 channels; a second template feature map with a scale of 40×40 and 256 channels; and a first template feature map with a scale of 80×80 and 256 channels.

[0075] In a specific embodiment, the first intermediate feature map, the second intermediate feature map, the third intermediate feature map, the first template feature map, the second template feature map, and the third template feature map are input into a redundant feature filtering network. The first intermediate feature map and the first template feature map are concatted together and then passed through a 1×1 convolutional kernel to obtain the first filtered feature map. The second intermediate feature map and the second template feature map are concatted together and then passed through a 1×1 convolutional kernel to obtain the second filtered feature map. The third intermediate feature map and the third template feature map are concatted together and then passed through a 1×1 convolutional kernel to obtain the third filtered feature map.

[0076] Specifically, the redundant feature filtering network can adaptively filter redundant features by using a 1×1 convolutional kernel to obtain feature maps from the FPN feature fusion network and the template feature extraction network.

[0077] refer to Figure 6 The redundant feature filtering network includes the following steps:

[0078] The first to third intermediate feature maps and the first to third template feature maps are input into a redundant feature filtering network, where they are concatenated and processed. The concatenated feature maps are then passed through a 1×1 convolutional kernel to adaptively remove redundant background features. Finally, the first to third filtered feature maps are obtained, including: a third filtered feature map with a scale of 20×20 and 512 channels; a second filtered feature map with a scale of 40×40 and 256 channels; and a first filtered feature map with a scale of 80×80 and 256 channels.

[0079] In a specific embodiment, the first filtered feature map, the second filtered feature map, and the third filtered feature map are input into the path aggregation network of the fusion convolutional block attention module. The first filtered feature map is input into the first DWConv module to obtain the first fused feature map. The feature map obtained by the first fused feature map through the first CBAM module is concatted with the second filtered feature map, and then fused through five second DWConv modules to obtain the second fused feature map. The feature map obtained by the second fused feature map through the third DWConv module and the second CBAM module is concatted with the third filtered feature map, and then fused through five fourth DWConv modules to obtain the third fused feature map. The first fused feature map, the second fused feature map, and the third fused feature map are input into the head prediction network to obtain the fabric detection result.

[0080] For details, please refer to Figure 7(b) The first and second CBAM modules mainly consist of two parts: the Channel Attention module (CAM) and the Spatial Attention module (SAM). The CAM module performs max pooling and average pooling operations on the feature map, and then passes the results through a weight-shared MLP network and a sigmoid activation unit. The CAM module outputs a 1×1 feature map with the number of channels remaining unchanged. It represents the weights of different channel features, focusing more on which channel features are more meaningful for the detection task. The SAM module is usually placed after the CAM module. The SAM module performs spatial max pooling and spatial average pooling operations on the feature map, and then passes the results through a 7×7 convolutional kernel with 2 elements and a sigmoid activation unit. The SAM module outputs a feature map with a constant scale and 1 channel, representing the weights of different spatial locations, focusing more on the significance of features at a particular location for the detection task. Finally, both the first CBAM module and the second CBAM module obtain channel weights and spatial weights, focusing more on important features in the detection task.

[0081] The first through fourth DWConv modules each consist of two parts: depthwise convolution and pointwise convolution. Depthwise convolution processes width and height information, with the number of kernels matching the number of channels in the input feature map. Each kernel only calculates the feature map of its corresponding channel. Pointwise convolution processes channel orientation information, with a 1×1 kernel size and the same number of channels as the input feature map. The number of kernels determines the number of channels in the output feature map. These four DWConv modules reduce network computation and improve computational efficiency, thus achieving a lightweight network.

[0082] refer to Figure 7 (a) The path aggregation network (CBAM-PAN network) that integrates convolutional block attention modules includes the following steps:

[0083] First, the first to third filtered feature maps are input into the CBAM-PAN network. The first filtered feature map is used for feature extraction by the first DWConv module to obtain the first fused feature map. Second, the feature map obtained from the first fused feature map by the first CBAM module is concatted with the second filtered feature map. The concatenated feature map is then fused through five second DWConv modules. The objectives are: 1. To obtain the second fused feature map; 2. To retain the fusion information of the first and second filtered feature maps. Next, the second fused feature map is concatted with the feature map obtained from the third DWConv module and the second CBAM module, and then fused through five fourth DWConv modules. The objectives are: 1. To obtain the third fused feature map; 2. To retain the fusion information of the first, second, and third filtered feature maps. Finally, the first to third fused feature maps are obtained, including: a third fused feature map with a scale of 20×20 and 1024 channels; a second fused feature map with a scale of 40×40 and 512 channels; and a first fused feature map with a scale of 80×80 and 256 channels. (Reference) Figure 8 The first fused feature map, the second fused feature map, and the third fused feature map are respectively input into the three detection heads in the head prediction network to obtain the prediction map and the fabric defect category.

[0084] CBAM-PAN retains the bottom-up feature fusion idea of ​​PAN, aiming to solve the problem of blurred low-level information caused by FPN. By adding new fusion paths, it strengthens the localization information of the network. CBAM-PAN adds a CBAM module at the end of PAN to correct the feature map, enhance effective features with high detection value, and reduce redundant features with low detection value. Considering that CBAM-PAN requires multiple convolution operations, depthwise separable convolution (DWConv) is used instead of the Conv module to reduce the network's computational load and improve computational efficiency.

[0085] Specifically, the training process of the improved Yolov5s network also includes: obtaining the dataset and labeling it, specifically including:

[0086] From the fabric images provided by a textile company, select fabric images containing the following 7 types of defects: stains, knots, coarse fibers, lint, skipped yarns, holes, and sags, as well as 10 template images with different levels of texture information.

[0087] Images with problems such as blurriness, incorrect data format, or excessive similarity are removed from the defective fabric images and template images to obtain the initial screening images, namely the defective fabric images and template fabric images that have been initially screened.

[0088] Data augmentation was performed on the initially screened defective fabric images and template images. The augmentation operation involved dividing the initially screened images into blocks and then flipping the blocks to increase the number and diversity of samples. The images were then adjusted to a scale of 640×640 and 3 channels, and this was used as the dataset. The images in the dataset were labeled using a labeling tool to obtain a label file containing the fabric defect category, the center point coordinates of the rectangle, and the width and height. This label file was used as the dataset label.

[0089] In a specific embodiment, a multi-class focused confidence loss function is used instead of the cross-entropy confidence loss function during the training process of the improved Yolov5s network. The multi-class focused confidence loss function is as follows:

[0090]

[0091] Where y represents the label, p represents the confidence level, and (1-p) γ Let β represent the difficulty balance factor, where i represents the i-th type of fabric defect, N represents the total number of fabric defect types, and β c Represented as the class balance factor:

[0092] β c =ω / N c ;

[0093] Where ω represents the balance coefficient, N c This represents the number of samples belonging to category c.

[0094] Specifically, there are two problems with actual fabric defect detection:

[0095] (1) The distribution of fabrics with different defect types is extremely uneven. For example, the number of defect types "staining" is much greater than that of defect types "skipped yarn". Uneven distribution will lead to poor model training effect. The trained fabric defect detection model performs well in detecting fabrics with widely distributed defects, but performs poorly in detecting fabrics with small distribution defects, and the model's generalization ability is insufficient.

[0096] (2) In addition to the uneven distribution of defect types, there is also the problem that different types of defects are of different detection difficulty. Therefore, it is desirable to reduce the loss value of easily detectable categories and increase the loss value of difficult-to-detect categories.

[0097] Therefore, the Multi-Class Focused Confidence Loss Function (MFL) is used instead of the Cross-Entropy Confidence Loss Function (CEL) in YOLOv5s. The Multi-Class Focused Confidence Loss Function (MFL) is an improvement on the Focused Confidence Loss Function (FL).

[0098] Among them, the category balance factor β c Considering the uneven distribution of defect types during fabric defect detection, βc The purpose is to reduce the weight of the loss function for samples with a large distribution of defect types, and increase the weight of the loss function for samples with a small distribution of defect types. By balancing the weights of different defect types, the model corrects for uneven distribution of defect types and improves its generalization ability.

[0099] Difficulty balance factor (1-p) γ The varying difficulty of detecting different types of defects in fabric defect detection was considered. This was addressed by adding (1-p). γ The coefficients reduce the weight of the loss function for easily detectable samples and increase the weight of the loss function for difficult-to-detect samples. The hyperparameter γ increases the scalability of the loss function.

[0100] S3: Input the fabric image to be detected and the template image into the fabric defect detection model to obtain the fabric detection results.

[0101] Specifically, the collected fabric images to be detected and template images are input into the trained fabric defect detection model to predict the fabric detection results. The fabric detection results include the fabric defect type, the target rectangle and its center point coordinates and width and height.

[0102] Further reference Figure 9 As an implementation of the methods shown in the above figures, this application provides an embodiment of a fabric defect detection device, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0103] This application provides a fabric defect detection device, including:

[0104] Image acquisition module 1 is configured to acquire the fabric image to be detected and the template image;

[0105] Model building module 2 is configured to build and train an improved Yolov5s network to obtain a fabric defect detection model. The improved Yolov5s network includes a backbone feature extraction network, a neck feature fusion network, and a head prediction network. The backbone feature extraction network adopts CSPDarkNet, and the neck feature fusion network includes a feature filtering pyramid network and a path aggregation network that fuses convolutional block attention modules. During training, a multi-class focusing confidence loss function is used instead of the cross-entropy confidence loss function.

[0106] Prediction module 3 is configured to input the fabric image to be detected and the template image into the fabric defect detection model to obtain the fabric detection result.

[0107] The following is for reference. Figure 10It illustrates an electronic device suitable for implementing embodiments of this application (e.g., Figure 1 A schematic diagram of the structure of a computer device 1000 (shown as a server or terminal device). Figure 10 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0108] like Figure 10 As shown, the computer device 1000 includes a central processing unit (CPU) 1001 and a graphics processing unit (GPU) 1002, which can perform various appropriate actions and processes according to programs stored in read-only memory (ROM) 1003 or programs loaded from storage section 1009 into random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the device 1000. The CPU 1001, GPU 1002, ROM 1003, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus 1005.

[0109] The following components are connected to I / O interface 1006: an input section 1007 including a keyboard, mouse, etc.; an output section 1008 including an LCD, speakers, etc.; a storage section 1009 including a hard disk, etc.; and a communication section 1010 including a network interface card, such as a LAN card, modem, etc. The communication section 1010 performs communication processing via a network such as the Internet. A drive 1011 may also be connected to I / O interface 1006 as needed. A removable medium 1012, such as a hard disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 1011 as needed so that computer programs read from it can be installed into storage section 1009 as needed.

[0110] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1010, and / or installed from removable medium 1012. When the computer program is executed by central processing unit (CPU) 1001 and graphics processing unit (GPU) 1002, the functions defined in the methods of this application are performed.

[0111] It should be noted that the computer-readable medium described in this application can be a computer-readable signal medium, a computer-readable medium, or any combination thereof. A computer-readable medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor device, or any combination thereof. More specific examples of a computer-readable medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution device, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than a computer-readable medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution device, apparatus, or apparatus. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0112] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0113] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using dedicated hardware-based means to perform the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0114] The modules described in the embodiments of this application can be implemented in software or hardware. These modules can also be located within a processor.

[0115] In another aspect, this application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: acquire a fabric image to be detected and a template image; construct and train an improved Yolov5s network to obtain a fabric defect detection model. The improved Yolov5s network includes a backbone feature extraction network, a neck feature fusion network, and a head prediction network. The backbone feature extraction network uses CSPDarkNet, and the neck feature fusion network includes a feature filtering pyramid network and a path aggregation network that fuses convolutional block attention modules. During training, a multi-class focusing confidence loss function is used instead of the cross-entropy confidence loss function; and input the fabric image to be detected and the template image into the fabric defect detection model to obtain the fabric detection result.

[0116] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A method for detecting fabric defects, characterized in that, Includes the following steps: Obtain the fabric image to be detected and the template image; An improved Yolov5s network was constructed and trained to obtain a fabric defect detection model. The improved Yolov5s network includes a backbone feature extraction network, a neck feature fusion network, and a head prediction network. The backbone feature extraction network adopts CSPDarkNet. The neck feature fusion network includes a feature filtering pyramid network and a path aggregation network that fuses convolutional block attention modules. During training, a multi-class focusing confidence loss function is used instead of the cross-entropy confidence loss function. The fabric image to be detected and the template image are input into the fabric defect detection model. The CSPDarkNet includes a first unit, a second unit, a third unit, a fourth unit, and a fifth unit connected in sequence. The first unit includes a Focus module and a first CBS module connected in sequence. The second unit includes a second CBS module and a first CSP1_x module connected in sequence. The third unit includes a third CBS module and a second CSP1_x module connected in sequence. The fourth unit includes a fourth CBS module and a third CSP1_x module connected in sequence. The fifth unit includes a fifth CBS module, a fourth CSP1_x module, and an SPPF module connected in sequence. The fabric image is input into the CSPDarkNet, and the third, fourth, and fifth units output a first effective feature map, a second effective feature map, and a third effective feature map, respectively. The feature filtering pyramid network includes an FPN feature fusion network, a redundant feature filtering network, and a template feature extraction network. The first effective feature map, the second effective feature map, the third effective feature map, and the template image are input into the FPN feature fusion network, wherein the third effective feature map is input into the sixth CBS module. The process involves obtaining a third intermediate feature map, concatenating the third intermediate feature map with the feature map obtained by the first Upsample module and the second effective feature map, and then fusing it with the first CSP2_3 module and the seventh CBS module to obtain a second intermediate feature map. The second intermediate feature map is then concatenated with the feature map obtained by the second Upsample module and the first effective feature map, and then fused with the second CSP2_3 module and the eighth CBS module to obtain a first intermediate feature map. The template image is input into the template feature extraction network, and the template image is input into the ninth CBS module to obtain a first template feature map. The first template feature map is input into the Downsample1 module to obtain a second template feature map, and the second template feature map is input into the Downsample2 module to obtain a third template feature map. The first intermediate feature map, the second intermediate feature map, the third intermediate feature map, the first template feature map, the second template feature map, and the third template feature map are input into the redundant feature filtering network. The first intermediate feature map and the first template feature map are concatenated, and then passed through a 1×1 convolutional kernel to obtain a first filtered feature map.The second intermediate feature map and the second template feature map are concatenated, and then passed through a 1×1 convolutional kernel to obtain the second filtered feature map. The third intermediate feature map and the third template feature map are concatenated, and then passed through a 1×1 convolutional kernel to obtain the third filtered feature map. The first, second, and third filtered feature maps are input into the path aggregation network of the fusion convolutional block attention module. The first filtered feature map is input into the first DWConv module to obtain the first fused feature map. The feature map obtained from the first fused feature map through the first CBAM module is concatenated with the second filtered feature map, and then fused through five second DWConv modules to obtain the second fused feature map. The feature map obtained from the second fused feature map through the third DWConv module and the second CBAM module is concatenated with the third filtered feature map, and then fused through five fourth DWConv modules to obtain the third fused feature map. The first, second, and third fused feature maps are input into the head prediction network to obtain the fabric detection result.

2. The fabric defect detection method according to claim 1, characterized in that, In the training process of the improved Yolov5s network, a multi-class focused confidence loss function is used instead of the cross-entropy confidence loss function. The multi-class focused confidence loss function is as follows: ； in, Indicates a label, This is expressed as confidence level. This is represented as the difficulty balance factor, where i represents the i-th type of fabric defect, and N represents the total number of fabric defect types. Represented as the class balance factor: ； in, Represents the balance coefficient. Indicates category as The number of samples.

3. A fabric defect detection device, employing the fabric defect detection method according to claim 1 or 2, characterized in that, include: The image acquisition module is configured to acquire the fabric image to be detected and the template image; The model building module is configured to build and train an improved Yolov5s network to obtain a fabric defect detection model. The improved Yolov5s network includes a backbone feature extraction network, a neck feature fusion network, and a head prediction network. The backbone feature extraction network adopts CSPDarkNet, and the neck feature fusion network includes a feature filtering pyramid network and a path aggregation network that fuses convolutional block attention modules. During training, a multi-class focusing confidence loss function is used instead of the cross-entropy confidence loss function. The prediction module is configured to input the fabric image to be detected and the template image into the fabric defect detection model to obtain the fabric detection result.

4. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-2.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-2.

Images