Cloth defect detection method and system based on multi-scale feature fusion and diffusion pyramid network

Abstract

The invention discloses a cloth defect detection method based on multi-scale feature fusion and a diffusion pyramid network. The method comprises the following steps: S1, constructing a defective cloth image data set; s2, building a deep learning network model which comprises a backbone feature extraction network, a neck feature focusing diffusion pyramid network and a cross-stage multi-scale feature fusion module used in the backbone feature extraction network, the feature focusing module and the detection head structure are used in the neck feature focusing diffusion pyramid network; s3, training the deep learning network model; and S4, detecting a to-be-detected cloth image based on the trained deep learning network model. According to the invention, by introducing the cross-stage multi-scale feature fusion module, detail features of flaws in cloth can be effectively captured, so that the accuracy and robustness of detection are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of cloth detection, and in particular to a cloth defect detection method and system based on multi-scale feature fusion and diffusion pyramid network. Background Art

[0002] The rapid development of the textile industry has put forward more stringent requirements on the standards and efficiency of cloth quality inspection. In the past, manual visual inspection was not only time-consuming and laborious, but also very easy to miss minor defects, which obviously could not match the high-speed and high-quality dual standards of modern textile production. In this context, convolutional neural network (CNN), as a key technology in the field of deep learning, has shown extraordinary potential in multiple computer vision tasks such as image classification, object detection and image segmentation with its powerful image feature extraction capabilities.

[0003] However, despite the advanced technology of convolutional neural networks, when traditional target detectors are applied to the specific scenario of cloth defect detection, especially when faced with the defects of various types, complex shapes and subtle differences in the cloth produced by large circular textile machines, these detectors still seem to be unable to cope with the detection speed and model lightweight, and it is difficult to achieve rapid response while ensuring detection accuracy. Therefore, the development of a new target detection algorithm with high speed, light weight and high precision is of far-reaching significance for improving the efficiency of cloth defect detection and meeting the growing quality control needs of the textile industry. Summary of the invention

[0004] The purpose of the present invention is to provide a cloth defect detection method and system based on multi-scale feature fusion and diffusion pyramid network. The present invention can effectively capture the detailed features of defects in cloth by introducing a cross-stage multi-scale feature fusion module, thereby improving the accuracy and robustness of detection.

[0005] In order to achieve the above purpose, the following technical solutions are adopted:

[0006] A cloth defect detection method based on multi-scale feature fusion and diffusion pyramid network includes the following steps:

[0007] S1: Construct defective cloth image dataset;

[0008] S2: Building a deep learning network model, wherein the deep learning network model includes a backbone feature extraction network, a neck feature focused diffusion pyramid network, a cross-stage multi-scale feature fusion module used in the backbone feature extraction network, and a feature focusing module and a detection head structure used in the neck feature focused diffusion pyramid network;

[0009] S3: training the deep learning network model;

[0010] S4: Detect the cloth image to be detected based on the trained deep learning network model.

[0011] Furthermore, in said S1, the defective cloth image dataset constructed is I=[I1, I2, ...I K ], where K is the total number of elements in the dataset I and the image size is n ch ×h×w, where n c is the image channel, is the image height, and w is the image width. The image is annotated using the open source tool labelImg. The annotated content is the coordinates of the upper left and lower right corners of the defect in the cloth. The annotated information file format is yolo.

[0012] Furthermore, in S2, a size of n is used. ch ×h×w RGB image is taken as input and enters the backbone feature extraction network, and three feature maps of different sizes are output through convolution, cross-stage multi-scale feature fusion module and spatial pooling layer. b1 , I b2 and I b3 , the sizes are n c1 ×h / 8×w / 8,n c2 ×h / 16×w / 16,n c3 ×h / 32×w / 32; the cross-stage multi-scale feature fusion module is used to replace the C2F module in the original yolov8, wherein the number of cross-stage multi-scale feature fusion modules is set to 3, 6, 6, 3.

[0013] Furthermore, in S2, for the cross-stage multi-scale feature fusion module, assuming that the input feature map u∈R C×H×W , then firstly adjust the channel information of the feature map through a Conv() with a convolution kernel of 1, and divide the adjusted feature map into two parts from the channel through the chunk() function; then, one part of the feature map is subjected to feature extraction, multi-scale convolution processing and feature aggregation through a PMSFA feature extraction module, and the output result is concatenated and convolved, and then element-wise Add operation is performed with the other part of the feature map to obtain the final output. The process can be described by the following formula:

[0014] u1=Conv 1×1 (u),

[0015] y0,y1=chunk(u1,2,1),

[0016] y PMSFA =PMSFA(y1),

[0017] y concat=concat(y0,y PMSFA ),

[0018] output = Conv 1×1 (y concat ),

[0019] In the above formula, PMSFA represents the feature extraction module, concat represents the concatenation operation on the channel, and Conv 1×1 (·) indicates a convolution module with a convolution kernel of one.

[0020] Furthermore, for the PMSFA feature extraction module, assuming that the input feature map z∈R C×H×W , the feature map is first processed through a 3×3 convolution layer to generate feature maps conv1_outconv1_out, and is evenly divided in the channel dimension to obtain feature maps conv1_out1conv1_out1 and conv1_out2conv1_out2; then, the feature map conv1_out1conv1_out1 is processed in parallel through two convolution blocks with different structures, wherein the two convolution blocks with different structures are a local feature extraction block and a global feature extraction block, the local feature extraction block is used to compress and restore the number of channels of the feature map through a 5×5 convolution, and the global feature extraction block is used to process the feature map through a 7×7 convolution; then, the outputs of the two convolution blocks are concatenated with the feature map conv1_out2conv1_out2, and the channels are integrated through a 1×1 convolution layer, and the final output is added to the input feature map z by element-wise addition to obtain the final output of the module, which can be described by the following formula:

[0021] conv1_out = Conv 3×3 (z),

[0022] conv1_out1,conv1_out2=conv1_out.chunk(2,dim=1),

[0023] conv2_out = Conv 5×5 (conv1_out1),

[0024] conv2_out1,conv2_out2=conv2_out.chunk(2,dim=1),

[0025] conv3_out = Conv 7×7 (conv2_out1),

[0026] out=cat(conv3_out,conv2_out2,conv1_out2,dim=1),

[0027] out_final=Conv 1×1 (out),

[0028] output = out_final + z,

[0029] In the above formula, Conv n×n It represents a convolution module with a convolution kernel of n, concat represents a concatenation operation on the channel, and chunk(2, dim=1) represents dividing the feature map into two parts in the channel dimension.

[0030] Furthermore, in S2, for the feature focusing module, it is assumed that the input feature map x is a tuple containing three feature maps of different sizes, represented as (x1, x2, x3), and the sizes are x2∈R (C×H×W) , x3∈R (C ×2H×2W) ; First, the feature map x1 is upsampled and then processed through the convolution layer to obtain the feature map x1′; then, the feature map x2 is convolved to obtain the feature map x2′; then, the feature map x3 is convolved and then downsampled to obtain the feature map x3′; then, the processed feature maps x1′, x2′, x3′ are concatenated in the channel dimension, and the concatenated feature maps are subjected to depthwise separable convolution to generate new feature representations; then, the output of the depthwise separable convolution is convolved point by point to integrate the features; finally, the original feature map is added to the processed feature map to obtain the final output. The process can be described by the following formula:

[0031] x1′=Conv 1×1 (upsample(x1)),

[0032] x2′=Conv 1×1 (x2),

[0033] x3′=ADown(Conv(x3)),

[0034] x=concat(x1′,x2′,x3′),

[0035]

[0036] feature=Conv 1×1 (feature),

[0037] output = x + feature,

[0038] In the above formula, upsample(·) represents the upsample upsampling function; Conv n×n (·) indicates a convolution module with a convolution kernel of n; Adown(·) indicates a downsampling module; concat(·) indicates a concat function, and concatenation is performed on dimension 1; sum(·) indicates an element-wise sum operation; DWConv m×m (·) represents a depthwise convolution module with a kernel size of m; identity (·) means that no processing is performed on the original function.

[0039] Furthermore, in S3, the deep learning network model training method specifically includes: first freezing the weights of the backbone feature extraction network part and training N epochs; then unfreezing all weights and training M epochs to further optimize the model, wherein the network training parameters also need to be set: learning rate lr, batchsize, training set validation set division, optimizer and training cycle.

[0040] Furthermore, the S4 specifically includes the following steps:

[0041] The cloth image I to be detected t Input into the deep learning network model, where the image size is n c ××w;

[0042] Through network inference, the output of Yolohead is obtained, which includes three feature maps of different sizes. Then, the classification and regression prediction results are extracted from these feature maps of different sizes, and they are concatenated and dimensionally transformed.

[0043] According to whether the target has confidence, all targets are sorted in descending order, and then the intersection over union (IOU) of the target and other predictions is calculated one by one from high to low, and predictions with IOU greater than the set threshold are eliminated;

[0044] Restore the remaining detection frames to the original image size before network output, and perform non-maximum suppression to remove redundant detection frames to ensure that the number of output detection frames does not exceed the preset maximum number of detections;

[0045] The normalized position coordinates (X, Y, W, H) corresponding to the detection frame's own coordinates (x, y, w, h) are obtained, and drawn in the cloth image to be detected. If the detection frame is output in the image to be detected, it means that the cloth to be detected has defects; if the detection frame is not output, it means that the cloth has no defects.

[0046] A cloth defect detection system based on multi-scale feature fusion and diffusion pyramid network is also provided, which is used to implement the above method, and includes a computer, an industrial camera, and a large circular loom device; the computer is connected to the industrial camera, and the industrial camera is fixed on the large circular loom device; the industrial camera is used to collect images of cloth to be detected and transmit them to the computer; the computer is used to perform defect detection on the images of cloth to be detected through a deep learning network model.

[0047] By adopting the above scheme, the beneficial effects of the present invention are:

[0048] 1) The present invention skillfully combines the essence of the cross-stage multi-scale feature fusion module and the feature-focused diffusion pyramid network, which can deeply mine the rich feature information in the cloth images captured by industrial cameras, and significantly improve the accuracy and robustness of defect detection through the efficient feature extraction and fusion of the feature-focused diffusion pyramid network in the model;

[0049] 2) The feature-focused diffusion pyramid network designed by the present invention realizes cross-channel and cross-space information fusion between feature maps, which can reduce the complexity of the network and improve computational efficiency;

[0050] 3) The present invention has the advantages of being fast, lightweight and accurate, and is suitable for real-time applications and various scenarios of cloth defect detection, and has broad application prospects in industrial production, clothing production and other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A flowchart of the method of the present invention;

[0052] Figure 2 It is a schematic diagram of the overall architecture of the deep learning network model of the present invention;

[0053] Figure 3 Schematic diagram of the structure of the cross-stage multi-scale feature fusion module of the present invention;

[0054] Figure 4 It is a structural schematic diagram of the PMSFA feature extraction module of the present invention;

[0055] Figure 5 It is a schematic diagram of the structure of the neck feature focusing diffusion pyramid network of the present invention;

[0056] Figure 6 It is a structural schematic diagram of the feature focusing module of the present invention;

[0057] Figure 7 It is a structural schematic diagram of the system of the present invention;

[0058] The accompanying drawings illustrate:

[0059] 1. Computer; 2. Industrial camera; 3. Large circular loom device. DETAILED DESCRIPTION

[0060] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0061] Reference Figures 1 to 7 As shown, in the face of the problems of feature extraction difficulties, manual intervention dependence, and poor generalization performance in traditional visual cloth defect detection technology, the present invention proposes a cloth defect detection method based on multi-scale feature fusion and diffusion pyramid network. The present invention first uses a high-precision industrial camera 2 to capture images of the cloth surface, which will serve as the cornerstone of subsequent analysis; then, a series of preprocessing operations are performed on these images, including noise removal, contrast enhancement, affine transformation and other data enhancement methods to improve image quality and ensure the accuracy of subsequent image analysis; finally, a deep learning algorithm is used, especially a convolutional neural network (CNN) based on a multi-scale fusion backbone and a feature focusing network, to conduct in-depth learning and analysis of the preprocessed images, aiming to accurately identify various types of defects. In addition, the method and system also show strong adaptability to various types and colors of cloth by continuously optimizing image processing and machine learning algorithms, thereby significantly improving the accuracy and efficiency of defect detection. Specifically, it includes the following steps:

[0062] S1: Construct a defective cloth image dataset.

[0063] In this step, the industrial camera 2 takes the image of the cloth to be detected, and constructs a cloth defect detection data set I = [I1, I2, ...I K ], where K is the total number of elements in the data set I (in one embodiment, K=3700), and the image size is n ch ×h×w (in one embodiment, the size is 3×640×640), where n c is the image channel, is the image height, and w is the image width. The image is annotated using the open source tool labelImg. The annotated content is the coordinates of the upper left and lower right corners of the defect in the cloth. The annotated information file format is yolo.

[0064] S2: Build a deep learning network model, wherein the deep learning network model includes a backbone feature extraction network backbone, a neck feature focusing diffusion pyramid network FPDN, a cross-stage multi-scale feature fusion module CSP_PMSFA used in the backbone feature extraction network, and a FocusFeature feature focusing module and YoloHead detection head structure used in the neck feature focusing diffusion pyramid network.

[0065] In this step, a size of n is usedch ×h×w RGB image (in one embodiment, the size is 3×640×640) is used as input, enters the backbone feature extraction network, and passes through convolution, cross-stage multi-scale feature fusion module and spatial pooling layer respectively, outputting three feature maps of different sizes I b1 , I b2 and I b3 , the sizes are n c1 ×h / 8×w / 8,n c2 ×h / 16×w / 16,n c3 ×h / 32×w / 32 (in one embodiment, the sizes are 128×80×80, 256×40×40 and 512×20×20); the cross-stage multi-scale feature fusion module is used to replace the C2F module in the original yolov8 to reduce the number of parameters, wherein the number of cross-stage multi-scale feature fusion modules is set to 3, 6, 6, 3.

[0066] The CSPNet architecture in the CSP_PMSFA cross-stage multi-scale feature fusion module reduces redundant computation and improves computational efficiency through cross-stage connections and feature reuse. At the same time, by using convolution kernels of different scales (such as 3x3, 5x5, 7x7, etc.), the module can capture multiple features in the input image, which are different in scale, shape, and texture. Multi-scale feature aggregation helps the model to be more robust and accurate when dealing with complex target detection tasks; and feature map I b1 , I b2 and I b3 The FocusFeature feature focusing module is used to connect the CSP_PMSFA cross-stage multi-scale feature fusion module in the backbone network and the feature transfer of the neck FPDN feature extraction network. This module is used to realize cross-channel and cross-space information fusion between feature maps.

[0067] In the FDPN feature extraction network of the network model, in order to achieve fine adjustment of efficient fusion of feature maps and further reduce the amount of parameters, a FocusFeature feature focusing module is designed, and an FPDN neck network is proposed based on the yolov8 neck network; in the overall structure of the FDPN feature extraction network, a FocusFeature feature focusing module is added on the basis of yolov8 to help features of different scales to be more fine-grained, reduce the complexity of the network, and improve computational efficiency; at the same time, through the customized feature focusing module and feature diffusion mechanism, features of each scale can have detailed contextual information, which is more conducive to the detection and classification of subsequent targets; and the FDPN feature extraction network outputs feature maps I of three different scales n1 , I n2 and I n3The scales are 256×80×80, 512×40×40 and 1024×20×20 respectively. The above design realizes information interaction and fusion across feature maps, thereby improving the target detection performance.

[0068] In one embodiment, for a cross-stage multi-scale feature fusion module, such as Figure 3 As shown, assuming that the input feature map u∈R C×H×W , then firstly adjust the channel information of the feature map through a Conv() with a convolution kernel of 1, and divide the adjusted feature map into two parts from the channel through the chunk() function; then, one part of the feature map is subjected to feature extraction, multi-scale convolution processing and feature aggregation through a PMSFA feature extraction module (the PMSFA feature extraction module performs multi-scale convolution processing on the input feature map, gradually extracts and aggregates features), and the output result is subjected to splicing and convolution operations to further integrate the feature information. Finally, the processed feature map and the other part of the feature map that has not been processed by the PMSFA module are element-wise added, that is, element-by-element addition is performed to obtain the final output (this step helps to integrate the features on different processing paths and improve the performance of the model). The process can be described by the following formula:

[0069] u1=Conv 1×1 (u),

[0070] y0,y1=chunk(u1,2,1),

[0071] y PMSFA =PMSFA(y1),

[0072] y concat =concat(y0,y PMSFA ),

[0073] output = Conv 1×1 (y concat ),

[0074] In the above formula, PMSFA represents the feature extraction module, concat represents the concatenation operation on the channel, and Conv 1×1 (·) indicates a convolution module with a convolution kernel of one.

[0075] In addition, for the PMSFA feature extraction module, such as Figure 4 As shown, assuming that the input feature map z∈R C×H×W, the feature map is first processed through a 3×3 convolution layer to generate feature maps conv1_outconv1_out, and then evenly divided in the channel dimension to obtain feature maps conv1_out1conv1_out1 and conv1_out2conv1_out2; then, the feature map conv1_out1conv1_out1 is processed in parallel through two convolution blocks with different structures, such as Figure 4 As shown in the figure, the two convolution blocks with different structures are respectively a local feature extraction block and a global feature extraction block. The local feature extraction block is used to compress and restore the number of channels of the feature map through a 5×5 convolution to select important information, re-extract lost features and reduce overfitting, etc. The global feature extraction block is used to process the feature map through a 7×7 convolution, aiming to extract deeper features.

[0076] Finally, the outputs of the above two convolution blocks are concatenated with the feature map conv1_out2conv1_out2, and the channels are integrated through a 1×1 convolution layer. The final output is added to the input feature map z by element-wise addition to obtain the final output of the module. The process can be described by the following formula:

[0077] conv1_out = Conv 3×3 (z),

[0078] conv1_out1,conv1_out2=conv1_out.chunk(2,dim=1),

[0079] conv2_out = Conv 5×5 (conv1_out1),

[0080] conv2_out1,conv2_out2=conv2_out.chunk(2,dim=1),

[0081] conv3_out = Conv 7×7 (conv2_out1),

[0082] out=cat(conv3_out,conv2_out2,conv1_out2,dim=1),

[0083] out_final=Conv 1×1 (out),

[0084] output = out_final + z,

[0085] In the above formula, Conv n×nIt represents a convolution module with a convolution kernel of n, concat represents a concatenation operation on the channel, and chunk(2, dim=1) represents dividing the feature map into two parts in dimension one (i.e., on the channel).

[0086] In addition, the FDPN feature extraction network is as follows Figure 5 As shown in the figure, first, the FocusFeature feature focusing module focuses on the feature maps from the spatial pooling layer SPPF and the CSP_PMSFA cross-stage multi-scale feature fusion module to generate a new feature map; then, the feature map undergoes convolution operations and feature splicing to enhance feature expression; then, the resolution of the feature map is increased again through upsampling, and it is spliced ​​with the feature maps of other layers, and continues to be processed through the C2F module; finally, after multiple feature fusion steps, the processed features are input into the Detect module to output the final target bounding box and category probability. The overall process improves the network's detection ability for targets of different scales through the extraction and fusion of multi-level features.

[0087] Among them, for the feature focusing module, such as Figure 6 As shown, assume that the input feature map x is a tuple containing three feature maps of different sizes, represented as (x1, x2, x3), with sizes respectively x2∈R (C×H×W) , x3∈R (C ×2H×2W) ; First, the feature map x1 is upsampled and then processed through the convolution layer to obtain the feature map x1′; then, the feature map x2 is convolved to obtain the feature map x2′; then, the feature map x3 is convolved and then downsampled to obtain the feature map x3′; then, the processed feature maps x1′, x2′, x3′ are concatenated in the channel dimension, and the concatenated feature maps are subjected to depthwise separable convolution to generate new feature representations; then, the output of the depthwise separable convolution is convolved point by point to integrate the features; finally, the original feature map is added to the processed feature map to obtain the final output. The process can be described by the following formula:

[0088] x1′=Conv 1×1 (upsample(x1)),

[0089] x2′=Conv 1×1 (x2),

[0090] x3′=ADown(Conv(x3)),

[0091] x=concat(x1′,x2′,x3′),

[0092]

[0093] feature=Conv 1×1 (feature),

[0094] output = x + feature,

[0095] In the above formula, upsample(·) represents the upsample upsampling function; Conv n×n (·) indicates a convolution module with a convolution kernel of n; Adown(·) indicates a downsampling module; concat(·) indicates a concat function, and concatenation is performed on dimension 1; sum(·) indicates an element-wise sum operation; DWConv m×m (·) represents a depthwise convolution module with a kernel size of m; identity (·) means that no processing is performed on the original function.

[0096] S3: Training the deep learning network model.

[0097] In this step, the training method specifically includes: first freezing the weights of the backbone feature extraction network part and training N epochs (in one embodiment, N=20); then unfreezing all weights and training M epochs (in one embodiment, M=300) to further optimize the model, wherein the network training parameters also need to be set: learning rate lr, batchsize, training set validation set division, optimizer and training cycle. In one embodiment, the learning rate lr=0.001, batchsize=4, the training set validation set division is 9:1, the optimizer uses SGD and the training cycle is 300 rounds.

[0098] S4: Detect the cloth image to be detected based on the trained deep learning network model.

[0099] In this step, you need to use the trained network for prediction, input the cloth image to be detected, and output the cloth defect prediction target box. The specific steps include:

[0100] 1) The cloth image I to be detected t Input into the deep learning network model, where the image size is n c ××w (in one embodiment, the size is 3×640×640);

[0101] 2) Through network inference, the output of the Yolohead detection head is obtained, which includes three feature maps of different sizes (in one embodiment, the sizes are 80×80, 40×40 and 20×20 respectively), and then the classification and regression prediction results are extracted from these feature maps of different sizes, and they are spliced ​​and dimensionally transformed. For the convenience of processing, the original channel dimension is replaced to the end, and the shapes of the category prediction branch and the bbox prediction branch are (1,8400,80) and (1,8400,4) respectively;

[0102] 3) According to whether the target has confidence (in one embodiment, confidence conf = 0.001), all targets are arranged in descending order, and then the intersection over union (IOU) of the target and other predictions is calculated one by one from high to low, and predictions whose IOU is greater than a set threshold (in one embodiment, the threshold iou = 0.6) are eliminated;

[0103] 4) According to the previous preprocessing process, the remaining detection frames are restored to the original image size before the network output, and non-maximum suppression is performed to remove redundant detection frames to ensure that the number of output detection frames does not exceed the preset maximum detection number (in one embodiment, the maximum detection number max_per_img=300);

[0104] 5) Obtain the normalized position coordinates (X, Y, W, H) corresponding to the detection frame's own coordinates (x, y, w, h), and draw them in the cloth image to be detected. If the detection frame is output in the image to be detected, it means that the cloth to be detected has defects; if the detection frame is not output, it means that the cloth has no defects.

[0105] In addition, a cloth defect detection system based on multi-scale feature fusion and diffusion pyramid network is provided, which is used to implement the above method, including a computer 1, an industrial camera 2, and a large circular loom device 3; the computer 1 is connected to the industrial camera 2, and the industrial camera 2 is fixed on the large circular loom device 3; the industrial camera 2 is used to collect images of cloth to be inspected and transmit them to the computer 1; the computer 1 is used to perform defect detection on the image of the cloth to be inspected through a deep learning network model. The specific detection process is as described above and will not be repeated here.

[0106] This system cleverly combines the essence of the cross-stage multi-scale feature fusion module and the feature-focused diffusion pyramid network. By deeply mining the rich feature information in the cloth images captured by the industrial camera 2 and through the efficient feature extraction and fusion of the feature-focused diffusion pyramid network in the model, it significantly improves the accuracy and robustness of defect detection, bringing great improvements to defect detection in the textile industry.

[0107] At the same time, this system can effectively utilize the feature information in the cloth image collected by the industrial camera 2 to improve the detection accuracy and robustness of defective targets. In terms of optimizing the network structure, this system has deeply customized and optimized the YOLOv8 network, especially in the two core areas of the backbone network module and the neck structure. Breakthrough transformation has been implemented. In the backbone part, this system is based on the CSPNet (Cross Stage Partial Networks) idea, and designs and introduces a more efficient and lightweight cross-stage multi-scale feature fusion module. In the transformation of the neck structure, this system cleverly integrates multi-scale feature information, and through a carefully designed feature focusing strategy, the model's ability to capture and identify the detailed features of cloth defects is enhanced. The design of this module not only effectively simplifies the network architecture, significantly reduces the computational complexity, and provides strong hardware efficiency support for real-time detection, but also enhances the circulation and reuse of feature information through cross-stage partial connections, further improving the performance of the model.

[0108] With the continuous development and widespread application of convolutional neural network visual inspection technology, the defect detection algorithm constructed by this system combined with deep learning can be widely used in the textile industry, especially in areas with high requirements for product quality and production efficiency. It not only improves the efficiency and accuracy of inspection, but also helps to reduce production costs and enhance the market competitiveness of products.

[0109] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A cloth defect detection method based on multi-scale feature fusion and diffusion pyramid network, characterized in that: The steps include: S1: Construct defective cloth image dataset; S2: Building a deep learning network model, wherein the deep learning network model includes a backbone feature extraction network, a neck feature focused diffusion pyramid network, a cross-stage multi-scale feature fusion module used in the backbone feature extraction network, and a feature focusing module and a detection head structure used in the neck feature focused diffusion pyramid network; S3: training the deep learning network model; S4: Detect the cloth image to be detected based on the trained deep learning network model.

2. The cloth defect detection method based on multi-scale feature fusion and diffusion pyramid network according to claim 1 is characterized in that: In S1, the defective cloth image dataset constructed is I=[I1, I2, ...I K ], where K is the total number of elements in the dataset I and the image size is n ch ×h×w, where n c is the image channel, is the image height, and w is the image width. The image is annotated using the open source tool labelImg. The annotated content is the coordinates of the upper left and lower right corners of the defect in the cloth. The annotated information file format is yolo.

3. The cloth defect detection method based on multi-scale feature fusion and diffusion pyramid network according to claim 2 is characterized in that: In S2, a size of n is used. ch ×h×w RGB image is taken as input and enters the backbone feature extraction network, and three feature maps of different sizes are output through convolution, cross-stage multi-scale feature fusion module and spatial pooling layer. b1 , I b2 and I b3 , the sizes are n c1 ×h / 8×w / 8,n c2 ×h / 16×w / 16,n c3 ×h / 32×w / 32; the cross-stage multi-scale feature fusion module is used to replace the C2F module in the original yolov8, wherein the number of cross-stage multi-scale feature fusion modules is set to 3, 6, 6, 3.

4. The cloth defect detection method based on multi-scale feature fusion and diffusion pyramid network according to claim 3 is characterized in that: In S2, for the cross-stage multi-scale feature fusion module, assuming that the input feature map u∈R C ×H×W , then firstly adjust the channel information of the feature map through a Conv() with a convolution kernel of 1, and divide the adjusted feature map into two parts from the channel through the chunk() function; then, one part of the feature map is subjected to feature extraction, multi-scale convolution processing and feature aggregation through a PMSFA feature extraction module, and the output result is concatenated and convolved, and then element-wise Add operation is performed with the other part of the feature map to obtain the final output. The process can be described by the following formula: u1=Conv 1×1 (u), y0, y1=chunk(u1, 2, 1), y PMSFA =PMSFA(y1), and concat =Concat(y0,y PMSFA ), output=Conv 1×1 (y concat ), In the above formula, PMSFA represents the feature extraction module, concat represents the concatenation operation on the channel, and Conv 1×1 (·) indicates a convolution module with a convolution kernel of one.

5. The cloth defect detection method based on multi-scale feature fusion and diffusion pyramid network according to claim 4 is characterized in that: For the PMSFA feature extraction module, assuming that the input feature map z∈R C×H×W , the feature map is first processed through a 3×3 convolution layer to generate feature maps conv1_outconv1_out, and is evenly divided in the channel dimension to obtain feature maps conv1_out1conv1_out1 and conv1_out2conv1_out2; then, the feature map conv1_out1conv1_out1 is processed in parallel through two convolution blocks with different structures, wherein the two convolution blocks with different structures are a local feature extraction block and a global feature extraction block, the local feature extraction block is used to compress and restore the number of channels of the feature map through a 5×5 convolution, and the global feature extraction block is used to process the feature map through a 7×7 convolution; then, the outputs of the two convolution blocks are concatenated with the feature map conv1_out2conv1_out2, and the channels are integrated through a 1×1 convolution layer, and the final output is added to the input feature map z by element-wise addition to obtain the final output of the module, which can be described by the following formula: conv1_out=Conv 3×3 (z), conv1_out1, conv1_out2=conv1_out.chunk(2, dim=1), conv2_out=Conv 5×5 (conv1_out1), conv2_out1, conv2_out2=conv2_out.chunk(2, dim=1), conv3_out=Conv 7×7 (conv2_out1), out=cat(conv3_out, conv2_out2, conv1_out2, dim=1), out_final=Conv 1×1 (out), output = out_final + z, In the above formula, Conv n×n It represents a convolution module with a convolution kernel of n, concat represents a concatenation operation on the channel, and chunk(2, dim=1) represents dividing the feature map into two parts in the channel dimension.

6. The cloth defect detection method based on multi-scale feature fusion and diffusion pyramid network according to claim 5 is characterized in that: In S2, for the feature focusing module, it is assumed that the input feature map x is a tuple containing three feature maps of different sizes, represented as (x1, x2, x3), and the sizes are x2∈R (C×H×W) , x3∈R (C ×2H×2W ); first, the feature map x1 is upsampled, and then processed through the convolution layer to obtain the feature map x1′; then, the feature map x2 is convolved to obtain the feature map x2′; then, the feature map x3 is convolved and then downsampled to obtain the feature map x3′; then, the processed feature maps x1′, x2′, x3′ are concatenated in the channel dimension, and the concatenated feature maps are subjected to depthwise separable convolution to generate new feature representations; then, the output of the depthwise separable convolution is convolved point by point to integrate the features; finally, the original feature map is added to the processed feature map to obtain the final output. The process can be described by the following formula: x1′=Conv 1×1 (upsample(x1)), x2′=Conv 1×1 (x2), x3′=ADown(Conv(x3)), x=concat(x1′,x2′,x3′), feature=Conv 1×1 (feature), output = x + feature, In the above formula, upsample(·) represents the upsample upsampling function; Conv n×n (·) indicates a convolution module with a convolution kernel of n; Adown(·) indicates a downsampling module; concat(·) indicates a concat function, and concatenation is performed on dimension 1; sum(·) indicates an element-wise sum operation; DWConv m×m (·) represents a depthwise convolution module with a kernel size of m; identity (·) means that no processing is performed on the original function.

7. The cloth defect detection method based on multi-scale feature fusion and diffusion pyramid network according to claim 1 is characterized in that: In S3, the deep learning network model training method specifically includes: first freezing the weights of the backbone feature extraction network part and training N epochs; then unfreezing all weights and training M epochs to further optimize the model, wherein the network training parameters also need to be set: learning rate lr, batchsize, training set validation set division, optimizer and training cycle.

8. The cloth defect detection method based on multi-scale feature fusion and diffusion pyramid network according to claim 1, characterized in that: The S4 specifically includes the following steps: The cloth image I to be detected t Input into the deep learning network model, where the image size is n c ××w; Through network inference, the output of Yolohead is obtained, which includes three feature maps of different sizes. Then, the classification and regression prediction results are extracted from these feature maps of different sizes, and they are concatenated and dimensionally transformed. According to whether the target has confidence, all targets are sorted in descending order, and then the intersection over union (IOU) of the target and other predictions is calculated one by one from high to low, and predictions with IOU greater than the set threshold are eliminated; Restore the remaining detection frames to the original image size before network output, and perform non-maximum suppression to remove redundant detection frames to ensure that the number of output detection frames does not exceed the preset maximum number of detections; The normalized position coordinates (x, Y, W, H) corresponding to the detection frame's own coordinates (x, y, w, h) are obtained, and drawn in the cloth image to be detected. If the detection frame is output in the image to be detected, it means that the cloth to be detected has defects; if the detection frame is not output, it means that the cloth has no defects.

9. A cloth defect detection system based on multi-scale feature fusion and diffusion pyramid network, used to implement the method according to any one of claims 1 to 8, characterized in that: It includes a computer, an industrial camera, and a large circular loom device; the computer is connected to the industrial camera, and the industrial camera is fixed on the large circular loom device; the industrial camera is used to collect images of cloth to be inspected and transmit them to the computer; the computer is used to perform defect detection on the images of cloth to be inspected through a deep learning network model.

Images