Method, computer device and medium for identifying small and complex defects in cloth

Abstract

The present invention belongs to the field of image processing technology and discloses a method, computer equipment, and medium for identifying small and complex defects in cloth. The present invention builds a model for identifying small and complex defects in cloth based on the improved YOLOv9 model, that is, an improved serpentine dynamic convolution module C‑DSC is introduced into the backbone network of the main branch structure to adaptively and dynamically extract defect points to improve sensitivity to target shapes and boundaries and enhance defect detection for complex and slender styles; secondly, the AIFI module in the RT‑DETR model is used to replace the SPPELAN module in the YOLOv9 model to enhance the interaction between features within the same scale, reduce processing delays, and increase speed; finally, a multi-scale feedforward network MSFN is added before the detection head network to aggregate multi-scale features and enhance nonlinear information conversion. The present invention improves the recognition accuracy and efficiency of small and complex defects in cloth defect images.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to a method, computer equipment and medium for identifying small and complex defects in cloth, which is particularly suitable for identifying small and complex defects in cloth. Background Art

[0002] For cloth defect detection, detection mainly relies on deep learning networks based on machine vision. Since cloth defects are of many types, with different shapes and sizes, it is particularly important to use deep learning-based methods for detection. The network is trained with a large amount of sample data to make it generalizable and robust for detecting various types of cloth. Therefore, during detection, the designed network needs to be able to handle samples of different colors, sizes, and specifications. Therefore, the network's generalization, accuracy, efficiency, and stability are particularly important. Currently, due to the large nonlinear fitting function of convolutional neural networks, it has greatly improved the detection efficiency when detecting defects. For example:

[0003] Zhao et al. proposed a VGG16 network-based detection method for linen and image fabric defect detection. The method can detect six defects, including holes and stains. However, the large number of parameters used in training and the long training time make it difficult to detect defects in a timely manner. Pandia et al. designed a deep convolutional neural network using fabric color as a key feature. The method learned from various defect datasets during the training phase and utilized a learned feature for defect classification during the testing phase. However, the method only classified defects with obvious features and failed to detect small and complex defects, thus exhibiting limitations. Zhu Junling et al. used AR model spectral estimation to model fabric texture images, generating a variance sequence. The defect type and location were determined by calculating the correlation coefficient between the spectral estimates of the defect image and the normal texture image. However, the modeling process is prone to problems such as false spectral peaks and spectral line splitting. Furthermore, noise can degrade the spectral estimate, resulting in reduced detection accuracy and significant deviations in the detection results. An Meng et al. optimized the convolutional neural network based on the Faster RCNN network model, introduced feature pyramid network models of different scales into the region proposing network (RPN) to increase detailed shallow features, and reduced the intra-class spacing and increased the inter-class spacing by regularizing the Softmax classifier. The network parameters were continuously updated to improve the network convergence ability. Although high-quality candidate regions can be generated, in some cases, the recognition effect is poor for regions with a large aspect ratio, and gradient explosion is prone to occur, and its performance may be limited under certain conditions.

[0004] Object recognition based on YOLO (You Only Look Once) has become mainstream. Detectors typically consist of two components: a backbone network (the base network) that extracts features and is typically pre-trained on the ImageNet dataset. A head (the head) predicts object categories and bounding boxes. The neck (the neck) is constructed between the backbone and the head to aggregate different feature maps. However, the depth of the network directly affects the accuracy and efficiency of object detection. A deeper network structure has greater nonlinear expression capabilities, allowing it to learn more complex transformations and fit more complex feature inputs. Shallower networks have the opposite effect. Researchers are constantly optimizing the network model and adjusting parameters to reduce computational overhead while maintaining high accuracy and improving efficiency. Currently, other versions of YOLO suffer from the problem that a significant amount of information is lost when the input data undergoes layer-by-layer feature extraction and spatial transformation. This information loss can lead to biased gradient flows, which are then used to update the model, potentially causing the deep network to establish incorrect connections.

[0005] The detection method based on YOLOv9 has great development potential in use. It proposes the concept of Programmable Gradient Information (PGI) to cope with the various changes required for deep networks to achieve multiple goals. At the same time, YOLOv9 also designs a new lightweight network architecture based on gradient path planning - Generalized Efficient Layer Aggregation Network (GELAN).

[0006] However, when applying YOLOv9 to identify small and complex defects in cloth, the following problems exist:

[0007] 1. High resource consumption: Although YOLOv9 has improved in accuracy and efficiency, it is still a complex deep learning model that requires large resources and a long time for training and inference.

[0008] 2. Model overfitting and underfitting affect recognition performance: Like other deep learning models, YOLOv9 also faces the problems of overfitting and underfitting. Overfitting means that the model performs well on the training set but poorly on the test set, while underfitting means that the model performs poorly on both the training set and the test set. This may cause the detection effect and efficiency of small defects to be affected due to fitting problems.

[0009] 3. Insufficient generalization affects defect recognition accuracy: Although YOLOv9 has achieved relatively good results in testing across multiple datasets, its generalization capabilities may still be limited. Specifically, if the amount of data used is too small, the quality is too poor, the specifications are non-standard, or the label features are not distinct enough, it can cause significant errors in training results. If the dataset is unbalanced, meaning that the number of samples in some categories far exceeds that of others, the model may favor the more numerous categories and ignore the less numerous ones. This can affect the accuracy of identifying small and complex defects, potentially leading to missed or false detections.

[0010] 4. Poor detection of targets with large aspect ratios: During the cloth production process, assembly line machines operate continuously, which may cause defects to be small, narrow, or densely distributed. YOLOv9 divides the entire image into large grids during prediction. Therefore, the detection of dense targets may not be ideal, resulting in missed detections or overlapping detection frames. Summary of the Invention

[0011] The purpose of the present invention is to propose a method for identifying small and complex defects in cloth. The method builds a model for identifying small and complex defects in cloth based on the improved YOLOv9 to improve the recognition accuracy and efficiency of small and complex defects.

[0012] In order to achieve the above-mentioned purpose, the present invention adopts the following technical solutions:

[0013] A method for identifying small and complex defects in cloth, comprising the following steps:

[0014] Step 1. First, preprocess the collected cloth defect images and form a dataset for model training;

[0015] Step 2. Build a model for identifying small and complex fabric defects based on the improved YOLOv9 model. The main branch structure of the improved YOLOv9 model is improved on the basis of the original YOLOv9 model as follows:

[0016] An improved serpentine dynamic convolution module (C-DSC) is introduced into the backbone network of the main branch structure to adaptively and dynamically extract defect points, thereby improving sensitivity to complex defect shapes and boundaries and enhancing the ability to detect slender and complex defects. The AIFI module in the RT-DETR model is used to replace the SPPELAN module in the YOLOv9 model to enhance the model's ability to process local and global information and strengthen the interaction between features at the same scale to reduce processing delays and increase speed. A multi-scale feedforward network (MSFN) is added before the head layer to aggregate multi-scale features and enhance nonlinear information conversion.

[0017] Step 3. Use the data set from step 1 to train a cloth small and complex defect recognition model based on the improved Yolov9 model. During the training process of the cloth small and complex defect recognition model, an auxiliary branch is introduced to assist in training.

[0018] Step 4. After the cloth defect image to be identified has been preprocessed in step 1, it is input into the cloth small and complex defect recognition model trained in step 3. The final output is the target recognition result, including the position and category of the bounding box, so as to identify the small and complex cloth defects in the cloth defect image.

[0019] In addition, based on the method for identifying small and complex defects in cloth, the present invention also proposes a computer device, which includes a memory and one or more processors. Executable code is stored in the memory.

[0020] When the processor executes the executable code, it is used to implement the above-mentioned method for identifying small and complex defects in cloth.

[0021] In addition, based on the method for identifying small and complex defects in cloth, the present invention also proposes a computer-readable storage medium on which a program is stored. When the program is executed by a processor, it is used to implement the method for identifying small and complex defects in cloth.

[0022] The present invention has the following advantages:

[0023] As described above, the present invention describes a method for identifying small and complex defects in cloth based on an improved YOLOv9 model. This method addresses the problem of detecting small and complex defects in cloth. First, a module C-DSC, which combines the Dynamic Snake Convolution (DSC) and the Convolutional Block Attention Module (CBAM) introduced into the YOLOv9 backbone network (Backbone), is used to improve sensitivity to target shapes and boundaries, thereby enhancing its ability to detect complex defects. The Dynamic Snake Convolution imitates the shape and characteristics of a snake and is mainly used to process data structures with slender, winding, and irregular shapes. It can not only adaptively focus on the slender curve local features of tubular structures, but also enhance the perception of geometric structures and prevent the occurrence of loops during the iterative process. The addition of the CBAM attention module, by combining channel attention and spatial attention, can adaptively learn the correlation between different features, thereby improving the model's feature extraction and generalization capabilities without increasing the network's computational load. The Attention-based Intra-scale Feature Interaction (AIFI) module from the transformer-based Real-Time Detection Transformer (RT-DETR) model is then used to replace the Spatial Pyramid Pooling with Enhanced Local Attention Network (SPPELAN) in YOLOv9. Similar to the Transformer Encoder, AIFI enhances the model's ability to process local and global information and reduces processing latency. By optimizing attention operations between multi-scale features, it reduces computational resource consumption, enabling the model to demonstrate greater flexibility and operational efficiency when dealing with complex scenarios. Finally, a Multi-Scale Feedforward Network (MSFN) is introduced before the Head layer. The MSFN design improves nonlinear feature transformations and strengthens nonlinear information conversion, enabling the model to better handle complex nonlinear problems. It also enhances its feature extraction capabilities by extracting features at different scales, making it particularly useful for enhancing the extraction of small and complex defects. Through the above improvements, the cloth small and complex defect recognition model based on the improved YOLOv9 model built by the present invention has greatly improved the recognition accuracy and efficiency of the present invention for small and complex defects in cloth defect images. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a flow chart of a method for identifying small and complex defects in cloth according to an embodiment of the present invention;

[0025] Figure 2 A network structure diagram of a model constructed in an embodiment of the present invention;

[0026] Figure 3 This is a diagram of the main branch structure in an embodiment of the present invention;

[0027] Figure 4 This is a flow chart of the auxiliary branch in an embodiment of the present invention;

[0028] Figure 5 This is a structural diagram of the auxiliary branch in an embodiment of the present invention;

[0029] Figure 6 This is a network structure diagram of a C-DSC in an embodiment of the present invention;

[0030] Figure 7 This is a flowchart of multi-view feature fusion in an embodiment of the present invention;

[0031] Figure 8 This is the CBAM network architecture in the embodiment of the present invention;

[0032] Figure 9 This is a flow chart of the channel attention mechanism in an embodiment of the present invention;

[0033] Figure 10 This is a flow chart of the spatial attention mechanism in an embodiment of the present invention;

[0034] Figure 11 This is a network structure diagram of AIFI in an embodiment of the present invention;

[0035] Figure 12 FIG. 4 is a network structure diagram of the MSFN in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0037] Example 1

[0038] This embodiment describes a method for identifying small, complex defects in fabric. This method builds a small, complex defect recognition model based on an improved YOLOv9 model to improve the accuracy and efficiency of identifying small, complex defects in fabric defect images. YOLOv9 incorporates an innovative backbone network architecture that significantly enhances feature extraction while maintaining computational efficiency. By cleverly increasing the number of network layers and introducing a more complex connection mechanism, YOLOv9 is able to more deeply capture contextual information in images, enabling more precise object recognition and localization. This design enables YOLOv9 to significantly improve its object detection accuracy while maintaining high efficiency. The PGI concept introduced in YOLOv9 addresses the issue of information loss in data transmission within deep neural networks. Through PGI, the model can efficiently transmit gradient information while maintaining the integrity of input data, improving learning efficiency and model performance. A lightweight network architecture, GELAN, is also designed, which optimizes network parameter utilization and computational efficiency based on gradient path planning. Through an improved network structure, YOLOv9 achieves higher accuracy and faster processing speed while remaining lightweight.

[0039] Based on the original YOLOv9 model, the present invention first introduces an improved serpentine dynamic convolution module C-DSC to adaptively and dynamically extract defect points, so as to improve the sensitivity to complex defect shapes and boundaries and enhance the ability to detect slender and complex defects. Then, the AIFI module in the RT-DETR model is used to replace the SPPELAN in YOLOv9 to enhance the model's ability to process local and global information and enhance the interaction between features at the same scale to reduce processing delays and increase speed. Finally, MSFN is added before the Head layer to aggregate multi-scale features and enhance nonlinear information conversion. Through the above improvements, not only the problem of small and irregular defects being difficult to detect is solved, but also the operating speed of the system is improved, and the stability during the detection process is enhanced, avoiding false detections caused by a large number of defect types and complex background patterns.

[0040] Specifically, the input feature map undergoes feature extraction in the convolutional and CR modules of the Backbone layer, and then enters the C-DSC module. This module performs changes in the x- and y-axis directions, marking the target point locations through multiple iterations. This data is then passed to the CBAM attention mechanism module. This operation not only improves the detection of small and complex defects, but also does not excessively increase computational overhead. This branch then passes to the AIFI module, which processes the three flattened matrices to enhance the model's expressive power. After feature extraction, it passes through the MSFN network, where it undergoes another convolution operation to integrate feature maps of different dimensions, improving the model's robustness and generalization capabilities in different scenarios and conditions. In the auxiliary branch, the multi-scale feature maps extracted from the backbone network are processed and combined with the backbone network's feature maps to form a richer feature representation, improving overall detection and recognition performance.

[0041] like Figure 1 As shown, the method for identifying small and complex defects in cloth in this embodiment includes the following steps:

[0042] Step 1. First, preprocess the collected cloth defect images and form a data set for model training.

[0043] First, the collected cloth defect images are preprocessed, the input images are adjusted to a fixed size (e.g., 640×640×3), and the dataset is enhanced (such as random cropping, flipping, rotation, etc.) to expand the dataset.

[0044] After the dataset was expanded, the defect areas of the preprocessed images were manually annotated according to the GB / T 24250-2009 "Terminology for the Description of Defects in Woven Fabrics", which were defined as including 12 common major categories of defects, including holes, stains, three-wire threads, knots, hanging wefts, tangles, fuzz, hanging warps, color difference, wear marks, double dimensions, and defective weft yarns.

[0045] The labeled cloth defect image dataset is divided into a training set, a validation set, and a test set for model training and testing. The training set, validation set, and test set are divided in a ratio of 6:2:2, for example.

[0046] Step 2. Build a cloth small and complex defect recognition model based on the improved YOLOv9 model, such as Figure 2 As shown in FIG, the improved YOLOv9 model includes two parts: a main branch and an auxiliary branch.

[0047] The main branch consists of the backbone network, the neck network, and the detection head network. The auxiliary branch is used to enhance training during dataset training and can be deleted during the model inference phase.

[0048] The improved YOLOv9 model main branch structure is based on the original YOLOv9 model. An improved serpentine dynamic convolution module C-DSC is introduced into the backbone network of the main branch structure to adaptively and dynamically extract defect points, thereby improving sensitivity to target shapes and boundaries and enhancing defect detection for complex and slender patterns. Secondly, the AIFI module in the RT-DETR model is used to replace the SPPELAN module in the YOLOv9 model to fuse features of the same scale to reduce processing delays and increase speed. Finally, a multi-scale feedforward network MSFN is added before the detection head network to aggregate multi-scale features and enhance nonlinear information conversion. Through the above improvements, not only the problem of small and irregular defects being difficult to detect is solved, but also the system operation speed is improved, the stability of the detection process is enhanced, and false detections caused by a large number of defect types and complex background patterns are reduced.

[0049] like Figure 3 As shown, the main branch structure includes a trunk network, a neck network and a detection head network.

[0050] In the backbone network, the CR module is a combination of the convolution module and the RepNCSPELAN4 module. It extracts features from the input image through multiple downsampling convolutions to achieve full feature extraction.

[0051] Taking a 3×640×640 image as input, the image first undergoes a convolution with 64 kernels and a stride of 2 to initially extract fabric features. This convolution operation reduces the size of the feature map and increases the number of channels, resulting in an output feature map of 64×320×320. The convolution module combines a convolutional layer, batch normalization, and a leaky ReLU activation function. The RepNCSPERLAN4 layer, combined with the convolutional block, is a residual module with a convolutional layer and skip connections. The image then passes through a CR module, which includes a convolution with 256 output channels and a stride of 2. Downsampling is then performed, resulting in a 256×160×160 feature map. This feature map is then passed through the RepNCSPERLAN4 module to improve feature extraction. The convolution operation through multiple CR modules repeats the downsampling and feature extraction process at different resolutions, extracting higher-level features. By repeating these steps, the detection range of the sample is continuously refined. Therefore, the sample image size decreases in length and width, ultimately becoming resolvable to 80*80*Y, 40*40*Y, and 20*20*Y, where Y represents the number of channels, which increases continuously. The RepNCSPELAN4 module combines the reparameterization technology of RepVGG, the branching structure of CSPNet, and the multi-layer convolution method of ELAN to reduce computational complexity and memory usage, further improving inference efficiency. Through these multiple convolutional sampling operations, feature maps of varying sizes, such as 512×80×80, 512×40×40, and 512×20×20, are ultimately generated. These feature maps of varying sizes are further fed into the neck network for multi-scale feature concatenation and upsampling. By fusing features of different scales, the model becomes more robust in scenes with complex backgrounds and diverse objects. Small objects may be more easily captured in lower-level features, while larger objects may be more prominent in higher-level features. Stitching these features together comprehensively leverages information from different levels, improving overall detection accuracy and creating richer, more expressive feature maps. The detection head network primarily maps the multi-scale features output by the neck network to specific object detection tasks. It performs object classification and localization through a series of convolution operations, generating object category probabilities, location coordinates, and confidence scores. It also uses methods such as non-maximum suppression (NMS) to remove overlapping detection boxes and retain the boxes with the highest confidence. Finally, the head layer outputs formatted detection results, ensuring the model can efficiently and accurately complete object detection tasks in a variety of application scenarios.

[0052] Specifically, the backbone network includes a convolution module, a CR module and an improved snake-shaped dynamic convolution module C-DSC; there are four CR modules, which are defined as the first CR module, the second CR module, the third CR module and the fourth CR module respectively.

[0053] Each CR module is composed of a convolution module and a RepNCSPELAN4 module. It extracts features from the input image through multiple downsampling convolutions, thereby achieving full feature extraction.

[0054] The processing flow of the backbone network is as follows: the signal first passes through a convolution module for preliminary feature extraction and increases the number of sample channels, then passes through the first, second, third, and fourth CR modules in sequence for feature extraction, and finally passes through the improved snake-shaped dynamic convolution module C-DSC for adaptive dynamic extraction of defect points.

[0055] The neck network includes AIFI module, upsampling module, splicing module, RepNCSPELAN4 module and CR module; there are two upsampling modules, which are defined as the first upsampling module and the second upsampling module respectively.

[0056] There are four feature splicing modules, which are defined as the first, second, third and fourth feature splicing modules respectively.

[0057] There are two CR modules, which are defined as the fifth and sixth CR modules respectively.

[0058] There are two RepNCSPELAN4 modules, which are defined as the first and second RepNCSPELAN4 modules respectively.

[0059] The processing flow of the neck network is as follows:

[0060] The output of the backbone network enters the AIFI module to enhance the interaction ability of features within the same scale and between scales, and reduce the attention operation between features of multiple scales; the output of the AIFI module is upsampled through the first upsampling module.

[0061] The upsampling result and the output of the fourth CR module enter the first feature splicing module for splicing, and the spliced ​​features pass through the first RepNCSPELAN4 module and the second upsampling module in sequence.

[0062] The output of the second upsampling module and the output of the third CR module enter the second feature splicing module for splicing, and the spliced ​​features enter the fifth CR module.

[0063] The output of the fifth CR module and the output of the first RepNCSPELAN4 module enter the third feature splicing module for splicing, and the spliced ​​features enter the sixth CR module.

[0064] The output of the sixth CR module and the output of the AIFI module enter the fourth feature splicing module for feature splicing, and the spliced ​​features enter the second RepNCSPELAN4 module.

[0065] There are three detection heads in the detection head network. The fifth CR module is connected to a detection head in the detection head network through an MSFN module. The sixth CR module is connected to a detection head in the detection head network through an MSFN module. The second RepNCSPELAN4 module is connected to a detection head in the detection head network through an MSFN module.

[0066] The above process combines feature maps of different scales through upsampling and concatenation to perform multi-level feature fusion. The final feature map passes through multiple detection heads, performing object detection at different scales and outputting bounding box and category prediction results.

[0067] like Figure 4 and Figure 5 As shown, the general processing ideas of the auxiliary branch are as follows:

[0068] After the image is input, preliminary feature extraction is performed, and the extracted features and multiple scale features of the main branch shallow network are combined through the CBFuse layer (i.e. Figure 5 The multi-scale feature fusion shown in the figure is fused to form a fused feature; after the CBLinear layer adjusts the channel, it is integrated by the CBFuse layer to ensure the uniformity of dimension and channel; the fused feature is divided into three branches for multi-scale prediction analysis, and the loss is calculated and fed back in the Head layer compared with the label to enhance the shallow feature gradient information and reduce information loss; this auxiliary branch is not used in the reasoning of the cloth small and complex defect recognition model to maintain computational efficiency.

[0069] Specifically, if Figure 5 As shown in Figure 5, the auxiliary branch includes two convolution modules, three multi-scale feature fusion CBFuse layers, three CBLinear layers, three CR modules, three MSFN modules and one RepNCSPELAN4 module.

[0070] Three CBFuse layers are defined as the first CBFuse layer, the second CBFuse layer, and the third CBFuse layer.

[0071] Define three CBLinear layers as the first CBLinear layer, the second CBLinear layer, and the third CBLinear layer.

[0072] Three CR modules are defined as the seventh, eighth and ninth CR modules.

[0073] The processing flow of the auxiliary branch is as follows:

[0074] After the image is input, it first passes through the convolution module and the seventh CR module for preliminary feature extraction. Then it passes through another convolution module and is input to the first CBFuse layer for feature fusion. At the same time, the second CR module adjusts the channel features through the first CBLinear layer, the third CR module adjusts the channel features through the second CBLinear layer, and the fourth CR module adjusts the channel features through the third CBLinear layer, and then inputs them into the first CBFuse layer.

[0075] The features of the first CBFuse layer are input for multi-scale feature fusion, and the fusion results enter the eighth CR module.

[0076] The eighth CR module is connected to a detection head through an MSFN module.

[0077] The output features of the eighth CR module and the features of the third CR module after adjusting the channel through the second CBLinear layer enter the second CBFuse layer, and multi-scale feature fusion is performed in the second CBFuse layer. The fusion result enters the ninth CR module.

[0078] The ninth CR module is connected to a detection head via an MSFN module. The output features of the ninth CR module and the features of the fourth CR module are adjusted through the third CBLinear layer and then enter the third CBFuse layer. Multi-scale feature fusion is performed in the third CBFuse layer, and the fusion results are entered into the RepNCSPELAN4 module.

[0079] The RepNCSPELAN4 module is connected to a detection head through a MSFN module.

[0080] like Figure 6 As shown in the figure, the improved snake-shaped dynamic convolution module C-DSC deformable convolution allows the network to learn geometric changes completely freely, so that the perception area can roam, especially on small tubular structures. In view of this, the free learning process is supplemented by constraints, thereby enhancing the perception of tubular structures in a targeted manner.

[0081] In order to meet the challenges of complex and changeable defect morphology detection, the network structure of the improved snake-shaped dynamic convolution module C-DSC is obtained by introducing the CBAM attention mechanism on the basis of the original DSC convolution.

[0082] CBAM consists of two independent sub-modules, namely the Channel Attention Module (CAM) and the Spatial Attention Module (SAM), which perform channel and spatial attention respectively. It not only saves parameters and computing power, but also ensures that it can be integrated into the existing network architecture as a plug-and-play module.

[0083] The feature map of the input C-DSC is first input into the DSC convolution, and horizontal and vertical feature extraction are performed respectively. After the extraction is completed, the features are fused, and then down-sampled to make the image fit the size of the display area, and a thumbnail of the corresponding image is generated. The result after multiple iterations is passed to the next process; the next DSC convolution receives the information and performs feature extraction again. After extraction, the horizontal and vertical features are fused, and up-sampled to enlarge the original image and restore the original size; the feature information after multiple iterations is passed to the CBAM module for spatial attention and channel attention operations respectively; then it is compared with the label, and TCLoss is fed back upward to finally obtain the output result.

[0084] First, the sample image is input. Since the defect sample image is two-dimensional, its defect position and shape are irregular, so it first passes through the snake convolution module DSC, which contains a 9*9 convolution kernel. Each grid position K in the convolution kernel K i±c The selection of is a cumulative process to extract the general features of the defect points and make a rough judgment on the location of the features, so as to prepare for the accurate positioning of the defects. i Initially, the position of the grid away from the center depends on the position of the previous grid: K i+1 Relative to K i An offset Δ={δ|δ∈[-1,1]} is added.

[0085] Therefore, the offset needs to be accumulated Σ to ensure that the convolution kernel conforms to the linear morphology structure.

[0086] The change in the x-axis direction is:

[0087]

[0088] The direction of change of the y-axis is:

[0089]

[0090] Among them, K i Represents the direction of change of the x-axis; K j represents the direction of change of the y-axis; x represents the horizontal coordinate; y represents the vertical coordinate; i, j, c represent constants; Δx represents the x-axis increment; Δy represents the y-axis increment.

[0091] Since coordinates are usually integers, every pixel in the feature map may have defects, and the size of the defects is small, so the offset Δ between adjacent distances will be a decimal. Bilinear interpolation considers the values ​​of the four points around the interpolated point and performs a weighted average based on the values ​​of these four points to obtain an estimated value of the interpolated point.

[0092] This method can better reflect the real changes of data than simple nearest neighbor interpolation, so it has higher interpolation accuracy, so bilinear interpolation is used here. Bilinear interpolation is expressed as:

[0093]

[0094] Among them, K represents the decimal position of the equation, K′ lists all integer spatial positions, and B is the bilinear interpolation kernel, which is decomposed into two one-dimensional kernels, namely: B(K,K′)=b(K x ,K x ′)·b(K y ,K y ′).

[0095] Through the above steps, the x-axis and y-axis directions are changed respectively, extending from the inside of the defect to the outside, and the horizontal and vertical positions are continuously adjusted. Then, the spatial attention mechanism and channel attention mechanism in the CBAM module are used to improve the performance of the network and enhance the accuracy of sample feature recognition.

[0096] Finally, the output value is compared with the label to obtain the loss value TCLoss. The reverse transmission is used to adjust the training strategy through multiple iterations to mark the target point position.

[0097] exist Figure 6 In the feature fusion part mentioned in [1], multi-view feature fusion is adopted to ensure that important information is retained from different global forms by supplementing the attention to features from multiple perspectives during feature fusion.

[0098] like Figure 7 As shown in Figure 2, the processing flow of multi-view feature fusion is as follows:

[0099] The sample after feature extraction has a multi-layer nature. The different layers of the unified feature map are named: feature Figure 1 ,feature Figure 2 ,feature Figure 3 ..., each layer of features is divided into horizontal features and vertical features. Figure 1 Randomly discard horizontal features or vertical features, and then Figure 2 Randomly discard horizontal features or vertical features until all layers are discarded, then fuse the processed features of each layer and finally output the feature map.

[0100] like Figure 8 The structural diagram of the CBAM module is shown, where × represents element-wise multiplication. It includes two modules: the first is the Channel Attention Module (CAM); the second is the Spatial Attention Module (SAM); and two multipliers, defined as the first multiplier and the second multiplier.

[0101] The specific implementation process is as follows: first, the input feature enters the channel attention module, and after the output, it enters the first multiplier for element-wise multiplication with the original input feature map and then enters the spatial attention module. The output of the spatial attention module and the output of the first multiplier enter the second multiplier for feature element multiplication, and finally the feature map is output.

[0102] like Figure 9 As shown, the channel attention module will take the feature map F H×W×C After global max pooling and global average pooling based on width and height respectively, two 1×1×C feature maps are obtained. Then, they are fed into a two-layer neural network (MLP). The number of neurons in the first layer is C / r, where r is the reduction rate and the activation function is Relu. The number of neurons in the second layer is C. The two layers of the neural network are shared. Then, the features output by the MLP are summed based on element-wise operation and then activated by sigmoid (activation function) to generate the final channel attention feature, namely M c .

[0103] In short, channel attention is calculated as follows:

[0104]

[0105] Where σ represents the sigmoid function; c stands for channel; F avg represents the average pooling feature; F max Represents the maximum pooling feature; F represents the feature map; W0 and W1 represent weights. The MLP weights W0 and W1 are shared by both inputs, and the ReLU activation function is followed by w0. Figure 9 In , “+” represents feature addition.

[0106] like Figure 10As shown in the figure, the feature map output by the channel attention module is used as the input feature map of this module. First, after a global maximum pooling and global average pooling based on the channel, two H×W×1 feature maps are obtained, and then these two feature maps are channel-joined. Then, after a 7×7 convolution operation, the dimension is reduced to 1 channel, that is, H×W×1. Then, after a sigmoid operation, the spatial attention feature is generated, that is, M s , and then splice CAM and SAM together front and back to output the final feature map.

[0107] In short, spatial attention is calculated as follows:

[0108]

[0109] Where σ represents the S-type function; s represents space; F avg represents the average pooling feature; F max represents the maximum pooling feature; f 7 ×7 Represents a convolution operation with a filter size of 7×7.

[0110] The present invention uses the AIFI module in the RT-DETR model to replace the SPPELAN enhancement model in YOLOv9 to enhance the processing ability of local and global information. The RT-DETR model is a deep learning model for target detection. It is based on the Transformer architecture, and the composition of the AIFI module is similar to that of the Transformer Encoder, which is particularly suitable for real-time processing of sequence data. The original SPPELAN module mainly calculates the correlation between feature maps and assigns different weights to different regions, thereby achieving local attention enhancement. SPPELAN adopts a multi-task learning approach to simultaneously optimize multiple tasks such as target detection, classification, and positioning, so that the model can better adapt to various complex scenarios. However, when processing large-scale images or high-resolution images, it may increase the computational complexity due to the need to generate feature maps of multiple scales and perform pooling operations. This may lead to a decrease in processing speed, affecting the performance of real-time applications or large-scale data processing scenarios, while the AIFI module can reduce unnecessary data processing processes, optimize the algorithm structure, and reasonably allocate data processing resources while ensuring calculation accuracy, further improve computing efficiency, reduce running time, and enable the model to achieve the purpose of lightweight.

[0111] like Figure 11 As shown in the figure, “×” represents the dot product between matrices. In this embodiment, the AIFI module receives the feature map from the improved snake-shaped dynamic convolution module C-DSC as input, as shown in the figure. Figure 2 As shown, the AIFI module performs the following processing:

[0112] The input feature map is first flattened into a sequence to form an image feature sequence containing all pixels; the flattening operation will preserve the spatial information of the feature and convert it into a form suitable for processing by the self-attention mechanism, which is then converted into three matrices: query Q, key K, and value V. Then, by calculating the dot product between Q and K, the attention weight matrix is ​​obtained; then, the attention weight matrix is ​​applied to V to produce a weighted feature representation; finally, after processing by the self-attention mechanism, the resulting feature sequence is restored to the same shape as the input. It can be expressed by the following formula:

[0113] Q = K = V = Flatten(input);

[0114] Output=Reshape(Attn(Q,K,V));

[0115] Where Q represents the query matrix; K represents the key matrix; V represents the value matrix; Flatten represents the flattening operation; and Reshape represents the matrix transformation.

[0116] A multi-scale feedforward network (MSFN) is added to the feature layers of different scales before the detection head, that is, before the head layer, to aggregate multi-scale features and enhance nonlinear information conversion. Previous experiments on adding multi-scale information to image denoising tasks have shown that it is very effective. In addition, the MSFN module also has the convenient plug-and-play feature when using the model.

[0117] The processing flow of the multi-scale feedforward network MSFN is as follows:

[0118] The input features are processed through two parallel paths, and the nonlinear transformation is enhanced by the element-wise product of the features of the two paths; in the upper path, depthwise convolution is used for feature extraction; in the lower path, multi-scale dilated convolution is used for multi-scale feature extraction, using two 3×3 dilated convolutions with dilation rates of 2 and 3 respectively.

[0119] Given an input tensor Y∈R H*W*C , then MSFN is defined as:

[0120]

[0121] Y out =W 1×1 ⊙Gating(Y);

[0122] Among them, H, W, and C represent the sample height, width, and number of channels respectively;

[0123] Gating(Y) represents the gating mechanism, ⊙ represents element-wise multiplication, and φ represents the GELU activation function. represents a 3×3 dilated convolution with a dilation rate of 2. Represents a 3×3 dilated convolution with a dilation rate of 3;

[0124] W 3×3×3 represents a 3×3×3 convolution; W 1×1 (Y) represents a 1×1 convolution of feature Y; out Indicates output.

[0125] Figure 12 In the figure, “+” represents the merging of two channels, and “×” indicates that the merged process is multiplied with the first path. Two 1×1 convolutions are used to expand the feature channels, with an expansion ratio of γ = 2.

[0126] When generating detection boxes, a positive and negative sample matching strategy is also used. TAL (Task Align Learning) unifies classification and regression anchors by constructing an alignment metric, ultimately achieving a higher-scoring classification box and more accurate localization box coefficient during inference.

[0127] The key to TAL is to construct an alignment metric. The calculation of the alignment metric consists of two parts. The first is to construct a classification score matrix (tensor) s for each label corresponding to each region, and u is the IoU between the predicted target box and the label corresponding to each anchor point.

[0128] The calculation of IoU is as follows:

[0129]

[0130] Among them, A represents the real box and B represents the predicted box.

[0131] Then, the new anchor alignment index t can be obtained by the following calculation.

[0132] t=s a *u b ;

[0133] Where t represents the anchor alignment metric, and a and b are parameters used to control the influence of the two tasks on the anchor alignment metric. The IOU information depends on the ground-truth and predicted box information, and the two are combined to obtain the alignment metric matrix.

[0134] Step 3. Use the data set in step 1 to train a cloth small and complex defect recognition model based on the improved YOLOv9 model. During the training process of the cloth small and complex defect recognition model, an auxiliary branch is introduced to assist in training.

[0135] Before model training, set the training parameters, set the number of epochs to 300, the number of batch training samples to batch_size to 16, the number of works to 8, and the number of imagi to 640. Set the learning rate optimizer to adam optimizer.

[0136] During model training, the loss function is as follows:

[0137] Distribution Focal Loss (DFL), the formula is as follows:

[0138]

[0139] Among them, y i represents the actual value of the model, Represents the model prediction value, and the Focal Loss function is defined as:

[0140]

[0141] Among them, α t It represents the balancing factor, which is used to adjust the influence between positive and negative samples; γ is the focus factor, which is used to adjust the weight of difficult and easy samples.

[0142] Complete Intersection over Union Loss (CIoU) is shown below:

[0143]

[0144] where ρ(b,b gt ) is the Euclidean distance between the center point of the predicted box and the real box; c is the diagonal length of the minimum bounding box of the predicted box and the real box; v is a parameter that measures the consistency of the aspect ratio of the predicted box and the real box; b is the position of the predicted box; b gt Indicates the true box position.

[0145] α is the weight balancing factor, and its formula is as follows:

[0146]

[0147] The expression of the total loss function of the model is: DFLoss+CIoU Loss.

[0148] Step 4. After the cloth defect image to be identified has been preprocessed in step 1, it is input into the cloth small and complex defect recognition model trained in step 3, and the target recognition result is output, including the position of the bounding box and the defect category, so as to identify the small and complex cloth defects in the cloth defect image.

[0149] Compared with the original YOLOv9 model, the present invention can improve the recognition efficiency of complex and irregular small defects through the proposed C-DSC network. At the same time, because YOLOv9 introduces an innovative basic network architecture, this design enhances the effectiveness of feature extraction while maintaining efficient computing. By increasing the depth and complexity of the network, YOLOv9 can more accurately capture the background information in the image, further improving its accuracy in target recognition and positioning, and can be applied to real-time cloth defect scene detection. In addition, the AIFI module focuses on achieving richer feature fusion between features of the same scale, which not only improves the efficiency and effectiveness of feature extraction, but also improves detection accuracy by capturing finer-grained information. The design of the AIFI module maintains computational efficiency while reducing the computational cost of attention operations between multi-scale features, making the model more flexible and efficient when processing complex scenes. In addition, the present invention also introduces MSFN before the detection head network, which significantly enhances the overall feature extraction capability of the network by fusing features of different scales. MSFN can also be flexibly integrated into various model networks. Its multi-scale feature extraction method enables the network to capture both global and local information, improving the model's ability to understand complex images while minimizing computing resource consumption while ensuring performance improvements. Through these improvements, the fabric fine and complex defect recognition model built based on the improved YOLOv9 model has significantly improved the accuracy and efficiency of the invention for identifying fine and complex defects in fabric defect images.

[0150] Example 2

[0151] This second embodiment describes a computer device comprising a memory and one or more processors. The memory stores executable code. When the processor executes the executable code, the computer device implements the steps of the method for identifying small and complex defects in fabric described in the first embodiment.

[0152] In this embodiment, the computer device is any device or apparatus with data processing capability, which will not be described in detail here.

[0153] Example 3

[0154] This embodiment 3 describes a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, it is used to implement the steps of the method for identifying small and complex defects in cloth in the above-mentioned embodiment 1.

[0155] The computer-readable storage medium can be an internal storage unit of any device or apparatus with data processing capabilities, such as a hard disk or memory, or an external storage device of any device with data processing capabilities, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc. equipped on the device.

[0156] Of course, the above description is only a preferred embodiment of the present invention, and the present invention is not limited to the above-mentioned embodiments. It should be noted that all equivalent substitutions and obvious deformation forms made by any technician familiar with this field under the guidance of this specification fall within the substantive scope of this specification and should be protected by the present invention.

Claims

1. A method for identifying small and complex defects in cloth, characterized in that: The steps include: Step 1. First, preprocess the collected cloth defect images to form a dataset for model training; Step 2. Build a model for identifying small and complex fabric defects based on the improved YOLOv9 model. The main branch structure of the improved YOLOv9 model is improved on the original YOLOv9 model as follows: An improved serpentine dynamic convolution module (C-DSC) is introduced into the backbone network of the main-branch structure to adaptively and dynamically extract defect points, thereby improving sensitivity to complex defect shapes and boundaries and enhancing the ability to detect slender and complex defects. The backbone network includes a convolution module, a CR module, and an improved snake-shaped dynamic convolution module C-DSC. There are four CR modules. The features input to the backbone network are sequentially extracted by the convolution module and the four CR modules before entering the C-DSC. C-DSC is obtained by introducing the CBAM attention mechanism on the basis of the original DSC convolution. The feature information after multiple iterations of DSC is passed into the CBAM module for spatial attention and channel attention operations respectively. The AIFI module in the RT-DETR model is used to replace the SPPELAN module in the YOLOv9 model to enhance the model's ability to process local and global information and enhance the interaction between features at the same scale. The output features of C-DSC are used as the input features of the AIFI module and input into the AIFI module of the neck network; The CBAM consists of two independent submodules, namely the channel attention module CAM and the spatial attention module SAM, which perform channel and spatial attention operations respectively. The processing flow of C-DSC is as follows: The input feature image is first input into the DSC convolution, and horizontal and vertical feature extraction is performed respectively. After the extraction is completed, the features are first fused, and then down-sampled to make the image fit the size of the display area, and a thumbnail of the corresponding image is generated. The result after multiple iterations is passed to the next process; the next DSC convolution receives the information and performs feature extraction again. After extraction, the horizontal and vertical features are fused and up-sampled to enlarge the original image and restore the original size; the feature information after multiple iterations is passed to the CBAM module for spatial attention and channel attention operations respectively; then it is compared with the label, and TCLos is fed back upward to finally obtain the output result; A multi-scale feedforward network (MSFN) is added before the Head layer to aggregate multi-scale features and enhance nonlinear information conversion. Step 3. Use the dataset from Step 1 to train a cloth small and complex defect recognition model based on the improved YOLOv9 model. During the training process, an auxiliary branch is introduced to assist in the training. Step 4. After the fabric defect image to be identified has been preprocessed in step 1, it is input into the small and complex fabric defect recognition model trained in step 3. The final output is the target recognition result, including the location and category of the bounding box, thereby identifying the small and complex fabric defects in the fabric defect image.

2. The method for identifying small and complex defects in cloth according to claim 1, characterized in that: The improved YOLOv9 model has a main branch structure including a backbone network, a neck network and a detection head network; The backbone network includes a convolution module, a CR module, and an improved snake-shaped dynamic convolution module C-DSC; there are four CR modules, which are defined as the first CR module, the second CR module, the third CR module, and the fourth CR module respectively; Each CR module is composed of a convolution module and a RepNCSPELAN4 module. It extracts features from the input image through multiple downsampling convolutions, thereby achieving full feature extraction. The processing flow of the backbone network is as follows: the input feature map first passes through a convolution module for convolution downsampling operation to reduce the image size and increase the number of channels of the image, and then passes through the first, second, third, and fourth CR modules in sequence for feature extraction. Finally, it passes through the improved snake-shaped dynamic convolution module C-DSC to perform adaptive dynamic extraction of defect points.

3. The method for identifying small and complex defects in cloth according to claim 2, characterized in that: The neck network includes an AIFI module, an upsampling module, a splicing module, a RepNCSPELAN4 module, and a CR module; wherein there are two upsampling modules, which are defined as a first upsampling module and a second upsampling module respectively; There are four feature splicing modules, which are defined as the first, second, third, and fourth feature splicing modules respectively; There are two CR modules, which are defined as the fifth and sixth CR modules respectively; There are two RepNCSPELAN4 modules, which are defined as the first and second RepNCSPELAN4 modules respectively; The processing flow of the neck network is as follows: The output of the backbone network enters the AIFI module; the output of the AIFI module is upsampled by the first upsampling module, and the upsampling result and the output of the fourth CR module are entered into the first feature splicing module for splicing; The output features of the first feature concatenation module pass through the first RepNCSPELAN4 module and the second upsampling module in sequence; The output of the second upsampling module and the output of the third CR module enter the second feature splicing module for splicing, and the output features of the second feature splicing module enter the fifth CR module; The output of the fifth CR module and the output of the first RepNCSPELAN4 enter the third feature splicing module for splicing, and the output features of the third feature splicing module enter the sixth CR module; The output of the sixth CR module and the output of the AIFI module enter the fourth feature splicing module for feature splicing, and the output features of the fourth feature splicing module enter the second RepNCSPELAN4 module; The detection head network includes three detection heads; the fifth CR module is connected to a detection head of the detection head network through an MSFN module; the sixth CR module is connected to a detection head of the detection head network through an MSFN module; the second RepNCSPELAN4 module is connected to a detection head of the detection head network through an MSFN module.

4. The method for identifying small and complex defects in cloth according to claim 1, characterized in that: The AIFI module receives the feature map from C-DSC as input and performs the following processing: The input feature map is first flattened into a sequence to form an image feature sequence containing all pixels. The flattening operation will maintain the spatial information of the feature and convert it into a form suitable for processing by the self-attention mechanism. It is then converted into three matrices: query Q, key K, and value V. Then, the attention weight matrix is ​​obtained by calculating the dot product between Q and K. Next, the attention weight matrix is ​​applied to V to produce a weighted feature representation. Finally, after processing by the self-attention mechanism, the obtained feature sequence is restored to a feature of the same shape as the input.

5. The method for identifying small and complex defects in cloth according to claim 1, characterized in that: The processing flow of the multi-scale feedforward network MSFN is as follows: The input features are processed through two parallel paths, and the nonlinear transformation is enhanced by the element-wise product of the features of the two paths. In the upper path, depthwise convolution is used for feature extraction. In the lower path, multi-scale dilated convolution is used for multi-scale feature extraction, using two 3 × 3 dilated convolutions with dilation rates of 2 and 3 respectively. If the input tensor is given , then MSFN is defined as: ； ； in, Represents sample height, width, and number of channels respectively; represents the gating mechanism, ⊙ represents element-wise multiplication, represents the GELU activation function, represents a 3×3 dilated convolution with a dilation rate of 2. represents a 3×3 dilated convolution with a dilation rate of 3; Represents 3×3×3 convolution; Indicates a 1×1 convolution of feature Y; Indicates output.

6. The method for identifying small and complex defects in cloth according to claim 1, characterized in that: The auxiliary branch includes two convolution modules, three multi-scale feature fusion CBFuse layers, three CBLinear layers, three CR modules, three MSFN modules and one RepNCSPELAN4 module; Define three CBFuse layers: the first CBFuse layer, the second CBFuse layer, and the third CBFuse layer Define three CBLinear layers: the first CBLinear layer, the second CBLinear layer, and the third CBLinear layer; Define three CR modules as the seventh, eighth and ninth CR modules; The processing flow of the auxiliary branch is as follows: After the image is input, it first passes through the convolution module and the seventh CR module for preliminary feature extraction, then passes through another convolution module and is input to the first CBFuse layer for feature fusion; at the same time, the second CR module adjusts the channel features through the first CBLinear layer, the third CR module adjusts the channel features through the second CBLinear layer, and the fourth CR module adjusts the channel features through the third CBLinear layer, and then inputs them into the first CBFuse layer at the same time; Multiple features of the first CBFuse layer are input for multi-scale feature fusion, and the fusion results are fed into the eighth CR module; The eighth CR module is connected to a detection head through an MSFN module; The output features of the eighth CR module and the features of the third CR module are adjusted through the second CBLinear layer and enter the second CBFuse layer. Multi-scale feature fusion is performed in the second CBFuse layer, and the fusion result enters the ninth CR module. The ninth CR module is connected to a detection head through an MSFN module. The output features of the ninth CR module and the features of the fourth CR module are adjusted through the third CBLinear layer and enter the third CBFuse layer. Multi-scale feature fusion is performed in the third CBFuse layer, and the fusion results enter the RepNCSPELAN4 module. The RepNCSPELAN4 module is connected to a detection head through a MSFN module.

7. The method for identifying small and complex defects in cloth according to claim 1, characterized in that: In step 3, the training parameters are set during the training process of the improved YOLOv9 model, the number of iterations epochs is set to 300, the number of batch training samples batch_size is set to 16, Works is set to 8, and Imgsz is set to 640; and the learning rate optimizer optimizer is set to Adam optimizer.

8. A computer device comprising a memory and one or more processors; executable code is stored in the memory; and When the processor executes the executable code, it is used to implement the method for identifying small and complex defects in cloth as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a program stored thereon; characterized in that: When the program is executed by a processor, it is used to implement the method for identifying small and complex defects in cloth as described in any one of claims 1 to 7.

Images