Method for identifying small and complex flaws of cloth, computer equipment and medium

Abstract

The invention belongs to the technical field of image processing, and discloses a method for identifying small and complex flaws of cloth, computer equipment and a medium. According to the method, a cloth fine and complex defect identification model based on an improved YOLOv9 model is built, that is, an improved S-shaped dynamic convolution module C-DSC is introduced into a backbone network of a main branch structure, and defect points are adaptively and dynamically extracted, so that the sensitivity to the shape and boundary of a target is improved, and the defect detection of a complex slender pattern is enhanced; secondly, an AIFI module in the RT-DETR model is used for replacing an SPPELAN module in the YOLOv9 model, interaction between features in the same scale is enhanced, processing delay is reduced, and the speed is increased; and finally, adding a multi-scale feedforward network MSFN in front of the detection head network, aggregating multi-scale features and enhancing non-linear information conversion. According to the invention, the identification accuracy and efficiency of tiny and complex flaws in the cloth flaw image are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a method, a computer device and a medium for identifying small and complex defects in cloth, which are particularly suitable for identifying small and complex defects in cloth. Background Art

[0002] For cloth defect detection, it mainly relies on deep learning networks based on machine vision. Since there are many types of cloth defects, different shapes and uneven sizes, it is particularly important to use deep learning-based methods for detection. The network is trained with a large amount of sample data to make it generalizable and robust for the detection of various types of cloth. Therefore, in the detection, the designed network needs to be able to handle samples of different colors, sizes and specifications, so the generalization, accuracy, efficiency and stability of the network are particularly important. At present, because the convolutional neural network has a large nonlinear fitting function, it greatly improves the detection efficiency when detecting defects. For example:

[0003] Zhao et al. proposed a detection method based on VGG16 network for defect detection of linen fabrics and image fabrics, which can detect six defects such as holes and stains. However, the number of parameters used in training is large, and the training time is long, which does not meet the requirements of timeliness in detecting defects. Pandia et al. designed a deep convolutional neural network using fabric color as an important feature, learned from the training stage of various defect data sets, and used a learning feature to classify defects in the test stage, but only classified defects with obvious characteristics, and failed to meet the detection of small and complex defects, which has limitations. Zhu Junling et al. used AR model spectral estimation to model cloth texture images and obtained variance sequences. The defect type and position were obtained by calculating the correlation coefficient between the defect image and the normal texture image spectral estimation. However, false spectral peaks and spectral line splitting are prone to occur in the modeling process, and noise will deteriorate the spectral estimation, resulting in a decrease in detection accuracy and a large deviation in the detection results. An Meng et al. optimized the convolutional neural network based on the Faster RCNN network model, introduced feature pyramid network models of different scales into the region proposing network (RPN) to increase detailed shallow features, and reduced the intra-class spacing and increased the inter-class spacing by regularizing the Softmax classifier. The network parameters were continuously updated to improve the network convergence ability. Although high-quality candidate regions can be generated, in some cases, the recognition effect is not good for regions with a large aspect ratio, and gradient explosion is prone to occur, and its performance may be limited under certain conditions.

[0004] Object recognition based on YOLO (You Only Look Once) has become mainstream. The detector usually consists of two parts. One is the backbone network (backbone) for extracting features, that is, the basic network, which is generally pre-trained on the ImageNet dataset. The other is the head (Head) that predicts the object category and bounding box, and the neck (Neck) is built between the backbone and the head to collect different feature maps. However, the depth of the network layer will directly affect the accuracy and efficiency of object detection. The deeper the network structure, the more nonlinear expression ability, the more complex transformations can be learned, and the more complex feature inputs can be fitted; the shallower the network layer, the opposite is true. Researchers have been optimizing the network model and constantly adjusting the parameters to reduce the amount of calculation and improve efficiency on the basis of high accuracy. At present, for other versions of YOLO, there is a problem that a large amount of information will be lost when the input data undergoes layer-by-layer feature extraction and spatial transformation. The loss of information may lead to biased gradient flows, which are then used to update the model, which is likely to cause the deep network to establish incorrect association connections.

[0005] The detection method based on YOLOv9 has great development potential in use. It proposes the concept of Programmable Gradient Information (PGI) to cope with the various changes required for deep networks to achieve multiple goals. At the same time, YOLOv9 also designs a new lightweight network architecture based on gradient path planning - Generalized Efficient Layer Aggregation Network (GELAN).

[0006] However, when YOLOv9 is applied to the recognition of small and complex defects in cloth, there are the following problems:

[0007] 1. Large resource consumption: Although YOLOv9 has improved in accuracy and efficiency, it is still a complex deep learning model that requires large resources and a long time for training and inference.

[0008] 2. Model overfitting and underfitting affect recognition results: Like other deep learning models, YOLOv9 also faces the problems of overfitting and underfitting. Overfitting means that the model performs well on the training set but poorly on the test set, while underfitting means that the model performs poorly on both the training set and the test set. It is very likely that the detection effect and efficiency of small defects will be affected due to fitting problems.

[0009] 3. Insufficient generalization ability affects the accuracy of defect recognition: Although YOLOv9 has obtained relatively good results through testing on multiple data sets, its generalization ability may still be lacking. In other words, if the amount of data used is too small, the quality is too poor, the specifications are not standard, and the label features are not obvious enough, it will cause serious errors in the training results. If the data set is unbalanced, that is, the number of samples in some categories is much larger than that in other categories, then the model may tend to recognize categories with a larger number and ignore categories with a smaller number, which will affect the accuracy of small and complex defect recognition and may cause missed detection or wrong detection.

[0010] 4. The detection effect of targets with a large length-to-width ratio is poor: Since the assembly line machines are in continuous operation during the cloth production process, this may cause the defects to be small, narrow, or densely distributed. YOLOv9 divides the entire image into larger grids during prediction. Therefore, the processing of dense targets may not be ideal during detection, which may easily lead to missed detections or overlapping detection frames. Summary of the invention

[0011] The purpose of the present invention is to propose a method for identifying small and complex defects in cloth. The method builds a model for identifying small and complex defects in cloth based on the improved YOLOv9 to improve the recognition accuracy and efficiency of small and complex defects.

[0012] In order to achieve the above object, the present invention adopts the following technical scheme:

[0013] A method for identifying small and complex defects in cloth, comprising the following steps:

[0014] Step 1. First, preprocess the collected cloth defect images and form a data set for model training;

[0015] Step 2. Build a cloth small and complex defect recognition model based on the improved YOLOv9 model; the main branch structure of the improved YOLOv9 model is obtained by making the following improvements on the basis of the original YOLOv9 model:

[0016] An improved serpentine dynamic convolution module C-DSC is introduced into the backbone network of the main branch structure to adaptively and dynamically extract defect points, so as to improve the sensitivity to complex defect shapes and boundaries and enhance the defect detection capability for slender and complex styles; the AIFI module in the RT-DETR model is used to replace the SPPELAN module in the YOLOv9 model to enhance the model's ability to process local and global information and enhance the interaction between features at the same scale to reduce processing delays and increase speed; a multi-scale feedforward network MSFN is added before the Head layer to aggregate multi-scale features and enhance nonlinear information conversion;

[0017] Step 3. Use the data set in step 1 to train a cloth small and complex defect recognition model based on the improved Yolov9 model. During the training process of the cloth small and complex defect recognition model, an auxiliary branch is introduced to help the training;

[0018] Step 4. After the cloth defect image to be identified has been preprocessed in step 1, it is input into the cloth small and complex defect recognition model trained in step 3, and the target recognition result is finally output, including the position and category of the bounding box, so as to identify the small and complex cloth defects in the cloth defect image.

[0019] In addition, based on the method for identifying small and complex defects of cloth, the present invention also proposes a computer device, which includes a memory and one or more processors. Executable code is stored in the memory.

[0020] When the processor executes the executable code, it is used to implement the above-mentioned method for identifying small and complex defects in cloth.

[0021] In addition, based on the method for identifying small and complex defects in cloth, the present invention also proposes a computer-readable storage medium on which a program is stored. When the program is executed by a processor, it is used to implement the method for identifying small and complex defects in cloth.

[0022] The present invention has the following advantages:

[0023] As described above, the present invention relates to a method for identifying small and complex defects of cloth based on an improved YOLOv9 model. The method aims at the problem of detecting small and complex defects of cloth. First, a module C-DSC combining a dynamic snake convolution (DSC) and a convolutional block attention module (CBAM) introduced in the backbone network (Backbone) of YOLOv9 is used to improve the sensitivity to the shape and boundary of the target, so as to enhance the detection capability of complex defects. The dynamic snake convolution imitates the shape and characteristics of a snake, and is mainly used to process data structures with slender, circuitous and irregular shapes. It can not only adaptively focus on the local features of the slender curves of the tubular structure, but also enhance the perception of the geometric structure, and prevent the situation of falling into a circular loop during the iteration process. The addition of the CBAM attention module, by combining channel attention and spatial attention, can adaptively learn the correlation between different features, and can improve the feature extraction capability and generalization capability of the model without increasing the computational load of the network. Then, the Attentionbased Intra-scale Feature Interaction (AIFI) module in the transformer-based real-time detection transformer (RT-DETR) model is used to replace the SPPELAN (Spatial PyramidPooling with Enhanced Local Attention Network) in YOLOv9. Since it is similar to the encoder of the transformer, it can enhance the model's ability to process local and global information and reduce processing delays. By optimizing the attention operation between multi-scale features, the consumption of computing resources is reduced, and the model shows higher flexibility and operating efficiency when dealing with complex scenes. Finally, the Multi-Scale Feedforward Network (MSFN) is introduced before the Head layer. The design of MSFN can improve nonlinear feature transformation and strengthen nonlinear information conversion, so that the model can better handle complex nonlinear problems. It can also enhance its ability to extract features by extracting features of different scales, especially for enhancing the extraction of small and complex defects. Through the above improvements, the cloth small and complex defect recognition model based on the improved YOLOv9 model built by the present invention has greatly improved the recognition accuracy and efficiency of the present invention for small and complex defects in cloth defect images. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A flowchart of a method for identifying small and complex defects in cloth according to an embodiment of the present invention;

[0025] Figure 2 A network structure diagram of a model constructed in an embodiment of the present invention;

[0026] Figure 3 This is a main branch structure diagram in an embodiment of the present invention;

[0027] Figure 4 This is a flow chart of the auxiliary branch in an embodiment of the present invention;

[0028] Figure 5 This is a structural diagram of an auxiliary branch in an embodiment of the present invention;

[0029] Figure 6 A network structure diagram of a C-DSC in an embodiment of the present invention;

[0030] Figure 7 This is a flow chart of multi-view feature fusion in an embodiment of the present invention;

[0031] Figure 8 The CBAM network architecture in the embodiment of the present invention;

[0032] Fig. 9 This is a flow chart of the channel attention mechanism in an embodiment of the present invention;

[0033] Fig.10 This is a flow chart of the spatial attention mechanism in an embodiment of the present invention;

[0034] Fig.11 A network structure diagram of AIFI in an embodiment of the present invention;

[0035] Fig.12 FIG. 4 is a network structure diagram of the MSFN in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0037] Example 1

[0038] This embodiment describes a method for identifying small and complex defects in cloth. The method builds a small and complex defect recognition model for cloth based on the improved YOLOv9 model to improve the recognition accuracy and efficiency of small and complex defects in cloth defect images. YOLOv9 introduces an innovative backbone network architecture in design, which significantly improves the feature extraction capability while maintaining computational efficiency. By cleverly increasing the network level and introducing a more complex connection mechanism, YOLOv9 can capture the contextual information in the image more deeply, thereby achieving more accurate recognition and positioning of the target. This design enables YOLOv9 to significantly improve the accuracy of its target detection while maintaining high efficiency. The concept of PGI introduced in YOLOv9 solves the problem of information loss in data transmission in deep neural networks. Through PGI, the model can efficiently transmit gradient information while maintaining the integrity of the input data, thereby improving learning efficiency and model performance. A lightweight network architecture GELAN is also designed, which optimizes the utilization and computational efficiency of network parameters based on gradient path planning. Through the improved network structure, YOLOv9 achieves higher accuracy and faster processing speed while remaining lightweight.

[0039] Based on the original YOLOv9 model, the present invention first introduces an improved serpentine dynamic convolution module C-DSC to adaptively and dynamically extract defect points, so as to improve the sensitivity to complex defect shapes and boundaries and enhance the defect detection capability of slender and complex styles. Then, the AIFI module in the RT-DETR model is used to replace the SPPELAN in YOLOv9 to enhance the model's processing capability for local and global information and enhance the interaction between features within the same scale to reduce processing delays and increase speed. Finally, MSFN is added before the Head layer to aggregate multi-scale features and enhance nonlinear information conversion. Through the above improvements, not only the problem of small irregular defects being difficult to detect is solved, but also the operating speed of the system is improved, and the stability during the detection process is enhanced, avoiding false detections caused by a variety of defect types and complex background patterns.

[0040] Specifically, the input feature map is extracted by the convolution module and CR module in the Backbone layer, and then the data enters the C-DSC module. The module changes the x-axis and y-axis directions respectively. After multiple iterations, the target point position is marked and passed to the CBAM attention mechanism module. This operation can not only improve the detection of small and complex defects, but also does not increase the calculation too much. Then this branch is passed to the AIFI module, and AIFI processes the three matrices after flattening to enhance the expression ability of the model. After feature extraction, it finally passes through the MSFN network and performs another convolution operation in the MSFN network to integrate feature maps of different dimensions, which is used to improve the robustness and generalization ability of the model in different scenarios and conditions. In the auxiliary branch, the multi-scale feature map extracted from the backbone network is processed and combined with the feature map of the backbone network to form a richer feature representation, improving the overall detection and recognition performance.

[0041] like Figure 1 As shown, the method for identifying small and complex defects of cloth in this embodiment includes the following steps:

[0042] Step 1. First, preprocess the collected cloth defect images and form a data set for model training.

[0043] First, the collected cloth defect images are preprocessed, the input images are adjusted to a fixed size (e.g., 640×640×3), and the data set is enhanced (such as random cropping, flipping, rotation, etc.) to expand the data set.

[0044] After the dataset was expanded, the defective parts of the preprocessed images were manually annotated according to the GB / T 24250-2009 "Terminology for Description of Defects in Woven Fabrics", which were defined as 12 common types of defects, including holes, stains, three-wires, knots, hanging wefts, hundred feet, fuzz, hanging warps, color difference, wear marks, double dimensions, and bad weft yarns.

[0045] The cloth defect image dataset with label annotation is divided into a training set, a validation set and a test set for the following model training and testing. The distribution ratio of the training set, the validation set and the test set is, for example, 6:2:2.

[0046] Step 2. Build a cloth small and complex defect recognition model based on the improved YOLOv9 model, such as Figure 2 As shown, the improved YOLOv9 model includes two parts: a main branch and an auxiliary branch.

[0047] The main branch includes the backbone network Backbone, the neck network Neck and the detection head network Head. The auxiliary branch is used to enhance the training during the data set training. In the model inference stage, the auxiliary branch can be deleted.

[0048] The improved main branch structure of the YOLOv9 model is based on the original YOLOv9 model. The improved serpentine dynamic convolution module C-DSC is introduced into the backbone network of the main branch structure to adaptively and dynamically extract defect points to improve the sensitivity to the target shape and boundary and enhance the defect detection of complex and slender styles. Secondly, the AIFI module in the RT-DETR model is used to replace the SPPELAN module in the YOLOv9 model to fuse features of the same scale to reduce processing delays and increase speed. Finally, a multi-scale feedforward network MSFN is added before the detection head network to aggregate multi-scale features and enhance nonlinear information conversion. Through the above improvements, not only the problem of small irregular defects being difficult to detect is solved, but also the system operation speed is improved, the stability of the detection process is enhanced, and the false detection caused by the various types of defects and complex background patterns is reduced.

[0049] like Figure 3 As shown, the main branch structure includes a trunk network, a neck network and a detection head network.

[0050] In the backbone network, the CR module is a combination of the convolution module and the RepNCSPELAN4 module. It extracts features from the input image through multiple downsampling convolutions to achieve full feature extraction.

[0051] Taking a 3×640×640 image as input, it first passes through a convolution with 64 convolution kernels and a stride of 2 to preliminarily extract the features of the cloth. The convolution operation reduces the size of the feature map and increases the number of channels to obtain a feature map with an output size of 64×320×320. The convolution module is a combination of a convolutional layer, batch normalization, and an activation function (Leaky ReLU). The RepNCSPERLAN4 layer combined with the convolution block is a residual module of a convolutional layer and a jump connection. Then it passes through a CR module, which contains a convolution with an output channel number of 256 and a stride of 2, and continues to downsample to obtain a feature map of 256×160×160. The feature map can be improved by passing through the RepNCSPELAN4 module. The convolution operation after multiple CR modules is to repeat the process of downsampling and feature extraction at different resolutions, which is used to extract higher-level features. By repeating the above steps many times, the detection range of the sample is continuously refined. Therefore, the size of the sample image will continue to decrease in length and width, and finally become 80*80*Y, 40*40*Y, and 20*20*Y, where Y represents the number of channels, and the number of channels continues to increase. The RepNCSPELAN4 module combines the reparameterization technology of RepVGG, the branch structure of CSPNet, and the multi-layer convolution method of ELAN to reduce the amount of calculation and memory usage, and further improve the efficiency of reasoning. Through the above multiple sampling operations of different convolutions, feature maps of different sizes such as 512×80×80, 512×40×40, and 512×20×20 can be obtained. These feature maps of different sizes are further transmitted to the neck network for multi-scale feature splicing and upsampling. By fusing features of different scales, the model can be more robust when dealing with scenes with complex backgrounds and diverse targets. Small objects may be more easily captured in lower-level features, while large objects may be more obvious in higher-level features. By splicing these features together, information at different levels can be comprehensively utilized to improve the accuracy of overall detection and create richer and more expressive feature maps. In the detection head network, the multi-scale features output by the neck network are mainly mapped to specific target detection tasks. It performs target classification and positioning through a series of convolution operations, generates the target's category probability, location coordinates, and confidence score, and removes overlapping detection frames through methods such as non-maximum suppression (NMS) to retain the frame with the highest confidence. Finally, the Head layer outputs formatted detection results to ensure that the model can efficiently and accurately complete target detection tasks in various application scenarios.

[0052] Specifically, the backbone network includes a convolution module, a CR module and an improved snake-shaped dynamic convolution module C-DSC; there are four CR modules, which are defined as the first CR module, the second CR module, the third CR module and the fourth CR module respectively.

[0053] Each CR module is composed of a convolution module and a RepNCSPELAN4 module. It extracts features from the input image through multiple downsampling convolutions, thereby achieving full feature extraction.

[0054] The processing flow of the backbone network is as follows: the signal first passes through a convolution module for preliminary feature extraction and increases the number of sample channels, and then passes through the first, second, third, and fourth CR modules in sequence for feature extraction, and finally passes through the improved snake-shaped dynamic convolution module C-DSC for adaptive dynamic extraction of defect points.

[0055] The neck network includes an AIFI module, an upsampling module, a splicing module, a RepNCSPELAN4 module, and a CR module; there are two upsampling modules, which are defined as the first upsampling module and the second upsampling module respectively.

[0056] There are four feature splicing modules, which are defined as the first, second, third and fourth feature splicing modules respectively.

[0057] There are two CR modules, which are defined as the fifth and sixth CR modules respectively.

[0058] There are two RepNCSPELAN4 modules, which are defined as the first and second RepNCSPELAN4 modules respectively.

[0059] The processing flow of the neck network is as follows:

[0060] The output of the backbone network enters the AIFI module to enhance the interaction capability of features within the same scale and between scales, and reduce the attention operation between features of multiple scales; the output of the AIFI module is upsampled through the first upsampling module.

[0061] The upsampling result and the output of the fourth CR module enter the first feature splicing module for splicing, and the spliced ​​features pass through the first RepNCSPELAN4 module and the second upsampling module in sequence.

[0062] The output of the second upsampling module and the output of the third CR module enter the second feature splicing module for splicing, and the spliced ​​features enter the fifth CR module.

[0063] The output of the fifth CR module and the output of the first RepNCSPELAN4 module enter the third feature splicing module for splicing, and the spliced ​​features enter the sixth CR module.

[0064] The output of the sixth CR module and the output of the AIFI module enter the fourth feature splicing module for feature splicing, and the spliced ​​features enter the second RepNCSPELAN4 module.

[0065] There are three detection heads in the detection head network, the fifth CR module is connected to a detection head of the detection head network through an MSFN module, the sixth CR module is connected to a detection head of the detection head network through an MSFN module, and the second RepNCSPELAN4 module is connected to a detection head of the detection head network through an MSFN module.

[0066] The above process is to perform multi-level feature fusion on feature maps of different scales through upsampling and splicing operations. The final feature map passes through multiple detection heads, performs object detection at different scales, and outputs bounding boxes and category prediction results.

[0067] like Figure 4 and Figure 5 As shown, the general processing ideas of the auxiliary branch are as follows:

[0068] After the image is input, preliminary feature extraction is performed, and the extracted features and multiple scale features of the shallow network of the main branch are combined through the CBFuse layer (i.e. Figure 5 The multi-scale feature fusion shown in the figure is fused to form a fused feature; after the CBLinear layer adjusts the channel, it is integrated by the CBFuse layer to ensure the unity of dimension and channel; the fused feature is divided into three branches for multi-scale prediction analysis, and the loss is calculated and fed back in the Head layer compared with the label to enhance the shallow feature gradient information and reduce information loss; this auxiliary branch is not used in the reasoning of the cloth small and complex defect recognition model to maintain computational efficiency.

[0069] Specifically, if Figure 5 As shown, the auxiliary branch includes two convolution modules, three multi-scale feature fusion CBFuse layers, three CBLinear layers, three CR modules, three MSFN modules and one RepNCSPELAN4 module.

[0070] Three CBFuse layers are defined as a first CBFuse layer, a second CBFuse layer, and a third CBFuse layer.

[0071] Define three CBLinear layers as the first CBLinear layer, the second CBLinear layer, and the third CBLinear layer.

[0072] Three CR modules are defined as the seventh, eighth and ninth CR modules.

[0073] The processing flow of the auxiliary branch is as follows:

[0074] After the picture is input, it first passes through the convolution module and the seventh CR module for preliminary feature extraction, and then passes through another convolution module and is input to the first CBFuse layer for feature fusion. At the same time, the second CR module adjusts the channel features through the first CBLinear layer, the third CR module adjusts the channel features through the second CBLinear layer, and the fourth CR module adjusts the channel features through the third CBLinear layer, and then inputs them to the first CBFuse layer.

[0075] The features of the first CBFuse layer are input for multi-scale feature fusion, and the fusion results enter the eighth CR module.

[0076] The eighth CR module is connected to a detection head through an MSFN module.

[0077] The output features of the eighth CR module and the features of the third CR module after adjusting the channel through the second CBLinear layer enter the second CBFuse layer, and multi-scale feature fusion is performed in the second CBFuse layer. The fusion result enters the ninth CR module.

[0078] The ninth CR module is connected to a detection head through an MSFN module. The output features of the ninth CR module and the features of the fourth CR module after adjusting the channel through the third CBLinear layer enter the third CBFuse layer, and multi-scale feature fusion is performed in the third CBFuse layer. The fusion result enters the RepNCSPELAN4 module.

[0079] The RepNCSPELAN4 module is connected to a detection head through an MSFN module.

[0080] like Figure 6 As shown in the figure, the improved snake-shaped dynamic convolution module C-DSC deformable convolution allows the network to learn geometric changes completely freely, so that the perception area can roam, especially on small tubular structures. In view of this, the free learning process is supplemented by constraints to enhance the perception of tubular structures in a targeted manner.

[0081] In order to meet the challenges of complex and changeable defect morphology detection, the network structure of the improved snake-shaped dynamic convolution module C-DSC is obtained by introducing the CBAM attention mechanism on the basis of the original DSC convolution.

[0082] CBAM consists of two independent sub-modules, namely the Channel Attention Module (CAM) and the Spatial Attention Module (SAM), which perform channel and spatial attention respectively. It not only saves parameters and computing power, but also ensures that it can be integrated into the existing network architecture as a plug-and-play module.

[0083] The feature map of the input C-DSC is first input into the DSC convolution, and the horizontal and vertical features are extracted respectively. After the extraction is completed, the features are fused, and then down-sampled to make the image fit the size of the display area, and a thumbnail of the corresponding image is generated. The result after multiple iterations is passed to the next process; the next step of the DSC convolution receives the information and extracts features again. After extraction, the horizontal and vertical features are fused, and up-sampled to enlarge the original image and restore the original size; the feature information after multiple iterations is passed to the CBAM module for spatial attention and channel attention operations respectively; then it is compared with the label, and the TCLoss is fed back upward to finally obtain the output result.

[0084] First, the sample image is input. Since the defect point sample image is two-dimensional, its defect position and shape are irregular, so it first passes through the snake convolution module DSC, which contains a 9*9 convolution kernel. Each grid position K in the convolution kernel K i±c The selection of is a cumulative process, which is used to extract the general features of the defect points and make a rough judgment on the location of the features, in preparation for the accurate positioning of the defects later. i Initially, the position of the grid away from the center depends on the position of the previous grid: K i+1 Relative to K i An offset Δ={δ|δ∈[-1,1]} is added.

[0085] Therefore, the offset needs to be accumulated Σ to ensure that the convolution kernel conforms to the linear morphology structure.

[0086] The change in the x-axis direction is:

[0087]

[0088] The y-axis changes in direction:

[0089]

[0090] Among them, K i Represents the direction of change of the x-axis; K j represents the direction of change of the y-axis; x represents the horizontal coordinate; y represents the vertical coordinate; i, j, c represent constants; Δx represents the x-axis increment; Δy represents the y-axis increment.

[0091] Since the coordinates are usually integers, there may be defects in every pixel of the feature map, and the size of the defects is small, so the offset Δ between adjacent distances will be a decimal. Bilinear interpolation considers the values ​​of the four points around the point to be interpolated and performs weighted averaging based on the values ​​of these four points to obtain an estimated value of the point to be interpolated.

[0092] This method can better reflect the real changes of data than simple nearest neighbor interpolation, so it has higher interpolation accuracy, so bilinear interpolation is used here. Bilinear interpolation is expressed as:

[0093]

[0094] Among them, K represents the decimal position of the equation, K′ lists all integer spatial positions, and B is the bilinear interpolation kernel, which is decomposed into two one-dimensional kernels, namely: B(K,K′)=b(K x ,K x ′)·b(K y ,K y ′).

[0095] Through the above steps, the x-axis and y-axis directions are changed respectively, extending from the inside of the defect to the outside, and the lateral and longitudinal positions are continuously adjusted. Then, the spatial attention mechanism and channel attention mechanism in the CBAM module are used to improve the performance of the network and enhance the accuracy of sample feature recognition.

[0096] Finally, the output value is compared with the label to obtain the loss value TCLoss. The reverse transmission is used to adjust the training strategy through multiple iterations to mark the target point position.

[0097] exist Figure 6 In the feature fusion part mentioned in , multi-view feature fusion is adopted to ensure that important information is retained from different global forms by supplementing the attention to features from multiple angles during feature fusion.

[0098] like Figure 7 As shown in the figure, the processing flow of multi-view feature fusion is as follows:

[0099] The samples after feature extraction have multi-layer properties. The different layers of the unified feature map are named as: feature Figure 1 ,feature Figure 2 ,feature Figure 3 ..., each layer of features is divided into horizontal features and vertical features. Figure 1 Randomly discard horizontal features or vertical features, and then Figure 2 Randomly discard horizontal features or vertical features until all layers are discarded, then fuse the processed features of each layer and finally output the feature map.

[0100] like Figure 8 The structural diagram of the CBAM module is shown, where × represents element multiplication. It includes two modules: the first is the channel attention module (CAM); the second is the spatial attention module (SpartialAttention Module, SAM); and two multipliers, defined as the first multiplier and the second multiplier.

[0101] The specific implementation process is as follows: first, the input features enter the channel attention module, and after the output, they enter the first multiplier for element-wise multiplication with the original input feature map and then enter the spatial attention module. The output of the spatial attention module and the output of the first multiplier enter the second multiplier for feature element multiplication, and finally the feature map is output.

[0102] like Fig. 9 As shown, the channel attention module will transform the feature map F H×W×C After global max pooling and global average pooling based on width and height, two 1×1×C feature maps are obtained. Then, they are sent to a two-layer neural network (MLP). The number of neurons in the first layer is C / r, where r is the reduction rate and the activation function is Relu. The number of neurons in the second layer is C. The two layers of the neural network are shared. Then, the features output by MLP are added based on element-wise operation, and then activated by sigmoid (activation function) to generate the final channel attention features, namely M c .

[0103] In short, channel attention is calculated as follows:

[0104]

[0105] Where σ represents the S-type function; c represents channel; F avg represents the average pooling feature; F max represents the maximum pooling feature; F represents the feature map; W0 and W1 represent weights respectively. The MLP weights W0 and W1 are shared by two inputs, and the ReLU activation function is followed by w0. Fig. 9 In the table, “+” represents feature addition.

[0106] like Fig.10As shown in the figure, the feature map output by the channel attention module is used as the input feature map of this module. First, after a global maximum pooling and global average pooling based on the channel, two H×W×1 feature maps are obtained, and then the two feature maps are channel-joined. Then, after a 7×7 convolution operation, the dimension is reduced to 1 channel, that is, H×W×1. Then, after sigmoid operation, the spatial attention feature is generated, that is, M s , and then splice CAM and SAM together front and back to output the final feature map.

[0107] In brief, spatial attention is calculated as follows:

[0108]

[0109] Where σ represents the S-type function; s represents space; F avg represents the average pooling feature; F max represents the maximum pooling feature; f 7 ×7 Represents a convolution operation with a filter size of 7×7.

[0110] The present invention uses the AIFI module in the RT-DETR model to replace the SPPELAN enhancement model in YOLOv9 to enhance the processing ability of local and global information. The RT-DETR model is a deep learning model for target detection, which is based on the Transformer architecture, and the composition of the AIFI module is similar to that of the Transformer Encoder, and is particularly suitable for real-time processing of sequence data. The original SPPELAN module mainly calculates the correlation between feature maps and assigns different weights to different regions, thereby achieving local attention enhancement. SPPELAN adopts a multi-task learning method to optimize multiple tasks such as target detection, classification and positioning at the same time, so that the model can better adapt to various complex scenes, but it may increase the computational complexity when processing large-scale images or high-resolution images due to the need to generate feature maps of multiple scales and perform pooling operations. This may lead to a decrease in processing speed, affecting the performance of real-time applications or large-scale data processing scenarios, and the AIFI module can reduce unnecessary data processing processes, optimize the algorithm structure, and reasonably allocate data processing resources while ensuring the accuracy of calculations, further improve the computational efficiency, and reduce the running time, so that the model can achieve the purpose of lightweight.

[0111] like Fig.11 As shown in FIG. 1 , “×” represents the dot product between matrices. In this embodiment, the AIFI module receives the feature map from the improved serpentine dynamic convolution module C-DSC as input, such as Figure 2 As shown, the AIFI module performs the following processing:

[0112] The input feature map is first flattened into a sequence to form an image feature sequence containing all pixels; the flattening operation will maintain the spatial information of the feature and convert it into a form suitable for self-attention mechanism processing, which is then converted into three matrices: query Q, key K, and value V; then, by calculating the dot product between Q and K, the attention weight matrix is ​​obtained; then, the attention weight matrix is ​​applied to V to produce a weighted feature representation; finally, after the self-attention mechanism is processed, the obtained feature sequence is restored to the same shape as the input. It can be expressed by the following formula:

[0113] Q = K = V = Flatten(input);

[0114] Output=Reshape(Attn(Q,K,V));

[0115] Where Q represents the query matrix; K represents the key matrix; V represents the value matrix; Flatten represents the flattening operation; and Reshape represents the matrix transformation.

[0116] A multi-scale feedforward network (MSFN) is added to the feature layers of different scales before the detection head, i.e., before the Head layer, to aggregate multi-scale features and enhance nonlinear information conversion. In previous experiments on adding multi-scale information to image denoising tasks, it has been very effective. In addition, the MSFN module has the convenient plug-and-play feature when using the model.

[0117] The processing flow of the multi-scale feedforward network MSFN is as follows:

[0118] The input features are processed through two parallel paths, and the nonlinear transformation is enhanced by the element-wise product of the features of the two paths; in the upper path, deep convolution is used for feature extraction; in the lower path, multi-scale dilated convolution is used for multi-scale feature extraction, using two 3×3 dilated convolutions with dilation rates of 2 and 3 respectively.

[0119] Given an input tensor Y∈R H*W*C , then MSFN is defined as:

[0120]

[0121] Y out =W 1×1 ⊙Gating(Y);

[0122] Among them, H, W, and C represent the sample height, width, and number of channels respectively;

[0123] Gating(Y) represents the gating mechanism, ⊙ represents element-wise multiplication, φ represents the GELU activation function, represents a 3×3 dilated convolution with a dilation rate of 2. Represents 3×3 dilated convolution with a dilation rate of 3;

[0124] W 3×3×3 represents 3×3×3 convolution; W 1×1 (Y) represents a 1×1 convolution on feature Y; Y out Represents output.

[0125] Fig.12 In , “+” represents the merging of two channels, and “×” represents the product processing of the merged process and the first path. Two 1×1 convolutions are used to expand the feature channel, and the expansion ratio γ=2.

[0126] When generating the detection box, the positive and negative sample matching strategy is also used. TAL (Task Align Learning) is task alignment learning, which is to unify the classification and regression anchors by building an "alignment metric", so as to finally obtain a higher scoring classification box and a more accurate positioning box coefficient during reasoning.

[0127] The key to TAL is to construct an alignment metric. The calculation of the alignment metric includes two parts. The first is to construct a classification score matrix (tensor) s for each label corresponding to each region, and u is the IoU between the predicted target box and the label corresponding to each anchor point.

[0128] The calculation of IoU is as follows:

[0129]

[0130] Among them, A represents the real box and B represents the predicted box.

[0131] Then, the new anchor alignment index t can be obtained through the following calculation.

[0132] t=s a *u b ;

[0133] Where t represents the anchor alignment index, a and b are parameters used to control the impact of the two tasks in the anchor alignment metric. The IOU information depends on the true box and the predicted box information, and the two are combined to obtain the alignment metric matrix.

[0134] Step 3. Use the data set in step 1 to train a cloth small and complex defect recognition model based on the improved YOLOv9 model. During the training process of the cloth small and complex defect recognition model, an auxiliary branch is introduced to assist in training.

[0135] Before model training, set the training parameters, set the number of iterations epochs to 300, the number of batch training samples batch_size to 16, Works to 8, and Imgsz to 640. Set the learning rate optimizer to the adam optimizer.

[0136] During model training, the loss function is as follows:

[0137] Distribution Focal Loss (DFL), the formula is as follows:

[0138]

[0139] Among them, y i represents the actual value of the model, Represents the model prediction value, and the Focal Loss function is defined as:

[0140]

[0141] Among them, α t It represents the balancing factor, which is used to adjust the influence between positive and negative samples; γ is the focus factor, which is used as the weight to adjust the difficult and easy samples.

[0142] Complete Intersection over Union Loss (CIoU) is shown below:

[0143]

[0144] where ρ(b,b gt ) is the Euclidean distance between the center points of the predicted box and the real box; c is the diagonal length of the minimum bounding box of the predicted box and the real box; v represents the parameter for measuring the consistency of the aspect ratio of the predicted box and the real box; b represents the position of the predicted box; b gt Indicates the real box position.

[0145] α is the weight balancing factor, and its formula is as follows:

[0146]

[0147] Then the expression of the total loss function of the model is: DFLoss+CIoU Loss.

[0148] Step 4. After the cloth defect image to be identified has been preprocessed in step 1, it is input into the cloth small and complex defect recognition model trained in step 3, and the target recognition result is output, including the position of the bounding box and the defect category, so as to identify the small and complex cloth defects in the cloth defect image.

[0149] Compared with the original YOLOv9 model, the present invention can improve the recognition efficiency of complex and irregular small defects through the proposed C-DSC network. At the same time, since YOLOv9 introduces an innovative basic network architecture, this design enhances the efficiency of feature extraction while maintaining efficient calculation. By increasing the depth and complexity of the network, YOLOv9 can more accurately capture the background information in the image, further improve its accuracy in target recognition and positioning, and can be applied to real-time cloth defect scene detection. In addition, the AIFI module focuses on achieving richer feature fusion between features of the same scale, which not only improves the efficiency and effectiveness of feature extraction, but also improves detection accuracy by capturing more fine-grained information. The design of the AIFI module maintains computational efficiency while reducing the computational consumption of attention operations between multi-scale features, making the model more flexible and efficient when processing complex scenes. In addition, the present invention also introduces MSFN before the detection head network, and significantly enhances the overall feature extraction capability of the network by fusing features of different scales. MSFN can also be flexibly inserted into various model networks. Its multi-scale feature extraction method enables the network to capture global and local information at the same time, improves the model's ability to understand complex images, and can ensure performance improvement while minimizing the consumption of computing resources. Through the above improvements, the cloth fine and complex defect recognition model built based on the improved YOLOv9 model of the present invention has greatly improved the recognition accuracy and efficiency of the present invention for fine and complex defects in cloth defect images.

[0150] Example 2

[0151] This embodiment 2 describes a computer device, which includes a memory and one or more processors. The memory stores executable code. When the processor executes the executable code, the steps of the method for identifying small and complex defects of cloth in the above embodiment 1 are implemented.

[0152] In this embodiment, the computer device is any device or apparatus with data processing capability, which will not be described in detail here.

[0153] Example 3

[0154] This embodiment 3 describes a computer-readable storage medium on which a program is stored. When the program is executed by a processor, it is used to implement the steps of the method for identifying small and complex defects in cloth in the above-mentioned embodiment 1.

[0155] The computer-readable storage medium can be an internal storage unit of any device or apparatus with data processing capabilities, such as a hard disk or memory, or an external storage device of any device with data processing capabilities, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), an SD card, a flash card, etc. equipped on the device.

[0156] Of course, the above description is only a preferred embodiment of the present invention, and the present invention is not limited to the above embodiments. It should be noted that all equivalent substitutions and obvious deformation forms made by any technician familiar with the field under the guidance of this specification fall within the essential scope of this specification and should be protected by the present invention.

Claims

1. A method for identifying small and complex defects in cloth, characterized in that: The steps include: Step 1. First, preprocess the collected cloth defect images and form a data set for model training; Step 2. Build a cloth small and complex defect recognition model based on the improved YOLOv9 model; the main branch structure of the improved YOLOv9 model is obtained by making the following improvements on the basis of the original YOLOv9 model: An improved serpentine dynamic convolution module C-DSC is introduced into the backbone network of the main branch structure to adaptively and dynamically extract defect points, which is used to improve the sensitivity to complex defect shapes and boundaries and enhance the defect detection capability for slender and complex styles. The AIFI module in the RT-DETR model is used to replace the SPPELAN module in the YOLOv9 model to enhance the model's ability to process local and global information and enhance the interaction between features at the same scale to reduce processing delays and increase speed. A multi-scale feedforward network MSFN is added before the Head layer to aggregate multi-scale features and enhance nonlinear information conversion. Step 3. Use the data set in step 1 to train a cloth small and complex defect recognition model based on the improved YOLOv9 model. During the training process of the cloth small and complex defect recognition model, an auxiliary branch is introduced to assist in the training; Step 4. After the cloth defect image to be identified has been preprocessed in step 1, it is input into the cloth small and complex defect recognition model trained in step 3, and the target recognition result is finally output, including the position and category of the bounding box, so as to identify the small and complex cloth defects in the cloth defect image.

2. The method for identifying small and complex defects in cloth according to claim 1, characterized in that: The improved YOLOv9 model has a main branch structure including a trunk network, a neck network and a detection head network; The backbone network includes a convolution module, a CR module, and an improved snake-shaped dynamic convolution module C-DSC; there are four CR modules, which are defined as the first CR module, the second CR module, the third CR module, and the fourth CR module respectively; Each CR module is composed of a convolution module and a RepNCSPELAN4 module. It extracts features from the input image through multiple downsampling convolutions, thereby achieving full feature extraction. The processing flow of the backbone network is as follows: the input feature map first passes through a convolution module for convolution downsampling operation to reduce the image size and increase the number of channels of the image, and then passes through the first, second, third, and fourth CR modules in sequence for feature extraction, and finally passes through the improved snake-shaped dynamic convolution module C-DSC to perform adaptive dynamic extraction of defect points.

3. The method for identifying small and complex defects of cloth according to claim 2, characterized in that: The neck network includes an AIFI module, an upsampling module, a splicing module, a RepNCSPELAN4 module, and a CR module; wherein there are two upsampling modules, which are defined as a first upsampling module and a second upsampling module respectively; There are four feature splicing modules, which are defined as the first, second, third, and fourth feature splicing modules respectively; There are two CR modules, which are defined as the fifth and sixth CR modules respectively; There are two RepNCSPELAN4 modules, which are defined as the first and second RepNCSPELAN4 modules; The processing flow of the neck network is as follows: The output of the backbone network enters the AIFI module; the output of the AIFI module is upsampled by the first upsampling module, and the upsampling result and the output of the fourth CR module enter the first feature splicing module for splicing; The output features of the first feature concatenation module are sequentially passed through the first RepNCSPELAN4 module and the second upsampling module; The output of the second upsampling module and the output of the third CR module enter the second feature splicing module for splicing, and the output features of the second feature splicing module enter the fifth CR module; The output of the fifth CR module and the output of the first RepNCSPELAN4 enter the third feature splicing module for splicing, and the output features of the third feature splicing module enter the sixth CR module; The output of the sixth CR module and the output of the AIFI module enter the fourth feature splicing module for feature splicing, and the output features of the fourth feature splicing module enter the second RepNCSPELAN4 module; The detection head network includes three detection heads; the fifth CR module is connected to a detection head of the detection head network through an MSFN module; the sixth CR module is connected to a detection head of the detection head network through an MSFN module; the second RepNCSPELAN4 module is connected to a detection head of the detection head network through an MSFN module.

4. The method for identifying small and complex defects in cloth according to claim 1, characterized in that: The network structure of the improved snake-shaped dynamic convolution module C-DSC is obtained by introducing the CBAM attention mechanism on the basis of the original DSC convolution. The CBAM includes two independent submodules, namely the channel attention module CAM and the spatial attention module SAM, which perform channel and spatial attention operations respectively. The processing flow of C-DSC is as follows: the input feature image is first input into the DSC convolution, and the horizontal and vertical features are extracted respectively. After the extraction is completed, the features are first fused, and then down-sampled to make the image fit the size of the display area, and a thumbnail of the corresponding image is generated. The result after multiple iterations is passed to the next process; the next step of the DSC convolution receives the information and performs feature extraction again. After extraction, the horizontal and vertical features are fused, and up-sampled to enlarge the original image and restore the original size; the feature information after multiple iterations is passed to the CBAM module for spatial attention and channel attention operations respectively; then it is compared with the label, and the TCLoss is fed back upward to finally obtain the output result.

5. The method for identifying small and complex defects of cloth according to claim 1, characterized in that: The AIFI module receives the feature map from the C-DSC as input and performs the following processing: The input feature map is first flattened into a sequence to form an image feature sequence containing all pixels, where the flattening operation will maintain the spatial information of the feature and convert it into a form suitable for processing by the self-attention mechanism, and then converted into three matrices: query Q, key K, and value V; then, the attention weight matrix is ​​obtained by calculating the dot product between Q and K; then, the attention weight matrix is ​​applied to V to produce a weighted feature representation; finally, after processing by the self-attention mechanism, the obtained feature sequence is restored to a feature of the same shape as the input.

6. The method for identifying small and complex defects in cloth according to claim 1, characterized in that: The processing flow of the multi-scale feedforward network MSFN is as follows: The input features are processed through two parallel paths, and the nonlinear transformation is enhanced by the element-wise product of the features of the two paths; in the upper path, deep convolution is used for feature extraction; in the lower path, multi-scale dilated convolution is used for multi-scale feature extraction, using two 3×3 dilated convolutions with dilation rates of 2 and 3 respectively; If the input tensor Y∈R H*W*C , then MSFN is defined as: Y out =W 1×1 ⊙Gating(Y); Among them, H, W, and C represent the sample height, width, and number of channels respectively; Gating(Y) represents the gating mechanism, ⊙ represents element-wise multiplication, φ represents the GELU activation function, represents a 3×3 dilated convolution with a dilation rate of 2. Represents 3×3 dilated convolution with a dilation rate of 3; W 3×3×3 represents 3×3×3 convolution; W 1×1 (Y) represents a 1×1 convolution on feature Y; Y out Represents output.

7. The method for identifying small and complex defects in cloth according to claim 1, characterized in that: The auxiliary branch includes two convolution modules, three multi-scale feature fusion CBFuse layers, three CBLinear layers, three CR modules, three MSFN modules and one RepNCSPELAN4 module; Define three CBFuse layers: the first CBFuse layer, the second CBFuse layer, and the third CBFuse layer Define three CBLinear layers: the first CBLinear layer, the second CBLinear layer, and the third CBLinear layer; Define three CR modules, namely the seventh, eighth and ninth CR modules; The processing flow of the auxiliary branch is as follows: After the picture is input, it first passes through the convolution module and the seventh CR module for preliminary feature extraction, and then passes through another convolution module and is input to the first CBFuse layer for feature fusion; at the same time, the second CR module adjusts the channel features through the first CBLinear layer, the third CR module adjusts the channel features through the second CBLinear layer, and the fourth CR module adjusts the channel features through the third CBLinear layer, and is input to the first CBFuse layer at the same time; Multiple features of the first CBFuse layer are input for multi-scale feature fusion, and the fusion results enter the eighth CR module; The eighth CR module is connected to a detection head through an MSFN module; The output features of the eighth CR module and the features of the third CR module after adjusting the channel through the second CBLinear layer enter the second CBFuse layer, and multi-scale feature fusion is performed in the second CBFuse layer. The fusion result enters the ninth CR module; The ninth CR module is connected to a detection head through an MSFN module; the output features of the ninth CR module and the features of the fourth CR module are adjusted through the third CBLinear layer, enter the third CBFuse layer, and multi-scale feature fusion is performed in the third CBFuse layer, and the fusion result enters the RepNCSPELAN4 module; The RepNCSPELAN4 module is connected to a detection head through an MSFN module.

8. The method for identifying small and complex defects in cloth according to claim 1, characterized in that: In step 3, the training parameters are set during the training process of the improved YOLOv9 model, the number of iterations epochs is set to 300, the number of batch training samples batch_size is set to 16, Works is set to 8, and Imgsz is set to 640. The learning rate optimizer optimizer is set to the adam optimizer.

9. A computer device comprising a memory and one or more processors; an executable code is stored in the memory; characterized in that: When the processor executes the executable code, it is used to implement the method for identifying small and complex defects of cloth as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a program stored thereon; characterized in that: When the program is executed by a processor, it is used to implement the method for identifying small and complex defects in cloth as described in any one of claims 1 to 8.

Images